A poisoning attack defense method for online learning

Abstract

The present application belongs to the field of information security and machine learning, and provides a poisoning attack defense method for online learning, which is used to solve the problem that all samples, attack time and total number of poisoned samples cannot be predicted when facing online attacks in an edge computing environment. First, the target function value and influence of each sample currently arriving at the edge device are calculated, and both are sorted in ascending order. Then, the total number of possible clean samples is calculated according to a given set of possible poisoning rates, and the samples with high target function value and influence are selected in turn to construct a candidate clean sample set, and the candidate clean sample set with the smallest loss is selected as the trusted clean sample set to train the model. Finally, the process is repeated until the online learning reaches a convergence state. In this process, the sample points with large influence and large target function value are excluded from the training model in an online incremental manner, thereby achieving the purpose of online defense against online poisoning attacks.

Description

Technical Field

[0001] This invention belongs to the fields of information security and machine learning, and relates to a method for defending against online poisoning attacks during online learning. Specifically, it provides a method for defending against poisoning attacks in online learning. Background Technology

[0002] In edge computing environments, data poisoning attacks (DPA) are among the most destructive potential attacks. Attackers can easily impersonate legitimate user terminals to generate malicious data online and attack edge AI models. They can inject poisoned samples to modify the data distribution of training data, thereby training models to meet their specific attack objectives. Such attacks can severely compromise the security and reliability of AI models and pose a serious threat to edge systems. For example, in the field of the Internet of Medical Things (IoMT), an attacker posing as a legitimate medical terminal and uploading only 2% malicious drug dosage data can severely disrupt drug dosage prediction models.

[0003] Currently, most research and applications on poisoning attack defense focus on offline environments. There is still no effective solution for poisoning attacks in online environments. The main reason is that current defense methods have the following shortcomings when facing online attacks: First, because it is impossible to predict all samples and the attack time, it is impossible to build a reliable and clean sample set on which data defense depends, causing the defense method to fail. Second, because the total number of poisoned samples is unknown, it is difficult to formulate an appropriate filtering threshold. This forces offline defense methods to use the costly subset traversal method to ensure the defense effect. However, this overhead is fatal for resource-constrained edge computing environments. Summary of the Invention

[0004] The purpose of this invention is to address the problem of unpredictable total number of samples, attack times, and the total number of poisoned samples (clean samples) in edge computing environments facing online attacks. It provides a poisoning attack defense method for online learning. First, the objective function value of each sample arriving at the edge device is calculated and sorted in ascending order. Simultaneously, the influence of each sample is calculated and sorted in ascending order. Then, based on a given set of possible poisoning rates, the total number of possible clean samples is calculated. Candidate clean sample sets are constructed by sequentially selecting samples with the highest objective function values ​​and highest influence, with each poisoning rate corresponding to one candidate clean sample set. The candidate clean sample set with the minimum loss is selected as the trusted clean sample set, and the model is trained using this trusted clean sample set. Finally, the previous operations are repeated for each time slice of samples arriving at the edge device until the online learning reaches a convergent state. This process incrementally excludes samples with high influence and high objective function values ​​from the training model (purifying the model with samples with lower influence and lower objective function values), thereby achieving online defense against online poisoning attacks.

[0005] To achieve the above objectives, the technical solution adopted by the present invention is as follows:

[0006] A method for defending against poisoning attacks in online learning includes the following steps:

[0007] S1. Initialize the parameters of the defense method;

[0008] S2. Set a set of possible poisoning rates;

[0009] S3. Calculate the objective function value for each sample in the current sample set and sort them in ascending order;

[0010] S4. Calculate the influence of each sample in the current sample set and sort them in ascending order;

[0011] S5. For each poisoning rate in the set of poisoning rates, calculate the total number of clean samples under that poisoning rate, and let num represent the total number of clean samples under the j-th poisoning rate. j ;

[0012] S6. At each poisoning rate, select the num that has both the objective function value and influence in the sequence. j Construct a candidate clean sample set from 100 samples. Calculate the loss value for each candidate clean sample set, and select the one with the smallest loss value as the reliable clean sample set. And record the corresponding poisoning rate as the optimal poisoning rate P. t ;

[0013] S7. Train the model using a reliable and clean sample set and update the model parameters;

[0014] S8. Update the time slice and repeat S2 to S7 until the model convergence condition is met.

[0015] Furthermore, in S1, random initialization is used to initialize the model parameters to θ0, and the current time slice t is initialized to 1, and the optimal poisoning rate P is initialized. t It is 0.

[0016] Furthermore, in S2, the combination of poisoning rates is represented by I, specifically as follows:

[0017]

[0018] Where γ is the preset poisoning rate, r is the poisoning rate interval parameter, and I j This represents the poisoning rate for the j-th time.

[0019] Furthermore, in S3, the objective function value is expressed as:

[0020]

[0021]

[0022]

[0023] in, The objective function of the model is... Let θ be the loss function of the model, t represent the current time slice, and θ be the loss function of the model. t-1 The model parameters up to the current time slice are λ and Ω(θ). t-1 ) represents the regularization coefficient and the regularization term; Represents the current sample set, cache1 and cache t Let x represent the 1st and tth time slices respectively, and b be the number of samples in each time slice; (x i y i ) represents the i-th sample point, x i With y i Let represent the feature vector and label value corresponding to the i-th sample point, respectively.

[0024] Furthermore, in S4, influence is expressed as:

[0025]

[0026]

[0027]

[0028]

[0029]

[0030] Among them, C i Let (x) represent the influence of the i-th sample point. i y i ) represents the i-th sample point, x i With y i Let represent the feature vector and label value corresponding to the i-th sample point, respectively, and k be the feature dimension of the sample; This represents the feature matrix corresponding to the current sample set, cache1 and cache t θ represents the 1st and tth time slices, respectively; t-1 θ represents the model parameters up to the current time slice. (i) This indicates the use of samples (x) i y i The model parameters θ are obtained by training the model using the current sample set. (-i) This indicates the use of samples (x) without inclusion. i y i The model parameters are obtained by training the model using the current sample set.

[0031] Furthermore, in S5, regarding the poisoning rate I j The total number of clean samples is:

[0032] num j = t × b × (1 - I j ),

[0033] Where, num j Indicates the poisoning rate I j The total number of clean samples, where t represents the current time slice and b represents the number of samples per time slice.

[0034] Furthermore, in S6, the loss value is expressed as:

[0035]

[0036]

[0037] in, Let j represent the candidate clean sample set under the j-th poisoning rate. Indicates a clean sample set of candidates The loss value; (x i y i ) represents the i-th sample point, x i With y i Let θ represent the feature vector and label value corresponding to the i-th sample point, respectively. t-1These are the model parameters up to the current time slice.

[0038] Based on the above technical solution, the beneficial effects of the present invention are as follows:

[0039] This invention provides a poisoning attack defense method for online learning. It fully considers the characteristics of sample influence and objective function value. During online learning, it sequentially detects low-influence and low-objective-function-value sample points arriving at the edge device in each time slice, constructing a reliable clean sample set. The model is then trained using this reliable clean sample set to achieve the defense against poisoning attacks. Furthermore, it has the following advantages:

[0040] 1) It solves the problem that it is impossible to predict all sample points and attack times during online learning, and dynamically constructs a reliable and clean sample set based on the changes of samples over time;

[0041] 2) It solves the problem of unpredictable poisoning rates during online learning by setting a set of possible poisoning rates and finding the optimal poisoning rate from them;

[0042] 3) By using a dual heuristic screening mechanism based on minimizing the objective function value and minimizing the influence, the purity of the trustworthy clean sample set is continuously improved iteratively, thereby reducing overhead and improving the defense effect. Attached Figure Description

[0043] Figure 1 This is a schematic diagram of the training sample data stream in an online regression task.

[0044] Figure 2 This is a schematic diagram illustrating the principle of poisoning attacks in online regression testing.

[0045] Figure 3 This is a schematic diagram illustrating the principle of the poisoning attack defense method for online learning provided by the present invention.

[0046] Figure 4 This is a flowchart illustrating the poisoning attack defense method for online learning provided by the present invention.

[0047] Figure 5 This is a classification diagram of training samples in an embodiment of the present invention. Detailed Implementation

[0048] To make the objectives, technical solutions, and beneficial effects of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments.

[0049] This embodiment provides a poisoning attack defense method for online learning (hereinafter referred to as: online defense method). In order to simplify the problem description and better explain the implementation process of the method, the application scenario is the online poisoning attack defense process for basic online regression tasks in the field of medical Internet of Things. Common online regression tasks in this field include drug dosage prediction and real-time health prediction. This embodiment takes the prediction of a certain drug dosage (e.g., warfarin) as the online regression task.

[0050] In this embodiment, the basic online regression model is defined as follows:

[0051]

[0052]

[0053] Where y represents the label vector and h represents the model function. y represents the feature matrix corresponding to the training sample data stream up to the current time slice t, and θ represents the parameter vector of the model; i Let x represent the i-th label value in the label vector. ij θ represents the j-th eigenvalue in the eigenvector corresponding to sample point i. j Let i = 1, 2, ..., n, j = 1, 2, ..., k, n be the total number of samples up to the current time slice t, and k be the feature dimension of the sample.

[0054] like Figure 1 As shown, cache1~cache t This represents the 1st to tth time slices that arrive at the model and begin training as time progresses, with each time slice containing b samples, i.e.: x i and y i Let x represent the feature vector and label value corresponding to sample point i, respectively. i Corresponding matrix The i-th row, y i The i-th label value corresponding to the label vector y.

[0055] In this embodiment, the feature vector includes: demographic information, indications of warfarin use, individual VKORC1 and CYP2C9 genotype data, and other drug usages affected by related VKORC1 and CYP2C9 polymorphisms, to predict the treatment dose labeled as warfarin.

[0056] A normal online regression task utilizes the cache for each time slice sequentially. tThe process of fitting the model function h with training sample data to solve for the optimal parameters θ is called online poisoning attack. In this process, some samples are contaminated through label inversion, gradient ascent, etc., ultimately achieving the goal of contaminating the model parameters. For example... Figure 2 The diagram illustrates the principle of poisoning attacks in online regression tasks. Solid squares represent normal training samples, while rounded squares, diamonds, and solid arrows together represent the online learning process. Each time slice is cached. t Upon arrival, the model is trained and convergence criteria are determined until convergence occurs. The model parameters θ at convergence are... * These are the optimal parameters; after the poisoning attack is complete, the dashed square represents the poisoned sample, and the dashed arrow represents the contamination process, ultimately resulting in the model parameters being contaminated. (Indicated by a dotted rounded square frame).

[0057] like Figure 3 The diagram shows the principle of the online defense method proposed in this invention. Solid and dashed squares represent normal and poisoned samples, respectively. Rounded squares, diamonds, and solid arrows also represent the online learning process. Trusted Clean Set1~Trusted Clean Set t Represents a dynamically trusted clean sample set; for TrustedClean Set t This dataset is based on samples up to the current time slice t. Construct (indicated by hollow arrow), this dataset is represented in subsequent formulas as S1 to S8 represent the steps of the online defense method. The online defense method executes starting from the first time slice. S1 to S5 sequentially initialize the defense algorithm parameters, set possible poisoning rates, sort the samples according to the objective function value, sort the samples according to the influence, and calculate the total number of clean samples. S6: For each poisoning rate, select samples with smaller objective function values ​​and influence values ​​from the current samples to construct a candidate clean sample set (denoted as...). S2: Select the sample set with the smallest loss function value from the set and record the current optimal poisoning rate; S7: Update the model parameters using the reliable clean sample set; S8: Repeat steps S2 to S7 until the model convergence condition is met at time slice t, at which point the model parameters θ are obtained. * This is the output of the online defense method.

[0058] Based on the above principles, the online defense method provided in this embodiment is as follows: Figure 4 As shown, it includes the following steps:

[0059] S1. Initialize the parameters of the defense method;

[0060] The model parameters are initialized to θ0 using random initialization, and the current time slice t is initialized to 1, with the optimal poisoning rate P. t The value is 0; because the sample size of the current time slice cannot be predicted. Whether the sample contains poisoned samples or not, the model cannot be directly trained using this sample dataset to obtain the initialization parameters. In this case, the model parameters θ0 can only be obtained by random initialization.

[0061] S2. Set a set of possible poisoning rates I, specifically:

[0062]

[0063] Where γ is the preset poisoning rate and r is the poisoning rate interval parameter; for example, if γ is set to 20% and r is set to 5, then: I = {0, 0.05, 0.1, 0.15, 0.2};

[0064] Since the attack poisoning rate cannot be predicted, this method sets a set of possible poisoning rates for subsequent steps to filter and determine the most likely poisoning rate, which is the optimal poisoning rate. The preset poisoning rate is generally set to the maximum possible poisoning rate, which can be set using empirical or experimental methods. However, the maximum poisoning rate cannot exceed 50%. This is because the basic assumption followed by the current poisoning attack and defense is that the proportion of normal samples in the sample is greater than the proportion of poisoned samples.

[0065] S3. Calculate the objective function value for each sample in the current sample set, and sort them in ascending order; the objective function value is expressed as:

[0066]

[0067]

[0068]

[0069] in, The objective function of the model is... Let θ be the loss function of the model. t-1 The model parameters up to the current time slice are λ and Ω(θ). t-1 ) represents the regularization coefficient and the regularization term; The current sample set represents the set of all samples up to the current time slice; b represents the number of samples in each time slice; (x i y i ) represents the i-th sample point, x i With y i Let represent the feature vector and label value corresponding to the i-th sample point, respectively;

[0070] The loss function of the basic online regression model is the mean squared error function, as shown above. The regularization coefficient of the basic online regression model is zero. Therefore, the smaller the objective function value, the more likely the sample is to be a clean sample, and vice versa.

[0071] S4. Calculate the influence of each sample in the current sample set and sort them in ascending order; influence is expressed as:

[0072]

[0073]

[0074]

[0075]

[0076]

[0077] Among them, C i h represents the influence of the i-th sample point, where k is the feature dimension of the sample; ii Represents the hat matrix The corresponding x i The diagonal elements, This represents the feature matrix corresponding to the current sample set; MSE represents the mean squared error of the model on the current sample set. This means that for each sample (x) in the current sample set... i y i The model prediction discrepancy calculated after leave-one-out evaluation, where θ (i) This indicates the use of samples (x) i y i The model parameters θ are obtained by training the model using the current sample set. (-i) This indicates the use of samples (x) without inclusion. i y i The model parameters are obtained by training the model using the current sample set.

[0078] In the formula for calculating influence, the first part represents the model difference generated by the sample, and the second part represents the contribution of the sample. For a normal legitimate model containing noise or outliers, a high influence means that the sample is most likely a noisy or outlier sample, while a low influence means that the sample is more likely to be a clean sample.

[0079] In this invention, steps S3 and S4 serve to implement a dual heuristic screening mechanism based on minimizing the objective function value and minimizing influence, as detailed below:

[0080] like Figure 5 As shown, solid dots represent normal sample points, hollow dots represent potentially abnormal sample points, solid lines represent the fitted line of the estimated regression equation including hollow dots, and dashed lines represent the fitted line of the estimated regression equation excluding hollow dots. Based on differences in image features, potentially abnormal sample points can be divided into three types, as follows: Figure 5 As shown in (a), (b), and (c); Figure 5 As shown in (a), hollow points are clearly outliers because they do not follow the overall trend of the rest of the data. However, when these points are added to the normal sample set for training, the change in the model is not significant. These points are outliers with a relatively small impact on the model. Figure 5 As shown in (b), the hollow point is far from other normal sample points, but it follows the overall trend of the rest of the data. Using this sample point for training will not bias the model. These types of sample points are normal sample points with minimal impact on the model; for example... Figure 5 As shown in (c), hollow points have a different overall trend from other normal sample points, and are obviously outliers. When a sample set containing such a sample point is used for model training, it will cause the model to produce significant bias. Such sample points are outliers that have a significant impact on the model. The goal of the dual heuristic screening mechanism based on minimizing the objective function value and minimizing the influence is to distinguish between the above three types of sample points, thereby filtering out the first and third types of outlier sample points, and retaining the second type of hollow points as reliable clean sample points.

[0081] Influence quantifies the contribution of a sample to the model and its ability to cause model bias; contribution is... Figure 5 A more intuitive representation is the distance between the center of the hollow point set and the center of the solid point set; model bias is... Figure 5 The angle between the solid and dashed lines is more intuitively represented in the middle; it can be seen that the influence can significantly distinguish the third type of sample points, but cannot accurately distinguish the first and second type of sample points.

[0082] The objective function value quantifies the consistency between the sample and the current model distribution. It is related to the current model parameters. The closer the model parameters are to the true values, the higher the discrimination of the objective function value for the sample points. Since θ0 is a randomly initialized model parameter, it cannot distinguish the third type of sample points yet. However, the objective function values ​​of the first and second types of sample points will be closer and have a greater difference from the objective function values ​​of the third type of sample points.

[0083] Therefore, this invention combines influence and objective function value to achieve a dual heuristic screening mechanism: First, a minimum influence-based filtering method is used to filter out sample points with high influence, mainly filtering out third-class sample points; Second, for the remaining sample points, the sample points with the smallest objective function value are selected to construct a reliable clean sample set and update the model parameters, mainly used to distinguish between first-class and second-class sample points. This step is implemented iteratively. Since the model parameters are not accurate at the beginning, they cannot accurately distinguish between the two types of sample points. However, as the iteration progresses, after filtering out the sample points with the greatest influence, the remaining sample points tend to have a consistent effect on the model, causing the model parameters to approach the optimal model parameters during iteration. At this time, the difference between the objective function values ​​of the first-class and second-class sample points will gradually increase. The objective function value of the second-class sample points will gradually decrease as the model parameters approach the optimal parameters, while the first-class sample points will do the opposite. Ultimately, the goal of retaining the second-class sample points and filtering the first-class sample points is achieved.

[0084] S5. For each poisoning rate I j Calculate the poisoning rate I j The total number of clean samples; the total number of clean samples is expressed as:

[0085]

[0086] Where, num j Indicates the poisoning rate I j The total number of clean samples in the set, where len() represents the size of the set. This indicates the total number of samples in the current sample set;

[0087] S6. For each poisoning rate I j Select the nums whose objective function value and influence are both located in the top nums of the sequence. j Construct a candidate clean sample set from 100 samples. Calculate the loss value for each candidate clean sample set, and select the one with the smallest loss value as the reliable clean sample set. And record the corresponding poisoning rate as the optimal poisoning rate P. t The loss value is expressed as:

[0088]

[0089]

[0090] in, Indicates a clean sample set of candidates The loss value;

[0091] S7. Train the model using a reliable and clean sample set, and update the model parameters; specifically:

[0092]

[0093] Here, argmin is the minimization operator in machine learning optimization computation. This indicates the use of a reliable and clean sample set. The model parameters are calculated when the model is trained and the loss function is minimized.

[0094] S8. Update the time slice, repeat S2 to S7 until the model convergence condition is met; the model convergence condition is expressed as:

[0095]

[0096] Where ε is the model convergence parameter, and the default value is usually 1e-8;

[0097] The defense method ends when the model convergence condition is met. The output of the defense method is the optimal poisoning rate, clean model parameters, and a set of reliable clean samples.

[0098] This invention, besides its application in the medical IoT field for drug dosage prediction, is also applicable to areas such as housing price prediction, earthquake prediction, stock prediction, forest fire prediction, and power generation prediction. Any application scenario using a basic online regression model can be used to construct an implementation of the online defense method based on the specific training data of the corresponding field. Unlike drug dosage prediction, the online defense method itself does not differ in terms of features, prediction labels, and corresponding values ​​for each field. For example, in housing price prediction, features include the population composition and housing market conditions of each region, with the prediction label being the median housing price of that region; in earthquake prediction, features include focal depth, latitude, and longitude, with the prediction label being the approximate earthquake intensity; in stock prediction, features are the daily stock prices of existing companies, with the prediction label being the estimated stock price of newly listed companies; in forest fire prediction, features are weather and forest type, with the prediction label being the burned area of ​​a forest fire; and in power generation prediction, features include hourly environmental information such as current, voltage, temperature, and humidity within the power grid, with the prediction label being the hourly net power output.

[0099] In addition to being applicable to basic regression models, this invention is also applicable to other regression models, such as linear regression models Huber, Lasso, Ridge, ElasticNet, Bayesian, etc. Implementation examples of online defense methods can be constructed based on different regression models and the above application areas. The difference between these and the basic online regression models lies in the loss function, regularization term, and regularization coefficient of other regression models; the online defense methods themselves are not different.

[0100] The above description is merely a specific embodiment of the present invention. Any feature disclosed in this specification may be replaced by other equivalent or similar features unless otherwise specified. All disclosed features, or steps in all methods or processes, may be combined in any way except for mutually exclusive features and / or steps.

Claims

1. A method for defending against poisoning attacks in online learning, characterized in that, Includes the following steps: S1. Initialize the parameters of the defense method; S2. Set a set of possible poisoning rates; S3. Calculate the objective function value for each sample in the current sample set, and sort them in ascending order; the objective function value is expressed as: ， ， ； in, The objective function of the model is... The loss function of the model. Indicates the current time slice. These are the model parameters up to the current time slice. and These are the regularization coefficients and regularization terms; Represents the current sample set. and They represent the first and the second, respectively. A time slice, The number of samples for each time slice; Indicates the first One sample point, and They represent the first The feature vector and label value corresponding to each sample point; S4. Calculate the influence of each sample in the current sample set and sort them in ascending order; influence is expressed as: ， ， ， ， ； in, Indicates the first The influence of each sample point Indicates the first One sample point, and They represent the first The feature vector and label value corresponding to each sample point The feature dimension of the sample; This represents the feature matrix corresponding to the current sample set. and They represent the first and the second, respectively. A time slice; These are the model parameters up to the current time slice. Indicates the use of samples The model parameters obtained by training the model on the current sample set. This indicates the use of samples not included. The model parameters obtained by training the model on the current sample set; S5. For each poisoning rate in the set of poisoning rates, calculate the total number of clean samples under that poisoning rate. The total number of clean samples under each poisoning rate is counted as: Regarding the poisoning rate The total number of clean samples is: ， in, Indicates the poisoning rate The total number of clean samples below Indicates the current time slice. The number of samples for each time slice; S6. At each poisoning rate, select the option whose objective function value and influence are both among the top in the sequence. Construct a candidate clean sample set from 100 samples. Calculate the loss value for each candidate clean sample set, and select the one with the smallest loss value as the reliable clean sample set. And record the corresponding poisoning rate as the optimal poisoning rate. The loss value is expressed as: ， ； in, Indicates the first A clean sample set of candidates under a given poisoning rate. Indicates a clean sample set of candidates The loss value; Indicates the first One sample point, and They represent the first The feature vector and label value corresponding to each sample point These are the model parameters up to the current time slice; S7. Train the model using a reliable and clean sample set and update the model parameters; S8. Update the time slice and repeat S2~S7 until the model convergence condition is met.

2. The poisoning attack defense method for online learning according to claim 1, characterized in that, In S1, the model parameters are initialized using random initialization. and initialize the current time slice.

1. Optimal poisoning rate It is 0.

3. The poisoning attack defense method for online learning according to claim 1, characterized in that, In S2, the set of poisoning rates is represented as: Specifically: ， ； in, To preset the poisoning rate, The parameter represents the poisoning rate range. Indicates the first The poisoning rate.

Images