Model defense training method and apparatus, computer device, and storage medium
By using similarity recognition algorithms and sample generation networks, abnormal model data for new backdoor attack methods are identified and generated, which solves the problem of poor defense training effect of artificial intelligence defense models in federated learning and improves the defense capability against new backdoor attacks.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- INDUSTRIAL AND COMMERCIAL BANK OF CHINA
- Filing Date
- 2023-04-25
- Publication Date
- 2026-05-15
AI Technical Summary
In federated learning scenarios, existing AI defense models are poorly trained and unable to effectively respond to unknown attacks due to the difficulty in identifying new backdoor attack methods.
By identifying abnormal model data through similarity recognition algorithms and generating target attack sample data, the model is trained using a sample generation network to improve the defense against new backdoor attack methods.
It effectively identifies and generates abnormal model data for new backdoor attack methods, improves the model's defense against new backdoor attack methods, and enhances the effectiveness of defense training.
Smart Images
Figure CN116468128B_ABST
Abstract
Description
Technical Field
[0001] This application relates to the field of information security technology, and in particular to a model defense training method, apparatus, computer equipment, and storage medium. Background Technology
[0002] Since there are thousands of clients participating in the joint training in the federated learning scenario, and the clients are highly heterogeneous, it is necessary to detect whether there is abnormal data in each client sending access requests to the AI defense model. However, it is difficult to find the backdoor attack method used by the attacker's client, which makes the attack defense of the AI defense model vulnerable. Therefore, how to identify the attacker's attack method is the focus of the defense training of the AI defense model.
[0003] Traditional AI defense model training involves generating multiple attack samples for different backdoor attack methods, and then using these attack samples to train the AI defense model. However, in actual training, there may be unknown new backdoor attack methods. Since it is impossible to generate corresponding attack samples for new backdoor attack methods, it is impossible to train the AI defense model for these new backdoor attack methods, resulting in poor defense training performance of the AI defense model. Summary of the Invention
[0004] Therefore, it is necessary to provide a model defense training method, apparatus, computer equipment, computer-readable storage medium, and computer program product to address the aforementioned technical problems.
[0005] Firstly, this application provides a model defense training method. The method includes:
[0006] Acquire multiple model data for the model, as well as attack sample data for multiple current backdoor attack methods;
[0007] Anomaly data identification strategy is used to identify anomalous model data in each of the model data sets.
[0008] Calculate the similarity between each abnormal model data and each of the attack sample data respectively, and if there is a case where the similarity between the abnormal model data and each of the attack sample data is less than the similarity threshold, determine the target abnormal model data where the similarity is less than the similarity threshold.
[0009] By generating sample attack sample data for each target using a sample generation network and target anomaly model data, the model is trained based on the target attack sample data to obtain the target model.
[0010] Optionally, the step of identifying anomalous model data in each of the model data using a similarity recognition algorithm includes:
[0011] The model data are categorized according to the client from which each model data originates, resulting in a set of model data groups.
[0012] Calculate the similarity between clients corresponding to each model data group, and identify clients that meet the similarity anomaly conditions as the first abnormal client;
[0013] Calculate the average gradient between each set of model data and the gradient value between the model data sets corresponding to each client. Based on the average gradient and the gradient value of each client, calculate the Mahalanobis distance between the gradient value of each client and the average gradient using the Mahalanobis distance similarity algorithm.
[0014] Clients whose Mahalanobis distance is below a preset Mahalanobis distance threshold are designated as second abnormal clients, and the same clients in the first and second abnormal clients are designated as target abnormal clients.
[0015] Each model data in the model data group corresponding to each target abnormal client is used as the abnormal model data.
[0016] Optionally, calculating the similarity between each abnormal model data and each of the attack sample data includes:
[0017] For each abnormal model data, an initial similarity between the abnormal model data and each attack sample data is calculated using a similarity algorithm;
[0018] If the initial similarity falls within a preset similarity range, the backdoor attack method corresponding to the attack sample data that meets the similarity range is determined, and the average of the initial similarities between all attack sample data of the backdoor attack method and the abnormal model data is taken as the similarity between the abnormal model data and each attack sample data of the backdoor attack method.
[0019] If the initial similarity does not meet the similarity range, the initial similarity between the abnormal model data and the attack sample data whose initial similarity does not meet the similarity range shall be used as the similarity between the abnormal model data and the attack sample data.
[0020] Optionally, the step of generating attack sample data for each target by using sample generation network and target anomaly model data includes:
[0021] The source client of the target anomaly model data is determined as the target client, and the anomaly model data of the target client is used as the target model data.
[0022] Based on the sample generation network and the target model data, attack sample data for each target is generated.
[0023] Optionally, generating target attack sample data based on the sample generation network and the target model data includes:
[0024] The feature vector extraction network extracts the feature vector of each target model data, and inputs each target model data into the sample generation network to generate multiple sample data.
[0025] The feature vector extraction network extracts the sample feature vectors of each sample data, and calculates the Euclidean distance between each sample feature vector and each of the feature vectors.
[0026] The sample data corresponding to the Euclidean distance feature vectors that are greater than the Euclidean distance threshold are selected as target attack sample data. Then, the process of inputting the target model data into the sample generation network to generate multiple sample data is repeated until the preset iteration conditions are met, and the target attack sample data is obtained.
[0027] Optionally, after generating the target attack sample data for each target backdoor attack method, the method further includes:
[0028] Based on the similarity algorithm, the similarity between each target attack sample data and each target model data is calculated, and the average similarity between all target attack sample data and each target model data is calculated.
[0029] If the average value is less than the average threshold, the target attack sample data corresponding to the average value that is greater than the average threshold is taken as the target model data, and the process of training the initial sample generation network based on each target feature vector is returned to obtain the sample generation network step until all average values are greater than the average threshold.
[0030] All target attack sample data corresponding to the average value of all targets that are greater than the average threshold are used as the optimized target attack sample data.
[0031] Optionally, if the similarity between the abnormal model data and each of the attack sample data is greater than a similarity threshold, the method further includes:
[0032] Backdoor attack methods of attack sample data with similarity greater than the similarity threshold are taken as target backdoor attack methods;
[0033] Based on the target backdoor attack method, target attack sample data corresponding to the target backdoor attack method is obtained from the attack sample database;
[0034] The target attack sample data are input into the model, and the model is trained to obtain the target model.
[0035] Secondly, this application also provides a model defense training device. The device includes:
[0036] The acquisition module is used to acquire multiple model data of the model, as well as attack sample data of multiple current backdoor attack methods;
[0037] The identification module is used to identify abnormal model data in each of the model data through an abnormal data identification strategy;
[0038] The determination module is used to calculate the similarity between each abnormal model data and each of the attack sample data respectively, and to determine the target abnormal model data in which the similarity between each of the abnormal model data and each of the attack sample data is less than the similarity threshold when there is a situation where the similarity between each of the abnormal model data and each of the attack sample data is less than the similarity threshold.
[0039] The training module is used to generate target attack sample data by using sample generation network and target anomaly model data, and to train the model based on the target attack sample data to obtain the target model.
[0040] Optionally, the identification module is specifically used for:
[0041] The model data are categorized according to the client from which each model data originates, resulting in a set of model data groups.
[0042] Calculate the similarity between clients corresponding to each model data group, and identify clients that meet the similarity anomaly conditions as the first abnormal client;
[0043] Calculate the average gradient between each set of model data and the gradient value between the model data sets corresponding to each client. Based on the average gradient and the gradient value of each client, calculate the Mahalanobis distance between the gradient value of each client and the average gradient using the Mahalanobis distance similarity algorithm.
[0044] Clients whose Mahalanobis distance is below a preset Mahalanobis distance threshold are designated as second abnormal clients, and the same clients in the first and second abnormal clients are designated as target abnormal clients.
[0045] Each model data in the model data group corresponding to each target abnormal client is used as the abnormal model data.
[0046] Optionally, the determining module is specifically used for:
[0047] For each abnormal model data, an initial similarity between the abnormal model data and each attack sample data is calculated using a similarity algorithm;
[0048] If the initial similarity falls within a preset similarity range, the backdoor attack method corresponding to the attack sample data that meets the similarity range is determined, and the average of the initial similarities between all attack sample data of the backdoor attack method and the abnormal model data is taken as the similarity between the abnormal model data and each attack sample data of the backdoor attack method.
[0049] If the initial similarity does not meet the similarity range, the initial similarity between the abnormal model data and the attack sample data whose initial similarity does not meet the similarity range shall be used as the similarity between the abnormal model data and the attack sample data.
[0050] Optionally, the determining module is specifically used for:
[0051] The source client of the target anomaly model data is determined as the target client, and the anomaly model data of the target client is used as the target model data.
[0052] Based on the sample generation network and the target model data, attack sample data for each target is generated.
[0053] Optionally, the determining module is specifically used for:
[0054] The feature vector extraction network extracts the feature vector of each target model data, and inputs each target model data into the sample generation network to generate multiple sample data.
[0055] The feature vector extraction network extracts the sample feature vectors of each sample data, and calculates the Euclidean distance between each sample feature vector and each of the feature vectors.
[0056] The sample data corresponding to the Euclidean distance feature vectors that are greater than the Euclidean distance threshold are selected as target attack sample data. Then, the process of inputting the target model data into the sample generation network to generate multiple sample data is repeated until the preset iteration conditions are met, and the target attack sample data is obtained.
[0057] Optionally, the device further includes:
[0058] The calculation module is used to calculate the similarity between each target attack sample data and each target model data based on a similarity algorithm, and to calculate the average similarity between all target attack sample data and each target model data;
[0059] The iterative module is used to take the target attack sample data corresponding to the average value that is greater than the average threshold as the target model data when there is an average value that is less than the average threshold, and return to execute the step of training the initial sample generation network based on each target feature vector to obtain the sample generation network, until all average values are greater than the average threshold.
[0060] The determination module is used to select the average value of all target attack sample data that are greater than the average threshold as the optimized target attack sample data.
[0061] Optionally, the device further includes:
[0062] The filtering module is used to identify backdoor attack methods in attack sample data with similarity values greater than a similarity threshold as target backdoor attack methods.
[0063] The sample acquisition module is used to acquire target attack sample data corresponding to the target backdoor attack method from the attack sample database based on the target backdoor attack method.
[0064] An optimization module is used to input the target attack sample data into the model and train the model to obtain the target model.
[0065] Thirdly, this application provides a computer device. The computer device includes a memory and a processor, the memory storing a computer program, and the processor executing the computer program to implement the steps of the method described in any one of the first aspects.
[0066] Fourthly, this application provides a computer-readable storage medium having a computer program stored thereon that, when executed by a processor, implements the steps of the method described in any one of the first aspects.
[0067] Fifthly, this application provides a computer program product. The computer program product includes a computer program that, when executed by a processor, implements the steps of the method described in any one of the first aspects.
[0068] The aforementioned model defense training method, apparatus, computer equipment, storage medium, and computer program product acquire multiple model data of the model and attack sample data of multiple current backdoor attack methods; identify abnormal model data in each of the model data using a similarity recognition algorithm; calculate the similarity between each abnormal model data and each of the attack sample data, and if the similarity between the abnormal model data and each of the attack sample data is less than a similarity threshold, designate the backdoor attack method corresponding to the abnormal model data whose similarity is less than the similarity threshold as the target backdoor attack method; generate target attack sample data for each of the target backdoor attack methods using a sample generation network based on the abnormal model data of the target backdoor attack methods; input the target attack sample data into the model and train the model to obtain the target model. This solution uses a similarity recognition algorithm to identify anomalous model data. It can further identify target anomalous model data that differs significantly from the anomalous model data corresponding to current backdoor attack methods, thereby identifying anomalous model data generated by new backdoor attack methods. Then, a sample generation network generates target attack sample data similar to the target anomalous model data, and the target model is trained based on each target attack sample data. This effectively improves the target model's defense against new backdoor attack methods and enhances the effectiveness of target model defense training. Attached Figure Description
[0069] Figure 1 This is a flowchart illustrating a model defense training method in one embodiment;
[0070] Figure 2 This is a flowchart illustrating the steps for generating target attack sample data in one embodiment;
[0071] Figure 3 This is a flowchart illustrating the model defense training method in another embodiment;
[0072] Figure 4 This is a structural block diagram of a model defense training device in one embodiment;
[0073] Figure 5 This is an internal structural diagram of a computer device in one embodiment. Detailed Implementation
[0074] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.
[0075] The model defense training method provided in this application can be applied to terminals, servers, and systems including both terminals and servers, and is implemented through interaction between the terminal and the server. The terminal can include, but is not limited to, various personal computers, laptops, tablets, etc. The terminal can identify abnormal model data using a similarity recognition algorithm, and can further identify target abnormal model data that differs significantly from the abnormal model data corresponding to the current backdoor attack method. This allows for the identification of abnormal model data generated by new backdoor attack methods. Then, a sample generation network generates target attack sample data similar to the target abnormal model data, and the target model is trained based on these target attack sample data. This effectively improves the target model's defense against new backdoor attack methods and enhances the effectiveness of the target model defense training.
[0076] In one embodiment, such as Figure 1 As shown, a model defense training method is provided. Taking the application of this method to a terminal as an example, the method includes the following steps:
[0077] Step S101: Obtain multiple model data of the model and attack sample data of multiple current backdoor attack methods.
[0078] The model is an artificial intelligence defense model.
[0079] In this embodiment, the terminal selects the client from multiple clients that needs to train the model and sends the model to each client respectively. The terminal trains the model through each client separately, with preset time points. After the model training begins and the preset time points are met, the terminal collects model data fed back by each client at that preset time point, obtaining multiple model data sets. These model data sets reflect information about the current training status of the model, including model training level, model defense level, and model fitness. The terminal also retrieves multiple attack sample data sets from the attack sample database, categorized by backdoor attack method, for each existing backdoor attack method. These backdoor attack methods may include, but are not limited to, trigger implantation backdoor attacks, label flipping backdoor attacks, and semantic backdoor attacks.
[0080] Step S102: Identify abnormal model data in each model data using an abnormal data identification strategy.
[0081] In this embodiment, the terminal uses anomaly data identification strategies to determine abnormal model data in each model dataset. These strategies include cosine similarity identification algorithms and Mahalanobis distance similarity identification algorithms. The specific identification process will be explained in detail later.
[0082] Step S103: Calculate the similarity between each abnormal model data and each attack sample data respectively, and if there is a case where the similarity between the abnormal model data and each attack sample data is less than the similarity threshold, determine the target abnormal model data where the similarity is less than the similarity threshold.
[0083] In this embodiment, for each abnormal model data, the terminal calculates the similarity between the abnormal model data and each attack sample data, and presets a similarity threshold. The terminal determines whether there is a situation where the similarity between the abnormal model data and each attack sample data is all less than the similarity threshold. If there is a situation where the similarity between the abnormal model data and each attack sample data is all less than the similarity threshold, the terminal takes the backdoor attack method corresponding to the abnormal model data with a similarity greater than the similarity threshold as the target backdoor attack method. This target backdoor attack method is a new backdoor attack method that is distinct from all current backdoor attack methods. If there is no situation where the similarity between the abnormal model data and each attack sample data is all less than the similarity threshold, the terminal takes the backdoor attack method of the attack sample data with a similarity greater than the similarity threshold as the target backdoor attack method, and takes the abnormal model data with a similarity less than the similarity threshold as the target abnormal model data. The specific calculation process will be explained in detail later. Among them, the target backdoor attack method is the backdoor attack method applied by the client corresponding to the abnormal model data. When the similarity between the abnormal model data and each attack sample data is less than the similarity threshold, the terminal determines that the target backdoor attack method corresponding to the abnormal model data does not belong to any of the current backdoor attack methods (i.e., a new backdoor attack method).
[0084] Step S104: Generate attack sample data for each target using the sample generation network and target anomaly model data, and train the model based on the attack sample data for each target to obtain the target model.
[0085] In this embodiment, when the similarity between the abnormal model data and each attack sample data is less than a similarity threshold, the terminal selects the abnormal model data corresponding to the target backdoor attack method from among the abnormal model data. Based on the abnormal model data of the target backdoor attack method, the terminal generates target attack sample data for the target backdoor attack method through a sample generation network. The specific process of generating target attack sample data will be described in detail later. The sample generation network can be, but is not limited to, any reinforcement learning neural network capable of generating attack sample data as described above.
[0086] The terminal inputs the target attack sample data obtained in the above steps into the model, performs defense training on the model, and obtains a trained model. The terminal then uses this trained model as the target model.
[0087] Based on the above scheme, anomalous model data can be identified using a similarity recognition algorithm. Furthermore, it can identify target anomalous model data that differs significantly from the anomalous model data corresponding to the current backdoor attack method. Experiments verify that when the difference between two model data sets is significant, the backdoor attack methods applied to the two models are different. Therefore, the above process can identify anomalous model data generated by new backdoor attack methods. Then, a sample generation network generates target attack sample data similar to this target anomalous model data, and the target model is trained based on these target attack sample data. This effectively improves the target model's defense against new backdoor attack methods and enhances the training effect of the target model's defense.
[0088] Optionally, anomaly model data in each model data set is identified using a similarity recognition algorithm, including: classifying each model data set according to the client from which each model data set originates, resulting in model data sets; calculating the similarity between clients corresponding to each model data set, and designating clients that meet the anomalous similarity criteria as first anomalous clients; calculating the average gradient between each model data set and the gradient value between model data sets corresponding to each client, and based on the average gradient and the gradient value of each client, calculating the Mahalanobis distance between the gradient value of each client and the average gradient using a Mahalanobis distance similarity algorithm; designating clients whose Mahalanobis distance is below a preset Mahalanobis distance threshold as second anomalous clients, and designating the same client in the first and second anomalous clients as target anomalous clients; and designating each model data set in the model data set corresponding to each target anomalous client as anomalous model data set.
[0089] In this embodiment, the terminal categorizes the model data according to the source client of each model data, obtaining model data groups. The terminal presets a similarity anomaly threshold and calculates the similarity between model data in each model data group using a cosine similarity algorithm, obtaining each first similarity. Based on each model training group, the terminal calculates the similarity between each model training group using a cosine similarity algorithm, obtaining each second similarity. The terminal calculates the similarity between each client based on the sum of the first similarities for each client and the second similarities between each client. Among the clients, the terminal selects clients whose similarity meets the anomaly conditions as first abnormal clients. The anomaly conditions include, for example, the change in each client's first similarity not exceeding a first fluctuation range, and the change trend of the client's second similarity with each client not exceeding a second change threshold. If a client meets the above-mentioned similarity anomaly threshold, the terminal designates that client as the first abnormal client.
[0090] The terminal uses a gradient algorithm to calculate the average gradient between each model's data, and based on this algorithm, calculates the gradient values between each model's data in each client's corresponding model data group. The terminal then uses a Mahalanobis distance similarity algorithm to calculate the Mahalanobis distance between each client's gradient value and this average gradient, obtaining the Mahalanobis distance for each client (i.e., based on the average gradient and each client's gradient value, the Mahalanobis distance between each client's gradient value and the average gradient is calculated using the Mahalanobis distance similarity algorithm). The terminal presets a Mahalanobis distance condition and selects clients whose Mahalanobis distances meet this condition as the second abnormal client. The Mahalanobis distance condition is a threshold; if a client's Mahalanobis distance is lower than the threshold, the terminal identifies that client as the second abnormal client.
[0091] The terminal selects the client that is both the first and second abnormal client from all clients as the target abnormal client, and uses the model data in the model data group corresponding to each target abnormal client as the abnormal model data.
[0092] Based on the above scheme, the similarity recognition algorithm is used to identify abnormal model data in each model data, thereby improving the identification efficiency of abnormal model data.
[0093] Optionally, the similarity between each abnormal model data and each attack sample data is calculated separately, including: for each abnormal model data, calculating the initial similarity between the abnormal model data and each attack sample data using a similarity algorithm; if the initial similarity falls within a preset similarity range, determining the backdoor attack method corresponding to the attack sample data that meets the similarity range, and taking the average of the initial similarities between all attack sample data of the backdoor attack method and the abnormal model data as the similarity between the abnormal model data and each attack sample data of the backdoor attack method; if the initial similarity does not meet the similarity range, taking the initial similarity between the abnormal model data and the attack sample data whose initial similarity does not meet the similarity range as the similarity between the abnormal model data and the attack sample data.
[0094] In this embodiment, the terminal presets a similarity range and, for each abnormal model data, calculates the initial similarity between the abnormal model data and each attack sample data using a similarity algorithm. The similarity algorithm can be, but is not limited to, cosine similarity and Mahalanobis distance similarity algorithms. If any of the initial similarities satisfy the similarity range, the terminal selects the backdoor attack methods corresponding to the attack sample data that satisfy the similarity range from each attack sample data set. The terminal then averages the initial similarities between all attack sample data of that backdoor attack method and the abnormal model data to obtain the total similarity between the abnormal model data and each attack sample data of that backdoor attack method. If any of the initial similarities do not satisfy the similarity range, the terminal selects the initial similarity between the abnormal model data and the attack sample data whose initial similarity does not satisfy the similarity range from each attack sample data set, and uses this initial similarity as the total similarity between the abnormal model data and the attack sample data.
[0095] Based on the above scheme, the similarity between abnormal model data and attack sample data is calculated using a similarity algorithm, thereby improving the efficiency of similarity calculation.
[0096] Optional, such as Figure 2 As shown, attack sample data for each target is generated using sample generation network and target anomaly model data, including:
[0097] Step S201: Determine the source client of the target anomaly model data as the target client, and use the anomaly model data of the target client as the target model data.
[0098] In this embodiment, the terminal, based on the abnormal model data corresponding to the target backdoor attack method, filters out abnormal model data whose source client is the same as that of the abnormal model data corresponding to the target backdoor attack method, and uses the abnormal model data from the same source client as the target model data. The terminal uses the source client of each target model data as the target client.
[0099] Step S202: Based on the sample generation network and the target model data, generate attack sample data for each target.
[0100] In this embodiment, the terminal extracts feature vectors for each target model data using a feature extraction network. Based on these target feature vectors, the terminal trains an initial sample generation network through iterative training to obtain a sample generation network. The feature extraction network can be any type of neural network capable of extracting feature vectors from model data. The initial sample generation network can be, but is not limited to, any reinforcement learning neural network capable of implementing the above steps. The terminal then uses the sample generation network to generate target attack sample data corresponding to each target backdoor attack method, based on the target model data. The specific generation process will be explained in detail later.
[0101] Based on the above scheme, after training the sample generation network, target attack sample data of the target backdoor attack method is generated, which improves the accuracy of generating target attack sample data.
[0102] Optionally, based on the sample generation network and the target model data, target attack sample data is generated, including: extracting feature vectors of each target model data through a feature vector extraction network, and inputting each target model data into the sample generation network to generate multiple sample data; extracting sample feature vectors of each sample data through a feature vector extraction network, and calculating the Euclidean distance between each sample feature vector and each feature vector; selecting sample data corresponding to sample feature vectors with Euclidean distances greater than the Euclidean distance threshold as target attack sample data, and returning to execute the step of inputting each target model data into the sample generation network to generate multiple sample data, until the preset iteration conditions are met to obtain target attack sample data.
[0103] In this embodiment, the terminal extracts feature vectors for each target model data using a feature vector extraction network. The terminal inputs all target model data into an initial sample generation network to randomly generate multiple sample data. The terminal then extracts the feature vectors for each sample data again using the feature vector extraction network. The terminal calculates the Euclidean distance between each sample feature vector and all other feature vectors using a feature vector distance algorithm, and presets an Euclidean distance threshold. From the sample feature vectors, the terminal selects sample data corresponding to feature vectors with Euclidean distances greater than the threshold as target attack sample data. The terminal iteratively executes the steps of inputting each target model data into the sample generation network to generate multiple sample data until a preset iteration condition is met. This preset iteration condition may include, but is not limited to, the number of iterations or the number of sample feature vectors with Euclidean distances greater than the threshold. The terminal outputs the target attack sample data obtained in each iteration.
[0104] Based on the above scheme, by setting a preset feature vector distance threshold and preset iteration conditions, the number of target attack sample data generated and the limiting standard for the number of target attack samples are determined, thereby improving the accuracy of generating target attack sample data.
[0105] Optionally, based on a similarity algorithm, the similarity between each target attack sample data and each target model data is calculated, and the average similarity between all target attack sample data and each target model data is calculated. If there is an average value less than the average threshold, the target attack sample data corresponding to the average value greater than the average threshold is taken as the target model data, and the process is repeated to train the initial sample generation network based on the feature vectors of each target, until all average values are greater than the average threshold. The target attack sample data corresponding to all average values greater than the average threshold is taken as each optimized target attack sample data.
[0106] In this embodiment, after generating target attack sample data for each target backdoor attack method, the terminal calculates the average similarity between each target attack sample data and each target model data based on a similarity algorithm, and presets an average threshold. The terminal then determines whether there are cases where the average similarity is less than the average threshold. If there are cases where the average similarity is less than the average threshold, the terminal uses the target attack sample data corresponding to the average values greater than the average threshold as the target model data, and returns to execute the step of training the initial sample generation network based on each target feature vector to obtain the sample generation network. This process continues until all average similarity values are greater than the average threshold, at which point the iteration stops, and all target attack sample data corresponding to the average values greater than the average threshold are used as each optimized target attack sample data. If there are no cases where the average similarity is less than the average threshold, the terminal uses all target attack sample data corresponding to the average values greater than the average threshold as each optimized target attack sample data.
[0107] Based on the above scheme, the accuracy of each target attack sample data is improved by optimizing the target attack sample data.
[0108] Optionally, if the similarity between the abnormal model data and each attack sample data is greater than a similarity threshold, the method further includes: taking the backdoor attack method of the attack sample data with a similarity greater than the similarity threshold as the target backdoor attack method; based on the target backdoor attack method, obtaining the target attack sample data corresponding to the target backdoor attack method from the attack sample database; inputting each target attack sample data into the model and training the model to obtain the target model.
[0109] In this embodiment, when there is no abnormal model data and the similarity between all attack sample data is less than the similarity threshold, the terminal determines that the backdoor attack method of the client corresponding to the abnormal model data is any one or more combinations of the current backdoor attack methods. The terminal selects backdoor attack methods from among the various backdoor attack methods whose similarity to the attack sample data is greater than the similarity threshold, and uses this backdoor attack method as the target backdoor attack method. The terminal presets a target number of target attack sample data and selects attack sample data of the target number corresponding to the target backdoor attack method from the attack sample database as the target attack sample data. The terminal inputs all target attack sample data into the model and trains the model to obtain the target model.
[0110] Based on the above approach, by selecting target attack sample data of backdoor attack methods used by the client, the model is trained, thereby improving the model's defense effectiveness against this type of backdoor attack.
[0111] This application also provides an example of model defense training, such as Figure 3 As shown, the specific processing procedure includes the following steps:
[0112] Step S301: Obtain multiple model data of the model and attack sample data of multiple current backdoor attack methods.
[0113] Step S302: Classify the model data according to the source client of each model data to obtain each model data group.
[0114] Step S303: Calculate the similarity between clients corresponding to each model data group, and identify clients that meet the abnormal similarity conditions as the first abnormal client.
[0115] Step S304: Calculate the average gradient between each model data set and the gradient value between the model data sets corresponding to each client. Based on the average gradient and the gradient value of each client, calculate the Mahalanobis distance between the gradient value of each client and the average gradient using the Mahalanobis distance similarity algorithm.
[0116] Step S305: Clients whose Mahalanobis distance is lower than the preset Mahalanobis distance threshold are designated as second abnormal clients, and the same clients in the first and second abnormal clients are designated as target abnormal clients.
[0117] Step S306: Take each model data in the model data group corresponding to each target abnormal client as the abnormal model data.
[0118] Step S307: For each abnormal model data, calculate the initial similarity between the abnormal model data and each attack sample data using a similarity algorithm.
[0119] Step S308: If the initial similarity is within a preset similarity range, determine the backdoor attack method corresponding to the attack sample data that meets the similarity range, and take the average of the initial similarities between all attack sample data of the backdoor attack method and the abnormal model data as the similarity between the abnormal model data and each attack sample data of the backdoor attack method.
[0120] Step S309: If the initial similarity does not meet the similarity range, the initial similarity between the abnormal model data and the attack sample data whose initial similarity does not meet the similarity range is taken as the similarity between the abnormal model data and the attack sample data.
[0121] Step S310: Determine the source client of the target anomaly model data as the target client, and use the anomaly model data of the target client as the target model data.
[0122] Step S311: Extract the feature vector of each target model data through the feature vector extraction network, and input each target model data into the sample generation network to generate multiple sample data.
[0123] Step S312: Extract the sample feature vectors of each sample data through the feature vector extraction network, and calculate the Euclidean distance between each sample feature vector and each feature vector.
[0124] Step S313: Select sample data corresponding to sample feature vectors with Euclidean distance greater than the Euclidean distance threshold as target attack sample data, and return to execute the step of inputting each target model data into the sample generation network to generate multiple sample data until the preset iteration conditions are met to obtain each target attack sample data.
[0125] Step S314: Based on the similarity algorithm, calculate the similarity between each target attack sample data and each target model data, and calculate the average similarity between all target attack sample data and each target model data.
[0126] Step S315: If there is an average value less than the average threshold, take the target attack sample data corresponding to the average value greater than the average threshold as the target model data, and return to execute the initial sample generation network training based on each target feature vector to obtain the sample generation network step, until all average values are greater than the average threshold.
[0127] Step S316: Take the average value of all target attack sample data that is greater than the average threshold as the optimized target attack sample data.
[0128] It should be understood that although the steps in the flowcharts of the embodiments described above are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowcharts of the embodiments described above may include multiple steps or multiple stages. These steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the steps or stages of other steps.
[0129] Based on the same inventive concept, this application also provides a model defense training apparatus for implementing the model defense training method described above. The solution provided by this apparatus is similar to the implementation described in the above method; therefore, the specific limitations in one or more embodiments of the model defense training apparatus provided below can be found in the limitations of the model defense training method described above, and will not be repeated here.
[0130] In one embodiment, such as Figure 4 As shown, a model defense training device is provided, including: an acquisition module 410, an identification module 420, a determination module 430, and a training module 440, wherein:
[0131] The acquisition module 410 is used to acquire multiple model data of the model and attack sample data of multiple current backdoor attack methods;
[0132] The identification module 420 is used to identify abnormal model data in each of the model data through an abnormal data identification strategy;
[0133] The determination module 430 is used to calculate the similarity between each abnormal model data and each of the attack sample data respectively, and when there is a case where the similarity between the abnormal model data and each of the attack sample data is less than the similarity threshold, the target abnormal model data whose similarity is less than the similarity threshold is determined.
[0134] The training module 440 is used to generate target attack sample data by using the sample generation network and target anomaly model data, and to train the model based on the target attack sample data to obtain the target model.
[0135] Optionally, the 420 identification modules are specifically used for:
[0136] The model data are categorized according to the client from which each model data originates, resulting in a set of model data groups.
[0137] Calculate the similarity between clients corresponding to each model data group, and identify clients that meet the similarity anomaly conditions as the first abnormal client;
[0138] Calculate the average gradient between each set of model data and the gradient value between the model data sets corresponding to each client. Based on the average gradient and the gradient value of each client, calculate the Mahalanobis distance between the gradient value of each client and the average gradient using the Mahalanobis distance similarity algorithm.
[0139] Clients whose Mahalanobis distance is below a preset Mahalanobis distance threshold are designated as second abnormal clients, and the same clients in the first and second abnormal clients are designated as target abnormal clients.
[0140] Each model data in the model data group corresponding to each target abnormal client is used as the abnormal model data.
[0141] Optionally, the determining module 430 is specifically used for:
[0142] For each abnormal model data, an initial similarity between the abnormal model data and each attack sample data is calculated using a similarity algorithm;
[0143] If the initial similarity falls within a preset similarity range, the backdoor attack method corresponding to the attack sample data that meets the similarity range is determined, and the average of the initial similarities between all attack sample data of the backdoor attack method and the abnormal model data is taken as the similarity between the abnormal model data and each attack sample data of the backdoor attack method.
[0144] If the initial similarity does not meet the similarity range, the initial similarity between the abnormal model data and the attack sample data whose initial similarity does not meet the similarity range shall be used as the similarity between the abnormal model data and the attack sample data.
[0145] Optionally, the determining module 430 is specifically used for:
[0146] The source client of the target anomaly model data is determined as the target client, and the anomaly model data of the target client is used as the target model data.
[0147] Based on the sample generation network and the target model data, attack sample data for each target is generated.
[0148] Optionally, the determining module 430 is specifically used for:
[0149] The feature vector extraction network extracts the feature vector of each target model data, and inputs each target model data into the sample generation network to generate multiple sample data.
[0150] The feature vector extraction network extracts the sample feature vectors of each sample data, and calculates the Euclidean distance between each sample feature vector and each of the feature vectors.
[0151] The sample data corresponding to the Euclidean distance feature vectors that are greater than the Euclidean distance threshold are selected as target attack sample data. Then, the process of inputting the target model data into the sample generation network to generate multiple sample data is repeated until the preset iteration conditions are met, and the target attack sample data is obtained.
[0152] Optionally, the device further includes:
[0153] The calculation module is used to calculate the similarity between each target attack sample data and each target model data based on a similarity algorithm, and to calculate the average similarity between all target attack sample data and each target model data;
[0154] The iterative module is used to take the target attack sample data corresponding to the average value that is greater than the average threshold as the target model data when there is an average value that is less than the average threshold, and return to execute the step of training the initial sample generation network based on each target feature vector to obtain the sample generation network, until all average values are greater than the average threshold.
[0155] The determination module is used to select the average value of all target attack sample data that are greater than the average threshold as the optimized target attack sample data.
[0156] Optionally, the device further includes:
[0157] The filtering module is used to identify backdoor attack methods in attack sample data with similarity values greater than a similarity threshold as target backdoor attack methods.
[0158] The sample acquisition module is used to acquire target attack sample data corresponding to the target backdoor attack method from the attack sample database based on the target backdoor attack method.
[0159] An optimization module is used to input the target attack sample data into the model and train the model to obtain the target model.
[0160] Each module in the aforementioned model defense training device can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in or independent of the processor in a computer device, or stored in the computer device's memory as software, so that the processor can call and execute the corresponding operations of each module.
[0161] In one embodiment, a computer device is provided, which may be a terminal, and its internal structure diagram may be as follows: Figure 5 As shown, the computer device includes a processor, memory, communication interface, display screen, and input devices connected via a system bus. The processor provides computing and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system and computer programs. The internal memory provides an environment for the operation of the operating system and computer programs stored in the non-volatile storage media. The communication interface is used for wired or wireless communication with external terminals; wireless communication can be achieved through Wi-Fi, mobile cellular networks, NFC (Near Field Communication), or other technologies. When the computer program is executed by the processor, it implements a model defense training method. The display screen can be an LCD screen or an e-ink screen. The input devices can be a touch layer covering the display screen, buttons, a trackball, or a touchpad mounted on the computer device casing, or an external keyboard, touchpad, or mouse.
[0162] Those skilled in the art will understand that Figure 5 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.
[0163] In one embodiment, a computer device is provided, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps of the method described in any one of the first aspects.
[0164] In one embodiment, a computer-readable storage medium is provided having a computer program stored thereon, which, when executed by a processor, implements the steps of the method described in any one of the first aspects.
[0165] In one embodiment, a computer program product is provided, including a computer program that, when executed by a processor, implements the steps of the method described in any one of the first aspects.
[0166] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties.
[0167] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. Any references to memory, databases, or other media used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM). The databases involved in the embodiments provided in this application may include at least one type of relational database and non-relational database. Non-relational databases may include, but are not limited to, blockchain-based distributed databases. The processors involved in the embodiments provided in this application may be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, quantum computing-based data processing logic devices, etc., and are not limited to these.
[0168] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.
[0169] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of this patent application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this application should be determined by the appended claims.
Claims
1. A model defense training method, characterized in that, The method includes: Acquire multiple model data for the model, as well as attack sample data for multiple current backdoor attack methods; Anomaly data identification strategy is used to identify anomalous model data in each of the model data sets; the anomaly data identification strategy includes a cosine similarity identification algorithm and a Mahalanobis distance similarity identification algorithm. Calculate the similarity between each abnormal model data and each of the attack sample data respectively, and if there is a case where the similarity between the abnormal model data and each of the attack sample data is less than the similarity threshold, determine the target abnormal model data where the similarity is less than the similarity threshold. The source client of the target anomaly model data is determined as the target client, and the anomaly model data of the target client is used as the target model data. Based on the sample generation network and the target model data, attack sample data for each target is generated, and the target attack sample data is input into the model and trained to obtain the target model.
2. The method according to claim 1, characterized in that, The step of identifying anomalous model data in each model data set using an anomaly data identification strategy includes: The model data are categorized according to the client from which each model data originates, resulting in a set of model data groups. Calculate the similarity between clients corresponding to each model data group, and identify clients that meet the similarity anomaly conditions as the first abnormal client; Calculate the average gradient between each set of model data and the gradient value between the model data sets corresponding to each client. Based on the average gradient and the gradient value of each client, calculate the Mahalanobis distance between the gradient value of each client and the average gradient using the Mahalanobis distance similarity algorithm. Clients whose Mahalanobis distance is below a preset Mahalanobis distance threshold are designated as second abnormal clients, and the same clients in the first and second abnormal clients are designated as target abnormal clients. Each model data in the model data group corresponding to each target abnormal client is used as the abnormal model data.
3. The method according to claim 1, characterized in that, The step of calculating the similarity between each abnormal model data and each of the attack sample data includes: For each abnormal model data, an initial similarity between the abnormal model data and each attack sample data is calculated using a similarity algorithm; If the initial similarity falls within a preset similarity range, the backdoor attack method corresponding to the attack sample data that meets the similarity range is determined, and the average of the initial similarities between all attack sample data of the backdoor attack method and the abnormal model data is taken as the similarity between the abnormal model data and each attack sample data of the backdoor attack method. If the initial similarity does not meet the similarity range, the initial similarity between the abnormal model data and the attack sample data whose initial similarity does not meet the similarity range shall be used as the similarity between the abnormal model data and the attack sample data.
4. The method according to claim 1, characterized in that, The generation of attack sample data for each target based on the sample generation network and the target model data includes: The feature vector extraction network extracts the feature vector of each target model data, and inputs each target model data into the sample generation network to generate multiple sample data. The feature vector extraction network extracts the sample feature vectors of each sample data, and calculates the Euclidean distance between each sample feature vector and each of the feature vectors. The sample data corresponding to the Euclidean distance feature vectors that are greater than the Euclidean distance threshold are selected as target attack sample data. The process is then repeated until the preset iteration conditions are met, and the target attack sample data is obtained.
5. The method according to claim 4, characterized in that, After generating the attack sample data for each target, the process also includes: Based on the similarity algorithm, the similarity between each target attack sample data and each target model data is calculated, and the average similarity between all target attack sample data and each target model data is calculated. If the average value is less than the average threshold, the target attack sample data corresponding to the average value that is greater than the average threshold is taken as the target model data, and the step of training the initial sample generation network based on each target feature vector is returned to obtain the sample generation network until all average values are greater than the average threshold. All target attack sample data corresponding to the average value of all targets that are greater than the average threshold are used as the optimized target attack sample data.
6. The method according to claim 1, characterized in that, If the similarity between the abnormal model data and each of the attack sample data is greater than a similarity threshold, the method further includes: Backdoor attack methods of attack sample data with similarity greater than the similarity threshold are taken as target backdoor attack methods; Based on the target backdoor attack method, target attack sample data corresponding to the target backdoor attack method is obtained from the attack sample database; The target attack sample data are input into the model, and the model is trained to obtain the target model.
7. A model defense training device, characterized in that, The device includes: The acquisition module is used to acquire multiple model data of the model, as well as attack sample data of multiple current backdoor attack methods; The identification module is used to identify abnormal model data in each of the model data through anomaly data identification strategies; the anomaly data identification strategies include cosine similarity identification algorithms and Mahalanobis distance similarity identification algorithms; The determination module is used to calculate the similarity between each abnormal model data and each of the attack sample data respectively, and when there is a case where the similarity between the abnormal model data and each of the attack sample data is less than the similarity threshold, the target abnormal model data whose similarity is less than the similarity threshold is determined. The training module is used to determine the source client of the target anomaly model data, which is the target client, and to use the anomaly model data of the target client as the target model data. Based on the sample generation network and the target model data, attack sample data for each target is generated, and the target attack sample data is input into the model and trained to obtain the target model.
8. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the method according to any one of claims 1 to 6.
9. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 6.
10. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 6.