A robust security evaluation method of a federated learning system

Abstract

The application discloses a kind of robustness safety evaluation methods of federated learning system, belong to artificial intelligence security technical field, by superimposing candidate trigger disturbance on benign sample to generate candidate poisoned sample, the classification loss of candidate poisoned sample is predicted as target class label based on local model, and poisoned sample is screened from candidate poisoned sample;Candidate trigger disturbance in poisoned sample is iteratively updated until the minimum classification loss is obtained, and the optimal trigger disturbance is obtained;The optimal trigger disturbance is injected into the poisoned sample, and the sum of the classification loss of the optimized poisoned sample and the classification loss of benign sample by the local model is used as the target, and the local model is updated;The updated local model is submitted to the server, and the global model returned by the server is input into the test image sample with trigger disturbance, based on the deviation of the prediction result and the target class label, generate the robustness evaluation result of federated learning system, effectively improve the concealment and stability of evaluation.

Description

Technical Field

[0001] This invention relates to the field of artificial intelligence security technology, and more specifically to a robust security assessment method for a federated learning system. Background Technology

[0002] With the rapid deployment of the Internet of Things, edge computing, and privacy-sensitive artificial intelligence services, Federated Learning (FL), with its distributed training paradigm that transmits only gradients and not data outside the domain, has become a core technical architecture for solving the risks of data silos and privacy leaks. It is widely used in key mission areas such as smart healthcare, financial risk control, and autonomous driving.

[0003] Currently, the mainstream methods for robustness evaluation of federated learning systems primarily rely on model poisoning evaluation, the core of which lies in constructing malicious model updates to test the tolerance limits of the aggregation mechanism. This method controls one or more simulated malicious clients. After local training, instead of directly submitting normal gradients, these clients arithmetically amplify and directionally tamper with the local model parameters (e.g., multiplying the parameter increment of the malicious model by a large scaling factor), constructing an anomalous gradient vector with a parameter norm far exceeding that of benign client updates. This malicious update is uploaded to a central server and participates in federated averaging aggregation. The federated learning system quantifies the robustness tolerance of the aggregation mechanism to anomalous inputs by observing the probability of erroneous outputs of the global model under specific test inputs. Its core underlying mechanism lies in leveraging the federated averaging algorithm's characteristic of assigning higher aggregation weights to gradients with larger numerical amplitudes; that is, the larger the gradient norm, the higher its influence on the global model during parameter averaging. By controlling the scaling ratio of the malicious gradient, the global model parameters are significantly shifted in the malicious direction, thereby determining the level of perturbation at which the system loses its ability to discriminate trigger inputs.

[0004] However, the above method relies too heavily on malicious clients continuously participating in and triggering the perturbation construction process. This makes the backdoor features easily weakened by the gradient dilution effect of a large number of benign client updates during multiple rounds of global aggregation. Although this method can quickly achieve initial detection of system robustness, its malicious updates inevitably have measurable differences from benign updates in terms of parameter norm, gradient direction, or statistical distribution, resulting in insufficient evaluation stability. Summary of the Invention

[0005] To address the problems existing in the above-mentioned fields, this invention proposes a robust security assessment method for federated learning systems. By screening poisoned samples from benign samples and iteratively optimizing the trigger perturbation, the backdoor features and the local model co-evolve. At the same time, the model update is constrained by the dual classification loss of optimized poisoned and benign samples, so that the statistical features of malicious updates approach benign updates. This overcomes the gradient dilution effect in multi-round aggregation and achieves continuous and stable assessment of the robustness of federated learning systems.

[0006] To address the aforementioned technical problems, this invention discloses a robust security assessment method for a federated learning system, comprising the following steps: The global model distributed by the server in the federated learning system is obtained as the local model, and the training image sample set is obtained. The target category label is determined, and the samples in the training image sample set whose original label is not the target category label are regarded as benign samples. Candidate poisoned samples are generated by superimposing candidate trigger perturbations on benign samples. Based on the classification loss of predicting the target category label of the candidate poisoned samples using the local model, poisoned samples are selected from the candidate poisoned samples. The candidate trigger perturbations in the poisoned samples are iteratively updated until the classification loss is minimized to obtain the optimal trigger perturbation. The optimal trigger perturbation is injected into the poisoned samples to obtain optimized poisoned samples. The local model is updated with the sum of the classification loss of the local model for optimized poisoned samples and the classification loss for benign samples as the objective. The updated local model is submitted to the server, and the global model returned by the server after aggregation is obtained. The test image samples with trigger perturbations are input into the returned global model, and the robustness evaluation results of the federated learning system are generated based on the deviation between the prediction results and the target category labels.

[0007] Preferably, the step of screening poisoned samples from candidate poisoned samples specifically includes: Extract deep features of each candidate poisoning sample from the local model; The classification loss of the local model in predicting each candidate poisoned sample as the target category label is obtained; Determine the distance between the deep features of each candidate poisoning sample and the feature center of the target category, and use the weighted sum of distance and classification loss as the evaluation cost; The candidate poisoning sample with the lowest evaluation cost will be selected as the poisoning sample. Wherein, the target category feature center is the mean value of the deep features extracted by the local model from the samples belonging to the target category label in the training image sample set.

[0008] Preferably, updating the local model with the goal of summing the classification loss of the local model for optimized poisoned samples and the classification loss for benign samples specifically includes: The classification loss of optimizing the prediction of poisoned samples as the target category label in the local model is used as the backdoor loss term; The classification loss of predicting benign samples as their original labels using the local model is called the benign loss term; The sum of the backdoor loss term and the benign loss term is used as the total loss. The parameters of the local model are then updated through backpropagation to obtain the updated local model.

[0009] Preferably, the total loss further includes a parameter distance regularization term, which is the L2 distance between the current parameters of the local model and the global model parameters issued by the server in the previous round; the product of the parameter distance regularization term and the preset weight coefficient is added to the total loss as a regularization term.

[0010] Preferably, the iterative updating of candidate triggering perturbations in the poisoned samples until the classification loss is minimized, thus obtaining the optimal triggering perturbation, specifically includes: With the current parameters of the local model fixed, the candidate triggering perturbations are iteratively updated using a multi-step projective gradient descent algorithm. After each update, the updated candidate triggering perturbation is projected onto a preset perturbation budget constraint, which is an Lp norm constraint on the triggering perturbation.

[0011] Preferably, before submitting the updated local model to the server, the method further includes a step of spoofing the gradients of the local model: Obtain the gradient of the local model update and determine the unit direction vector of the gradient; A random noise vector is sampled from a Gaussian distribution, and the random noise vector is projected onto the direction of a unit direction vector to obtain the projection component; The projected components are superimposed on the gradient to obtain the dummy gradient; the dummy gradient is then used as the gradient of the updated local model and submitted to the server.

[0012] Preferably, generating robustness evaluation results for the federated learning system based on the deviation between the prediction results and the target category labels specifically includes: Based on the prediction results of the returned global model for test image samples with trigger perturbations, the proportion of samples whose prediction results are the target category labels is counted and used as an evaluation index. And obtain the classification accuracy of the returned global model on test image samples without trigger perturbation; The evaluation metrics and classification accuracy are used together as the output of the robustness evaluation results of the federated learning system.

[0013] Preferably, it also includes a robust security assessment system for the federated learning system, comprising: The data acquisition module is used to acquire the global model issued by the server in the federated learning system as the local model, acquire the training image sample set, determine the target category label, and take the samples in the training image sample set whose original label is not the target category label as benign samples. The local model update module is used to generate candidate poisoned samples by superimposing candidate trigger perturbations on benign samples; to predict the candidate poisoned samples as the target category label based on the classification loss of the local model; to screen poisoned samples from the candidate poisoned samples; to iteratively update the candidate trigger perturbations in the poisoned samples until the classification loss is minimized, thus obtaining the optimal trigger perturbation; to inject the optimal trigger perturbation into the poisoned samples, thus obtaining optimized poisoned samples; to update the local model with the sum of the classification loss of the local model for optimized poisoned samples and the classification loss for benign samples as the objective; and to submit the updated local model to the server and obtain the global model returned by the server after aggregation. The robustness evaluation module is used to input test image samples with trigger perturbations into the returned global model, and generate robustness evaluation results of the federated learning system based on the deviation between the prediction results and the target category label.

[0014] Preferably, the device further includes a computer apparatus, the computer apparatus comprising a memory and a processor, the memory storing a computer program, which, when executed by the processor, causes the processor to perform the following steps: The global model distributed by the server in the federated learning system is obtained as the local model, and the training image sample set is obtained. The target category label is determined, and the samples in the training image sample set whose original label is not the target category label are regarded as benign samples. Candidate poisoned samples are generated by superimposing candidate trigger perturbations on benign samples. Based on the classification loss of predicting the target category label of the candidate poisoned samples using the local model, poisoned samples are selected from the candidate poisoned samples. The candidate trigger perturbations in the poisoned samples are iteratively updated until the classification loss is minimized to obtain the optimal trigger perturbation. The optimal trigger perturbation is injected into the poisoned samples to obtain optimized poisoned samples. The local model is updated with the sum of the classification loss of the local model for optimized poisoned samples and the classification loss for benign samples as the objective. The updated local model is submitted to the server, and the global model returned by the server after aggregation is obtained. The test image samples with trigger perturbations are input into the returned global model, and the robustness evaluation results of the federated learning system are generated based on the deviation between the prediction results and the target category labels.

[0015] Preferably, the system further includes a computer-readable storage medium storing a computer program that, when executed by a processor, causes the processor to perform the following steps: The global model distributed by the server in the federated learning system is obtained as the local model, and the training image sample set is obtained. The target category label is determined, and the samples in the training image sample set whose original label is not the target category label are regarded as benign samples. Candidate poisoned samples are generated by superimposing candidate trigger perturbations on benign samples. Based on the classification loss of predicting the target category label of the candidate poisoned samples using the local model, poisoned samples are selected from the candidate poisoned samples. The candidate trigger perturbations in the poisoned samples are iteratively updated until the classification loss is minimized to obtain the optimal trigger perturbation. The optimal trigger perturbation is injected into the poisoned samples to obtain optimized poisoned samples. The local model is updated with the sum of the classification loss of the local model for optimized poisoned samples and the classification loss for benign samples as the objective. The updated local model is submitted to the server, and the global model returned by the server after aggregation is obtained. The test image samples with trigger perturbations are input into the returned global model, and the robustness evaluation results of the federated learning system are generated based on the deviation between the prediction results and the target category labels.

[0016] Compared with the prior art, the present invention has the following beneficial effects: The robust security assessment method for federated learning systems proposed in this invention addresses the problem in existing technologies where the statistical characteristics of malicious updates differ from those of benign updates, leading to a rapid decay in assessment effectiveness after multiple rounds of aggregation. By screening poisoned samples from benign samples and iteratively optimizing the trigger perturbation, and simultaneously using a dual classification loss constraint model update that optimizes both poisoned and benign samples, malicious updates acquire statistical characteristics that converge with benign updates before participating in aggregation. This effectively overcomes the gradient dilution effect in multiple rounds of aggregation and achieves robust assessment that is stable across rounds. In this process, the selected optimized poisoned samples provide an iterative optimization basis for triggering perturbations that matches the current model state, enabling the iteratively updated trigger perturbations to co-evolve with the model parameters. The dual classification loss constraint limits the model update magnitude in this co-evolving state to a reasonable range defined by the benign training task, so that malicious updates naturally approach benign updates in terms of statistical characteristics. The updated model is returned after being aggregated by the server. By obtaining the returned global model and detecting its output deviation under the trigger input, it is verified whether the backdoor effect produced by the above co-evolving constraints remains effective after multiple iterations, thereby achieving a continuous and stable evaluation of the robustness of the federated learning system. Attached Figure Description

[0017] Figure 1 This is a flowchart of the robust security assessment method for the federated learning system proposed in this invention; Figure 2 The following are the specific implementation steps of the robust security assessment method for the federated learning system of this invention; Figure 3This is the optimal position insertion strategy for triggering perturbation provided in the embodiments of the present invention; Figure 4 The adaptive gradient update strategy provided in this embodiment of the invention; Figure 5 The gradient masking strategy provided in the embodiments of the present invention; Figure 6 The attack success rate provided by the embodiments of the present invention on different datasets; Figure 7 The classification accuracy on different datasets provided in the embodiments of the present invention; Figure 8 The training results of visualized backdoor samples on different datasets are provided in the embodiments of the present invention. Detailed Implementation

[0018] The following will refer to the appendices in the embodiments of the present invention. Figures 1-8 The technical solutions in the embodiments of the present invention will be clearly and completely described. It should be understood that the terminology used in the present invention is only for describing particular implementation methods and is not intended to limit the present invention.

[0019] Example like Figure 1 As shown, this invention proposes a robust security assessment method for federated learning systems (Adaptive Gradient Distributed Backdoor Attacks Based on Federated Learning, AG-DBA), applied to a federated learning system comprising a server and at least one malicious client storing an image sample set. The method includes the following steps: S1: Obtain the global model issued by the server in the federated learning system as the local model, obtain the training image sample set, determine the target category label, and take the samples in the training image sample set whose original label is not the target category label as benign samples. S2: Generate candidate poisoned samples by superimposing candidate triggering perturbations on benign samples. Based on the classification loss of predicting the target category label of the candidate poisoned samples using the local model, filter poisoned samples from the candidate poisoned samples; iteratively update the candidate triggering perturbations in the poisoned samples until the classification loss is minimized to obtain the optimal triggering perturbation; inject the optimal triggering perturbation into the poisoned samples to obtain optimized poisoned samples; update the local model with the sum of the classification loss of the local model for optimized poisoned samples and the classification loss for benign samples as the objective; submit the updated local model to the server and obtain the global model returned by the server after aggregation. S3: Input the test image samples with trigger perturbations into the returned global model, and generate a robustness evaluation result of the federated learning system based on the deviation between the prediction results and the target category label.

[0020] This method aims to address the problems of insufficient persistence and lack of concealment in existing backdoor attacks in federated learning scenarios.

[0021] In this embodiment, the training image sample sets obtained by this invention are from three public datasets: MNIST, CIFAR-10, and GTSRB. A lightweight ResNet-18 model is selected for classification on each dataset. The acquisition process of the above three datasets is legal and compliant. The MNIST dataset contains 70,000 grayscale images of handwritten digits, each 28×28 pixels. 60,000 images are training samples and 10,000 images are test samples. Each image represents a digit from 0 to 9.

[0022] The CIFAR-10 dataset contains 60,000 32×32 pixel color images, of which 50,000 are training samples and 10,000 are test samples. The images are divided into 10 categories, including ships, trucks, frogs, dogs, horses, etc.

[0023] The GTSRB dataset contains 60,000 32×32 pixel color images for traffic sign recognition. Of these, 39,209 images are training samples and 12,630 images are test samples, categorized into 43 traffic sign classes, including slow down, keep right, and no parking.

[0024] The target category label is determined based on the distribution of each dataset.

[0025] The server initializes a lightweight ResNet-18 model as the global model (the global model for the current round) and distributes this global model to the clients selected by the server, forming a client set.

[0026] Prior to step S1, the method proposed in this invention further includes an attack scheduling step, specifically including: To determine whether the client set contains malicious clients, such as... Figure 2 As shown, where: If the client set does not contain malicious clients, the image sample set is trained normally based on the local model, and the trained local model is updated and submitted to the server.

[0027] When the client set contains malicious clients, samples whose original labels are not the target category labels are selected from the image sample set as benign samples.

[0028] Step S21: Screening poisoned samples Candidate poisoning samples are generated by superimposing candidate triggering perturbations on benign samples; Extract deep features of each candidate poisoning sample from the local model; Obtain the classification loss of the local model that predicts each candidate poisoned sample as the target category label; The mean of the deep features extracted by the local model from the samples belonging to the target category label in the training image sample set is used as the distance between the deep features of each candidate poisoned sample and the center of the target category features. The weighted sum of distance and classification loss is used as the evaluation cost, and the candidate poisoned sample with the minimum evaluation cost is selected as the poisoned sample.

[0029] In this embodiment, candidate poisoning samples are first generated by superimposing candidate triggering perturbations on benign samples. Then, a poisoning sample optimization screening strategy is used to select the poisoning samples that best match the target category features from the candidate poisoning samples. For example... Figure 3 The image shows an optimized screening strategy for poisoned samples, which includes:

[0030] (1) Trigger initialization In this embodiment, the original set of trigger perturbation pixels is first normalized to determine the insertion position of the candidate trigger; the trigger perturbation is defined as an additive perturbation tensor with the same dimension as the input image, and then processed by a mask matrix. This will trigger a disturbance. Superimposed on benign samples (correspond Figure 3 Candidate locations of benign poisoning samples (in the sample) are generated with target category labels. Candidate poisoning samples Based on the classification loss between the predicted output of the local model for candidate poisoned samples and the target category label, the optimal insertion position is selected from each candidate position. The trained local model will identify the image with the optimal trigger perturbation as the target category, while maintaining accurate classification of benign image samples. The selection process for the optimal insertion position of the trigger perturbation is shown in Equation (1).

[0031] in, These are benign samples selected from the image sample set whose original labels are not the target category labels. It is the Hadamard product (i.e., element-wise multiplication). It is a predefined spatial mask used to specify the shape and location of the triggering perturbation. These are learnable trigger perturbation mode parameters. It is a truncation function that ensures the generated pixel values ​​are within the valid dynamic range of the image. Inside.

[0032] (2) Selection of the location to trigger disturbance To evaluate the attack effect of the triggered perturbation at different spatial locations, the original set of triggered perturbation pixels first needs to be normalized. Let the original set of triggered perturbation pixels be... ,in Indicates the first The coordinates of the triggering perturbation pixel in the image height direction. This represents its coordinates along the width direction of the image. The number of pixels included in the triggering perturbation is represented by the spatial mask. The number of matrix values ​​that are 1 is determined by the matrix itself.

[0033] To eliminate the influence of the original coordinate starting point on the position search, the triggered perturbation pixels are converted into relative coordinate form. As shown in the formula.

[0034] in, , , where represents the minimum coordinates of the original set of triggered perturbation pixels in the height and width directions, respectively.

[0035] The height and width of the triggering disturbance are respectively , Construct a set of candidate insertion positions within the range that satisfy the boundary constraints. As shown in the formula.

[0036] For any candidate position Construct candidate poisoning samples The remaining positions retain the original image. To evaluate the impact of different candidate insertion positions on the attack target, this invention constructs evaluation metrics from two aspects: feature space proximity and target category prediction tendency.

[0037] Set the current local model The feature extraction function is ,in This represents the local model parameters. To characterize the distribution features of the target category in the deep feature space, this invention first randomly selects samples belonging to the target category label from the image sample set. We analyze samples and extract deep feature representations for each sample. We then calculate the feature centers of the target category. As shown in the formula.

[0038] in, Indicates the selected first One sample, .

[0039] The feature center is the mean value calculated by the local model after extracting deep features from all samples belonging to the target category label in the image sample set. It is used to characterize the average distribution position of the target category label samples in the feature space of the local model.

[0040] The squared Euclidean distance metric is used to optimize the distance between the deep features of poisoned samples and the feature centers of the target class. As shown in the formula.

[0041] in, The smaller the value, the closer the sample that triggers the perturbation is to the center of the target class in the feature space.

[0042] To measure the tendency for optimized poisoned samples to be predicted as the target class label by the local model, a target class cross-entropy loss is introduced. As shown in the formula.

[0043] in, This represents the classification result output by the local model. Indicates the local model for the first Output values ​​for each category. The smaller the value, the higher the probability that the optimized poisoned sample will be identified as the target category label by the local model.

[0044] Triggering perturbation insertion location selection cost function The evaluation criteria are shown in the formula.

[0045] Therefore, the final selected trigger perturbation insertion position It can be obtained by minimizing the cost function among all legal candidate positions, as shown in the formula.

[0046] This optimization process enables the dynamic selection of the most aggressive insertion position, providing a high-quality sample base for subsequent data poisoning.

[0047] S22: Iteratively update the candidate triggering perturbations in the poisoned samples until the classification loss is minimized, obtaining the optimal triggering perturbation. Inject the optimal triggering perturbation into the poisoned samples to obtain optimized poisoned samples. Update the local model with the sum of the classification loss of the optimized poisoned samples and the classification loss of the benign samples as the objective, specifically including: With the current parameters of the local model fixed, a multi-step projective gradient descent algorithm is used to iteratively update the candidate triggering perturbations; After each update, the updated candidate triggering perturbation is projected into the preset perturbation budget constraint range, which is the Lp norm constraint on the triggering perturbation.

[0048] The classification loss of optimizing the prediction of poisoned samples as the target category label in the local model is used as the backdoor loss term; The classification loss of predicting benign samples as their original labels using the local model is called the benign loss term; The sum of the backdoor loss term and the benign loss term is used as the total loss. The parameters of the local model are then updated through backpropagation to obtain the updated local model.

[0049] like Figure 4 The diagram shows an iterative update strategy based on adaptive gradient updates, which specifically includes: (1) Dynamically triggered perturbation optimization To maximize the attack effectiveness of the generated backdoor in the local model state, this invention models the generation process of the trigger perturbation as a perturbation-related process. The local model optimizes the triggered perturbation by solving the problem through multi-step Projected Gradient Descent (PGD). This makes the local model suitable for benign poisoning samples. Predicted target category label The classification loss is minimized. The solution process is shown in the formula.

[0050] in, The parameter is The local model. Cross-entropy loss is a function used to measure the difference between the local model's predicted distribution and the target one-hot encoding. This represents the subset of training data used to generate the trigger perturbation.

[0051] To limit the amplitude of the triggered disturbance, it is necessary to control the disturbance. Apply feasible region Norm constraint. After each gradient update step, Project back to feasible region Set of constraints The definition is shown in the formula.

[0052] in, It is the maximum perturbation budget allowed, typically used Norms are used for constraints.

[0053] (2) Covert update After obtaining the optimal triggering perturbation, a composite loss function is used. Update local model parameters using optimized poisoned and benign samples.

[0054] Composite loss function It consists of the following three parts: the sum of the backdoor loss term and the benign loss term, plus the product of the parameter distance regularization term and the preset weight coefficient, as shown in the formula.

[0055] in, It's a backdoor loss. It is a benign loss. This is the distance parameter, a distance regularization term. It is the weight balance coefficient between classification loss and distance constraint.

[0056] The backdoor loss term represents the classification loss of the local model when it optimizes the prediction of poisoned samples as the target category label. The benign loss term represents the classification loss of the local model when it predicts benign samples as their original labels.

[0057] The total loss also includes the parameter distance regularization term. The current parameters of the local model and the global model parameters issued by the server in the previous round. The L2 distance between them; the product of the parameter distance regularization term and the preset weight coefficient is added to the total loss as a regularization term.

[0058] The local model update follows standard stochastic gradient descent (SGD), as shown in the formula.

[0059] in, The learning rate is used. The local model update process can strike a balance between attack effectiveness, main task performance, and parameter concealment.

[0060] In this embodiment, before submitting the updated local model to the server, the step of spoofing the gradients of the local model is also included: Obtain the gradient of the local model update and determine the unit direction vector of the gradient; A random noise vector is sampled from a Gaussian distribution, and the random noise vector is projected onto the direction of a unit direction vector to obtain the projection component; The projected components are superimposed on the gradient to obtain the spoofed gradient; this spoofed gradient is then used as the gradient of the updated local model and submitted to the server. While preserving the attack direction, the gradient magnitude is randomized, thereby disrupting the statistical correlation between malicious updates. For example... Figure 5 The image shows a gradient masking strategy, which specifically includes:

[0061] (1) Noise projection mechanism Let the gradient obtained from the backpropagation of the adaptive gradient update be the original gradient of the local model. To confine noise to the optimization direction determined by the adaptive gradient update, the unit direction vector of the original gradient is calculated. As shown in the formula.

[0062] Sample noise vectors from a Gaussian distribution. ,in For privacy budget parameters, It is the identity matrix. The noise vector... Projecting onto the optimization direction yields the alignment noise component. As shown in the formula.

[0063] The noise projection mechanism can keep the adaptive gradient update direction stable, while increasing the uncertainty of gradient statistical features, preventing the uploaded gradient from being filtered by robust aggregation mechanisms or anomaly detection algorithms.

[0064] (2) Gradient correction and statistical confusion Will Superimposed on the original gradient obtained from adaptive gradient update Above, construct the final corrected gradient used for submission. As shown in the formula.

[0065] Through the above transformation, the corrected gradient In direction relative to the original gradient Maintain a high degree of consistency ( This ensures that the descent direction of the attack objective function remains unchanged, and the magnitude of the gradient becomes... This results in significant randomness in the gradient of the updated local model.

[0066] The updated local model corresponding to the selected client in this round is uploaded to the server. The server aggregates and averages the updated local models and then distributes them to the selected client in the next round. This process continues until the aggregated global model is determined to have converged or has reached the predetermined number of training rounds, at which point the local model training ends.

[0067] In step S3, the test image sample with the trigger perturbation is input into the returned global model. Based on the deviation between the prediction result and the target category label, a robustness evaluation result of the federated learning system is generated, specifically including: Based on the prediction results of the returned global model for test image samples with trigger perturbations, the proportion of samples whose prediction results are the target category labels is counted and used as the evaluation metric. And obtain the classification accuracy of the returned global model on test image samples without trigger perturbation; The evaluation metrics and classification accuracy are used together as the output of the robustness evaluation results of the federated learning system.

[0068] This invention also proposes a robust security assessment system for federated learning systems, characterized by comprising: The data acquisition module is used to acquire the global model issued by the server in the federated learning system as the local model, acquire the training image sample set, determine the target category label, and take the samples in the training image sample set whose original label is not the target category label as benign samples. The local model update module is used to generate candidate poisoned samples by superimposing candidate trigger perturbations on benign samples. Based on the classification loss of predicting the candidate poisoned samples as the target class label by the local model, poisoned samples are selected from the candidate poisoned samples. The candidate trigger perturbations in the poisoned samples are iteratively updated until the classification loss is minimized to obtain the optimal trigger perturbation. The optimal trigger perturbation is injected into the poisoned samples to obtain optimized poisoned samples. The local model is updated with the sum of the classification loss of the local model for optimized poisoned samples and the classification loss for benign samples as the objective. The updated local model is submitted to the server and the global model returned by the server after aggregation is obtained. The robustness evaluation module takes test image samples with trigger perturbations as input to the returned global model and generates robustness evaluation results for the federated learning system based on the deviation between the prediction results and the target category labels.

[0069] The present invention also proposes a computer device, characterized in that the computer device includes a memory and a processor, the memory storing a computer program, and when the computer program is executed by the processor, the processor performs the above-described method steps.

[0070] The present invention also proposes a computer-readable storage medium, characterized in that the computer-readable storage medium stores a computer program, which, when executed by a processor, causes the processor to perform the above-described method steps.

[0071] This embodiment uses the attack success rate and the classification accuracy of the updated local model on benign image samples to evaluate the overall performance of the backdoor attack, wherein: Attack Success Rate (ASR): Measures the proportion of backdoor attack samples that the model misclassifies as the target label.

[0072] Classification accuracy (Clean Data Accuracy, CDA): Evaluates the model's ability to maintain performance on benign samples, and calculates the classification accuracy of the model on a clean test set.

[0073] The calculation definitions for ASR and CDA are shown in the formula.

[0074] in, This indicates an optimization of the poisoned sample set. Indicates the target category label. This represents a set of benign image samples. This indicates the actual label.

[0075] like Figure 6 As shown, the proposed method achieves high attack success rates on three datasets: MNIST, CIFAR-10, and GTSRB. The proposed method demonstrates higher ASR and stronger resistance to forgetting across different datasets and training epochs, with the most significant advantage on MNIST, where the ASR remains consistently high at 99.5%. This verifies that the proposed method possesses strong effectiveness and stealth in federated learning backdoor attacks, maintaining high attack performance even in continuously updated environments.

[0076] like Figure 7 The figure shows the classification accuracy of the proposed method on the MNIST, CIFAR-10, and GTSRB datasets. The proposed method achieves high attack success rate (ASR) while maintaining good classification accuracy (CDA) on all three datasets. Particularly on MNIST and CIFAR-10, the CDA is consistently maintained at high levels of 98.3% and 81.2%, respectively, with minimal loss in primary task performance. While there are initial fluctuations on GTSRB, the CDA reaches 92.6% in the later stages. This verifies that the proposed method has good primary task compatibility in federated learning backdoor attacks, achieving effective and persistent backdoor effects by slightly sacrificing clean sample classification accuracy.

[0077] like Figure 8The figure shows the training results of the local model provided in this embodiment on the MNIST, CIFAR-10, and GTSRB datasets. The first column of the figure represents the original samples, with the output being the original labels; the middle three columns represent the backdoor samples in every 30 training rounds; and the last column represents the final backdoor samples, with the output being the target labels. It can be observed that the insertion position of the perturbation trigger is inconsistent for different images, and the perturbation trigger for the same image is constantly changing. Although the time for successful backdoor injection varies for different training samples, they will all eventually be successfully injected into the backdoor.

[0078] Experimental results show that the method proposed in this invention can steadily improve the success rate of backdoor attacks while ensuring the normal classification performance of the model, demonstrating good persistence and stealth.

[0079] In summary, this invention screens poisoned samples from benign samples and iteratively optimizes the trigger perturbation, enabling the trigger perturbation and local model parameters to co-evolve; it optimizes the dual classification loss of poisoned and benign samples to constrain model updates, making the statistical characteristics of malicious updates approach those of benign updates; further, it interacts with the server to obtain the aggregated global model and detects its output deviation under trigger input to verify the continued effectiveness of the backdoor effect, ultimately achieving a continuous and stable evaluation of the robustness of the federated learning system.

[0080] The above description is only a preferred embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any equivalent substitutions or modifications made by those skilled in the art within the scope of the technology disclosed in the present invention, based on the technical solution and inventive concept of the present invention, should be covered within the scope of protection of the present invention.

[0081] Furthermore, unless otherwise stated, all technical and scientific terms used in this invention have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains. All references to this specification are incorporated by way of citation to disclose and describe methods relating to those references. In the event of any conflict with any incorporated reference, the content of this specification shall prevail.

Claims

1. A robust security assessment method for a federated learning system, characterized in that, Includes the following steps: The global model distributed by the server in the federated learning system is obtained as the local model, and the training image sample set is obtained. The target category label is determined, and the samples in the training image sample set whose original label is not the target category label are regarded as benign samples. Candidate poisoned samples are generated by superimposing candidate trigger perturbations on benign samples. Based on the classification loss of predicting the target category label of the candidate poisoned samples using the local model, poisoned samples are selected from the candidate poisoned samples. The candidate trigger perturbations in the poisoned samples are iteratively updated until the classification loss is minimized to obtain the optimal trigger perturbation. The optimal trigger perturbation is injected into the poisoned samples to obtain optimized poisoned samples. The local model is updated with the sum of the classification loss of the local model for optimized poisoned samples and the classification loss for benign samples as the objective. The updated local model is submitted to the server, and the global model returned by the server after aggregation is obtained. The test image samples with trigger perturbations are input into the returned global model, and the robustness evaluation results of the federated learning system are generated based on the deviation between the prediction results and the target category labels.

2. The robustness and security assessment method for a federated learning system according to claim 1, characterized in that, The process of screening poisoned samples from candidate poisoned samples specifically includes: Extract deep features of each candidate poisoning sample from the local model; The classification loss of the local model in predicting each candidate poisoned sample as the target category label is obtained; Determine the distance between the deep features of each candidate poisoning sample and the feature center of the target category, and use the weighted sum of distance and classification loss as the evaluation cost; The candidate poisoning sample with the lowest evaluation cost will be selected as the poisoning sample. Wherein, the target category feature center is the mean value of the deep features extracted by the local model from the samples belonging to the target category label in the training image sample set.

3. The robustness and security assessment method for a federated learning system according to claim 1, characterized in that, The local model is updated with the goal of summing the classification loss of the local model for poisoned samples and the classification loss for benign samples. This update specifically includes: The classification loss of optimizing the prediction of poisoned samples as the target category label in the local model is used as the backdoor loss term; The classification loss of predicting benign samples as their original labels using the local model is called the benign loss term; The sum of the backdoor loss term and the benign loss term is used as the total loss. The parameters of the local model are then updated through backpropagation to obtain the updated local model.

4. The robustness and security assessment method for a federated learning system according to claim 3, characterized in that, The total loss also includes a parameter distance regularization term, which is the L2 distance between the current parameters of the local model and the global model parameters issued by the server in the previous round; the product of the parameter distance regularization term and the preset weight coefficient is added to the total loss as a regularization term.

5. The robustness and security assessment method for a federated learning system according to claim 1, characterized in that, The iterative update of candidate triggering perturbations in the poisoned samples continues until the classification loss is minimized, thus obtaining the optimal triggering perturbation. This specifically includes: With the current parameters of the local model fixed, the candidate triggering perturbations are iteratively updated using a multi-step projective gradient descent algorithm. After each update, the updated candidate triggering perturbation is projected onto a preset perturbation budget constraint, which is an Lp norm constraint on the triggering perturbation.

6. The robustness and security assessment method for a federated learning system according to claim 1, characterized in that, Before submitting the updated local model to the server, the process further includes a step of masquerading the gradients of the local model: Obtain the gradient of the local model update and determine the unit direction vector of the gradient; A random noise vector is sampled from a Gaussian distribution, and the random noise vector is projected onto the direction of a unit direction vector to obtain the projection component; The projected components are superimposed on the gradient to obtain the dummy gradient; the dummy gradient is then used as the gradient of the updated local model and submitted to the server.

7. The robustness and security assessment method for a federated learning system according to claim 1, characterized in that, The generation of robustness evaluation results for the federated learning system based on the deviation between the predicted results and the target category label specifically includes: Based on the prediction results of the returned global model for test image samples with trigger perturbations, the proportion of samples whose prediction results are the target category labels is counted and used as an evaluation index. And obtain the classification accuracy of the returned global model on test image samples without trigger perturbation; The evaluation metrics and classification accuracy are used together as the output of the robustness evaluation results of the federated learning system.

8. A robust security assessment system for a federated learning system, characterized in that, include: The data acquisition module is used to acquire the global model issued by the server in the federated learning system as the local model, acquire the training image sample set, determine the target category label, and take the samples in the training image sample set whose original label is not the target category label as benign samples. The local model update module is used to generate candidate poisoned samples by superimposing candidate trigger perturbations on benign samples; to predict the candidate poisoned samples as the target category label based on the classification loss of the local model; to screen poisoned samples from the candidate poisoned samples; to iteratively update the candidate trigger perturbations in the poisoned samples until the classification loss is minimized, thus obtaining the optimal trigger perturbation; to inject the optimal trigger perturbation into the poisoned samples, thus obtaining optimized poisoned samples; to update the local model with the sum of the classification loss of the local model for optimized poisoned samples and the classification loss for benign samples as the objective; and to submit the updated local model to the server and obtain the global model returned by the server after aggregation. The robustness evaluation module is used to input test image samples with trigger perturbations into the returned global model, and generate robustness evaluation results of the federated learning system based on the deviation between the prediction results and the target category labels.

9. A computer device, characterized in that, The computer device includes a memory and a processor. The memory stores a computer program, which, when executed by the processor, causes the processor to perform the following steps: The global model distributed by the server in the federated learning system is obtained as the local model, and the training image sample set is obtained. The target category label is determined, and the samples in the training image sample set whose original label is not the target category label are regarded as benign samples. Candidate poisoned samples are generated by superimposing candidate trigger perturbations on benign samples. Based on the classification loss of predicting the target category label of the candidate poisoned samples using the local model, poisoned samples are selected from the candidate poisoned samples. The candidate trigger perturbations in the poisoned samples are iteratively updated until the classification loss is minimized to obtain the optimal trigger perturbation. The optimal trigger perturbation is injected into the poisoned samples to obtain optimized poisoned samples. The local model is updated with the sum of the classification loss of the local model for optimized poisoned samples and the classification loss for benign samples as the objective. The updated local model is submitted to the server, and the global model returned by the server after aggregation is obtained. The test image samples with trigger perturbations are input into the returned global model, and the robustness evaluation results of the federated learning system are generated based on the deviation between the prediction results and the target category labels.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, causes the processor to perform the following steps: The global model distributed by the server in the federated learning system is obtained as the local model, and the training image sample set is obtained. The target category label is determined, and the samples in the training image sample set whose original label is not the target category label are regarded as benign samples. Candidate poisoned samples are generated by superimposing candidate trigger perturbations on benign samples. Based on the classification loss of predicting the target category label of the candidate poisoned samples using the local model, poisoned samples are selected from the candidate poisoned samples. The candidate trigger perturbations in the poisoned samples are iteratively updated until the classification loss is minimized to obtain the optimal trigger perturbation. The optimal trigger perturbation is injected into the poisoned samples to obtain optimized poisoned samples. The local model is updated with the sum of the classification loss of the local model for optimized poisoned samples and the classification loss for benign samples as the objective. The updated local model is submitted to the server, and the global model returned by the server after aggregation is obtained. The test image samples with trigger perturbations are input into the returned global model, and the robustness evaluation results of the federated learning system are generated based on the deviation between the prediction results and the target category labels.

Images