Robust Federated Learning Method for Image Classification

Abstract

The present invention discloses a robust federated learning method for image classification, belonging to the technical field of machine vision, and comprising the following steps: S1, screening each training client and initializing the global model parameters; S2, obtaining the simulated augmented data of each training client; S3, obtaining the corrected feature vectors of each training client; S4, obtaining the basic loss of local model training, and thereby retraining the local models of each training client; S5, obtaining the noisy local models of each training client, thereby obtaining the local updated models of each training client and uploading them to the server; S6, the server receives the local updated models of each training client, obtains the global updated model, and repeats the operations of S2 to S6 until the global updated model converges. It solves the problem of the decrease in model accuracy caused by data anomalies in the prior art under the condition of uneven data distribution.

Description

Technical Field

[0001] The present invention relates to the technical field of machine vision, and particularly to a robust federated learning method for image classification. Background Art

[0002] With the development of the big data era, machine learning technology is widely used, and federated learning is often adopted to ensure the privacy and security of its data. Federated learning mainly realizes data privacy protection through steps such as the server sending model updates to clients, clients receiving model updates and performing local training of the model, and the server aggregating local model updates of clients to obtain a new global model.

[0003] In practical applications, the data characteristics and distributions of each client vary greatly, showing a state of unbalanced data distribution (non-iid). Unbalanced data distribution will affect the accuracy of the model. Some existing methods use shared mapping spaces, global feature dictionaries, and data distribution statistical information to adjust local training or global aggregation to solve the non-iid problem, but ignore that the deviation of data statistics caused by data anomalies will affect the model performance and update direction.

[0004] In the federated learning framework, clients may be attacked by attackers, resulting in abnormal data, which will affect the accuracy of the model and even cause the model to deviate seriously from the normal update direction. Some other existing methods reduce the impact of updates obtained from data anomalies on the global model by removing abnormal updates or adjusting the weights of abnormal updates, but this may lead to the loss of data information of some categories or be unsuitable for the non-iid situation, thus affecting the model accuracy. Therefore, in the case of uneven data distribution, there is a problem of decreased model accuracy caused by the occurrence of data anomalies.

[0005] For example, a federated learning image classification method capable of defending against backdoor attacks announced in the invention patent announcement with the publication number: CN114494771B includes: processing the gradients submitted by Worker nodes by using matrix dimensionality reduction and clustering algorithms, and finally selecting the gradients submitted by normal Worker nodes to participate in aggregation, thus completely avoiding the possibility of backdoor implantation into the global model. Summary of the Invention

[0006] By providing a robust federated learning method for image classification in the embodiments of the present application, the problem in the prior art that in the case of uneven data distribution, the model accuracy decreases due to the occurrence of data anomalies is solved, and the classification accuracy of local models and the quality of feature extraction are improved in the federated learning environment.

[0007] The embodiments of the present application provide a robust federated learning method for image classification, including the following steps: S1. The server filters out the clients participating in the training through a dynamic client screening strategy and marks them as each training client. The server initializes the global model to obtain an initialized global model and distributes it to each training client; S2. Obtain the local image data of each training client, and perform noise simulation processing and data enhancement preprocessing to obtain the simulated enhanced data of each training client; S3. Based on the simulated enhanced data of each training client, train the local model of each training client to obtain the corrected feature vectors of each training client; S4. Perform adaptive contrast learning based on the corrected feature vectors of each training client, analyze to obtain the contrast loss of each training client, obtain the cross-entropy loss of each training client, and jointly analyze with the contrast loss to obtain the basic loss of local model training, thereby performing retraining of the local model of each training client; S5. Introduce random Gaussian noise into the simulated enhanced data of each training client to obtain the contaminated data of each training client, and obtain the noisy local model of each training client based on the contaminated data of each training client, thereby analyzing to obtain the local updated model of each training client and uploading it to the server; S6. The server receives the local updated models of each training client and analyzes to obtain the global updated model; Repeat the operations of S2 to S6 until the global updated model converges.

[0008] One or more technical solutions provided in the embodiments of the present application have at least the following technical effects or advantages:

[0009] 1. The robust federated learning method for image classification provided by the present invention performs adaptive contrast learning based on the corrected feature vectors of each training client, analyzes to obtain the contrast loss of each training client, thereby obtaining the cross-entropy loss of each training client, and jointly analyzes with the contrast loss to obtain the basic loss of local model training, thereby realizing the improvement of the classification accuracy and feature extraction quality of the local model in the federated learning environment, and effectively solving the problem that the model accuracy decreases due to data anomalies in the prior art when the data distribution is uneven;

[0010] 2. The present invention analyzes to obtain the ideal contribution effect value of each client by obtaining the computing resource parameters, data quality parameters of each client and the historical contribution coefficient of each client, thereby screening out the clients with the ideal contribution effect value higher than the threshold to participate in the training, thereby realizing the improvement of the training efficiency and model performance, and effectively solving the problem that the model accuracy decreases due to data anomalies in the prior art when the data distribution is uneven;

[0011] 3. The present invention respectively obtains the current-round feature vectors and the previous-round feature vectors of each training client at a predetermined intermediate layer, analyzes the cosine similarity of adjacent rounds of each training client, and thus analyzes the dynamic feature weights of each training client based on the cosine similarity of adjacent rounds of each training client, thereby optimizing the quality of model feature extraction and improving the classification accuracy and robustness of the model;

[0012] 4. The present invention obtains the noisy local models of each training client based on the contaminated data of each training client, and thus performs meta-learning strategy training based on the noisy local models of each training client, thereby enhancing the adaptability and noise resistance of the model to noise data and improving the stability and performance of the model in a complex data environment;

[0013] 5. The present invention introduces random Gaussian noise into the simulated enhanced data of each training client to obtain the contaminated data of each training client, and thus obtains the noisy local models of each training client based on the contaminated data of each training client, thereby analyzing the local update models of each training client and uploading them to the server, thereby realizing the improvement of the noise resistance performance of the model by constructing a noisy data environment and reducing the impact of data anomalies on the global model update. BRIEF DESCRIPTION OF THE DRAWINGS

[0014] Figure 1 is a flowchart of a robust federated learning method for image classification provided by an embodiment of the present application;

[0015] Figure 2 is a schematic diagram of the model accuracy effect provided by an embodiment of the present application under the CIFAR-10 dataset;

[0016] Figure 3 is a schematic diagram of the model accuracy effect provided by an embodiment of the present application under the CIFAR-100 dataset;

[0017] Figure 4 is a schematic diagram of the model accuracy effect provided by an embodiment of the present application under the Street View House Numbers (SVHN) dataset;

[0018] Figure 5 is a schematic diagram of the model accuracy effect of the CIFAR-10 dataset under different proportions of attacked clients provided by an embodiment of the present application;

[0019] Figure 6 is a schematic diagram of the model accuracy effect of the CIFAR-100 dataset under different proportions of attacked clients provided by an embodiment of the present application;

[0020] Figure 7Schematic diagram of the accuracy of the Street View HouseNumbers (SVHN) dataset model under different proportions of attacked clients provided by the embodiments of the present application;

[0021] Figure 8 Schematic diagram of the influence of different components on the model performance provided by the embodiments of the present application. Detailed implementation manners

[0022] By providing a robust federated learning method for image classification, the embodiments of the present application solve the problem of the decrease in model accuracy caused by data anomalies in the prior art under the condition of uneven data distribution, and achieve the improvement of the classification accuracy and feature extraction quality of local models in the federated learning environment.

[0023] To better understand the above technical solutions, the above technical solutions will be described in detail below in conjunction with the accompanying drawings of the specification and specific implementation manners.

[0024] As Figure 1 shown, it is a flowchart of a robust federated learning method for image classification provided by the embodiments of the present application. The method includes the following steps: S1. The server screens out the clients participating in the training through a dynamic client screening strategy and marks them as each training client. The server initializes the global model to obtain an initialized global model and distributes it to each training client; S2. Obtain the local image data of each training client, and perform noise simulation processing and data augmentation preprocessing to obtain the simulated augmented data of each training client; S3. Train the local models of each training client based on the simulated augmented data of each training client to obtain the corrected feature vectors of each training client; S4. Perform adaptive contrast learning based on the corrected feature vectors of each training client, analyze to obtain the contrast loss of each training client, obtain the cross-entropy loss of each training client, and jointly analyze with the contrast loss to obtain the basic loss of local model training, and thus perform retraining of the local models of each training client; S5. Introduce random Gaussian noise into the simulated augmented data of each training client to obtain the contaminated data of each training client, obtain the noisy local models of each training client based on the contaminated data of each training client, and thus analyze to obtain the local updated models of each training client and upload them to the server; S6. The server receives the local updated models of each training client and analyzes to obtain the global updated model; Repeat the operations of S2 to S6 until the global updated model converges.

[0025] In this embodiment, it should be noted that the clients marked as each training client are benign clients. Among them, benign clients (clients not under attack, that is, clients without data anomalies) can perform steps S2 to S5; clients under attack are malicious clients.

[0026] The operations in the contrastive learning part mainly calculate the cosine similarity between the data features of the previous round and the current round, adjust the local data features using the similarity, reduce the impact of noise on the model, use the contrastive learning method to aggregate the data features of the same class, and improve the model accuracy and convergence speed. The operations in the meta-learning part mainly add noise to the original data randomly, train to obtain a noise model, and use meta-learning to learn the knowledge of the noise model into the normal model to improve the model accuracy under data imbalance and data anomalies. It should be noted that the original data refers to the local image data of each training client.

[0027] Through experimental tests, using the framework of this application, compared with the existing methods, the model accuracy and the model convergence speed are improved in the presence of non-iid and data anomalies. Example: In the case of non-independent and identically distributed with a parameter of 0.5 on CIFAR-10, when the proportion of attacking clients is 0.2, 0.5, and 0.8, compared with the baseline method, the average accuracy of FedCM (robust federated learning architecture) is increased by 8.2%, 7.9%, and 4.6% respectively; it shows obvious advantages on the SVHN (Street View House Numbers) dataset, and the number of training rounds required for convergence is reduced by about 34.78% compared with other methods.

[0028] It should be noted that each training client receives the global model and simulates the noise environment after the client is attacked through methods such as Gaussian noise, shot noise, impulse noise, glass blur, motion model, zoom blur, fog effect, frost effect, brightness adjustment, contrast adjustment, pixelation, elastic transformation, and distortion effect to generate a noisy dataset. At the same time, through data augmentation strategies such as random flipping, cropping, and brightness adjustment, the data diversity is improved, the generalization ability of the model is enhanced, and the stable classification performance can be maintained in complex environments.

[0029] It should also be noted that the generated noisy dataset is used to simulate the abnormal data generated when the client is attacked.

[0030] Further, the server filters out the clients participating in the training through a dynamic client screening strategy and marks them as each training client. The specific steps are as follows: Obtain the computing resource parameters, data quality parameters, and historical contribution coefficients of each client, and analyze to obtain the contribution ideal effect value of each client; Obtain the preset contribution ideal effect threshold in the database and compare it with the contribution ideal effect value of each client. If the contribution ideal effect value of a certain client is above the contribution ideal effect threshold, then use this client as the training client; Allocate the global model of the server to each training client, and train the model of the global model sent by the server based on the image sample data set of each training client to obtain the local model of each training client; The computing resource parameters include the average task execution duration and the maximum number of concurrent tasks; The data quality parameters include the proportion of abnormal data, data update frequency, and available data sample size; The contribution ideal effect value of each client is obtained by differentiating the computing resource parameters and data quality parameters of each client from the average level of the client, then introducing the corresponding influence weights, and finally performing a coupling process with the historical contribution coefficient of each client to obtain the contribution ideal effect value of each client.

[0031] In this embodiment, the method for obtaining the contribution ideal effect value of each client is as follows:

[0032] ;

[0033] ;

[0034] ;

[0035] In the formula, represents the contribution ideal effect value of the m-th client, represents the historical contribution coefficient of the m-th client, m represents the client number, , represents the total number of clients, represents the average task execution duration of the m-th client, represents the maximum number of concurrent tasks of the m-th client, represents the proportion of abnormal data of the m-th client, represents the data update frequency of the m-th client, represents the available data sample size of the m-th client, represents the influence weight of the task execution duration, represents the influence weight of the number of concurrent tasks, represents the influence weight of the proportion of abnormal data, represents the influence weight of the data update frequency, represents the influence weight of the available data sample size.

[0036] It should be noted that by analyzing the computing resource parameters, data quality parameters, and historical contribution coefficients of each client, the ideal contribution effect value of each client is obtained by considering the mutual influence relationships among these parameters. For example, clients with shorter average task execution times usually have more abundant computing resources, can handle more tasks simultaneously, and thus support a larger maximum number of concurrent tasks. Clients with a high data update frequency usually have more timely and accurate data, so the proportion of abnormal data is relatively low. Clients with a high data update frequency tend to accumulate more data samples and thus have a larger available data sample size. Clients with a high historical contribution coefficient usually showed good task execution capabilities and stability in the past, and thus have a shorter average task execution time and a larger maximum number of concurrent tasks.

[0037] A shorter average task execution time indicates that the client can complete tasks more efficiently and thus is closer to the ideal contribution effect value. A larger maximum number of concurrent tasks enables the client to handle more tasks simultaneously, improving resource utilization and task processing efficiency, and helping to approach the ideal contribution effect value. A lower proportion of abnormal data indicates higher data quality of the client, better model training effects, and being closer to the ideal contribution effect value. A higher data update frequency indicates that the client's data is more timely and accurate, and the model can be trained and optimized based on the latest data, being closer to the ideal contribution effect value. A larger available data sample size provides a richer data basis for model training, helps improve the generalization ability and accuracy of the model, and is closer to the ideal contribution effect value. A higher historical contribution coefficient reflects the positive contribution of the client to model training in the past, indicating that it performs well in aspects such as computing resources, data quality, and task execution capabilities.

[0038] Furthermore, based on the simulated enhanced data of each training client, the local models of each training client are trained to obtain the corrected feature vectors of each training client. The specific steps include: respectively obtaining the current-round feature vectors and the previous-round feature vectors of each training client at a predetermined intermediate layer, and analyzing the cosine similarity of adjacent rounds of each training client; analyzing the dynamic feature weights of each training client based on the cosine similarity of adjacent rounds of each training client; based on the dynamic feature weights of each training client, using the previous-round feature vectors to correct the current-round feature vectors obtained by each training client to obtain the corrected features of each training client. The dynamic feature weights of each training client are obtained by performing differential analysis on the current-round feature vectors and the previous-round feature vectors of each training client to obtain the dynamic feature weights of each training client.

[0039] In this embodiment, it should be noted that the predetermined intermediate layer is, for example, the last convolutional layer or the fully connected layer.

[0040] Obtain the cosine similarity of adjacent rounds of the training client. The specific method is as follows:

[0041] ;

[0042] In the formula, represents the cosine similarity of adjacent rounds of the training client, represents the similarity between the feature vector of the current round and the feature vector of the previous round, represents the feature vector of the current round, represents the feature vector of the previous round, represents the matrix norm of the feature vector of the current round, represents the matrix norm of the feature vector of the previous round.

[0043] By aggregating the cosine similarities of adjacent rounds of the training client, the cosine similarities of adjacent rounds of each training client can be obtained.

[0044] Obtain the dynamic feature weight of the training client. The specific method is as follows:

[0045] ;

[0046] In the formula, represents the dynamic feature weight of the training client, represents the cosine similarity of adjacent rounds of the training client.

[0047] It should be noted that, among them, when , it indicates that the feature similarity is very low. This situation rarely occurs, and it means that the direction of the feature after the local model update seriously deviates from the direction of the feature of the previous round model. Therefore, only the feature of the local model is considered at this time; when , although the similarity is relatively low, the current local model is mainly considered. Therefore, a larger proportional weight should be assigned to the local model; when , the similarity is relatively high, but since the current local model is more affected by noise, the local model of the previous round is still needed for correction.

[0048] By aggregating the dynamic feature weights of the training client, the dynamic feature weights of each training client can be obtained.

[0049] Obtain the corrected feature of the training client. The specific method is as follows:

[0050] ;

[0051] In the formula, represents the corrected feature of the training client, Represents the dynamic feature weight of the training client, Represents the feature vector of the current round, Represents the feature vector of the previous round.

[0052] Based on the corrected features of the training client for summarization, the corrected features of the training client can be obtained.

[0053] Furthermore, based on the corrected feature vectors of each training client for adaptive contrast learning, the contrast losses of each training client are analyzed. The specific method is as follows: Based on the corrected feature vectors of each training client, perform adaptive contrast learning for each training client, calculate the cosine similarity between the sample and all its positive samples; calculate the cosine similarity between the sample and all its contrast samples; for each sample, calculate its similarity with the positive samples and normalize it through the Softmax distribution to convert the similarities of all positive and negative samples into a probability distribution; calculate the negative log-likelihood, and maximize the similarity of the positive samples in the Softmax distribution to obtain the contrast losses of each training client.

[0054] In this embodiment, based on the corrected feature vectors of each training client for adaptive contrast learning, the contrast losses of each training client are analyzed. The specific method is as follows: Obtain the corrected feature vectors of the training client for adaptive contrast learning, analyze the contrast losses of the training client, and summarize the contrast losses of the training client to obtain the contrast losses of each training client;

[0055] The contrast losses of the training client are obtained. The specific method is as follows:

[0056] ;

[0057] In the formula, Represents the contrast loss of the training client, Represents the sample set of the i-th training client, where i represents the number of the training client, , Represents the total number of training clients, j represents the contrast sample, Represents the set of positive samples with the same class as the contrast sample j, Represents the number of samples with the same class as the sample j in the sample set of the i-th client, Represents all local samples except the sample j, Represents the corrected feature vector of the contrast sample, k represents the same-class sample, Represents the corrected feature vector of the same-class sample, a represents any contrast sample except the sample j, Represents the corrected feature vector of any contrast sample, Represents a temperature parameter used to adjust the smoothness of the similarity distribution. Represents The cosine similarity between and The cosine similarity between

[0058] It should be noted that the comparison samples include positive samples and negative samples, and the positive samples are samples of the same category. Calculate the cosine similarity between a sample and all its comparison samples, where all comparison samples are other samples in the same batch. The temperature parameter can be obtained by retrieving the temperature parameter after expert statistical experiments in the database.

[0059] It should be noted that ; .

[0060] Furthermore, obtain the cross-entropy loss of each training client and jointly analyze it with the comparison loss to obtain the basic loss for local model training. The specific method is as follows: Use the cross-entropy loss to measure the matching degree between the predicted values of the local models of each training client and the true labels; Calculate the prediction probability for each image sample data in the image sample dataset of each training client and calculate the cross-entropy loss between its distribution and the true labels to obtain the cross-entropy loss of each training client; Based on the cross-entropy loss of each training client, add the comparison loss and minimize the cross-entropy loss and the comparison loss of each training client to obtain the basic loss for local model training.

[0061] In this embodiment, the cross-entropy loss is:

[0062] ;

[0063] In the formula, Represents the cross-entropy loss, Represents the size of the dataset of the i-th training client, i represents the number of the training client, , Represents the total number of training clients, Represents the input sample data, Represents the true label corresponding to the sample, Represents the predicted probability distribution generated by the model for the input sample x, Represents the model parameters of each training client;

[0064] ;

[0065] In the formula, Represents the total loss of the local model of the client, Represents the comparison loss of the training client, Represents the cross - entropy loss.

[0066] Furthermore, random Gaussian noise is introduced into the simulated augmented data of each training client to obtain contaminated data for each training client. The specific steps are as follows: Obtain a part of the data in the simulated augmented data of each training client, and obtain the sample data information and its corresponding sample labels in this part of the data; Set the Gaussian distribution conditions, where the Gaussian distribution conditions are set such that the mean of the Gaussian distribution is zero, and the random number of the Gaussian distribution is set as the standard deviation of the Gaussian distribution; Based on the simulated augmented data of each training client and the Gaussian distribution conditions, introduce random Gaussian noise to obtain the contaminated data for each training client.

[0067] In this embodiment, first, random Gaussian noise is used to add noise to a part of the data in the simulated augmented data of each training client to obtain the contaminated data for each training client. The specific method is as follows:

[0068] ;

[0069] In the formula, represents the contaminated data for each training client, x is the sample data, y is the true label, is the Gaussian distribution, represents the input sample after being perturbed by Gaussian noise, m represents the mean of the Gaussian distribution, represents the standard deviation of the Gaussian distribution.

[0070] It should be noted that the local dataset of each training client is marked as Di, where i represents the number of the training client, , represents the total number of training clients. Select a small batch of data and mark it as Bi, Bi = {(x, y)}. Let the Gaussian distribution be g(m, std), where m is the mean of the Gaussian distribution, set to 0 to avoid excessive deviation of the data, resulting in the model being overly biased towards the noise direction and causing performance degradation, and std is the standard deviation of the Gaussian distribution, set as a random number.

[0071] Furthermore, based on the contaminated data of each training client, a noisy local model for each training client is obtained. The specific method is as follows: Based on the contaminated data of each training client, perform model training on the local model of each current training client, and use the stochastic gradient descent method to update the local model parameters of each training client to obtain a noisy local model for each training client; Output the noisy local model of each training client.

[0072] In this embodiment, the noisy small - batch data is used to train the original local model (the local model of each training client) using the traditional cross - entropy loss, and the stochastic gradient descent is used to update the model parameters:

[0073] ;

[0074] In the formula, represents the cross-entropy loss of the noisy data of each training client, represents the input sample after Gaussian noise perturbation, represents the true label, represents the model parameters of the i-th training client in the current training round, represents the cross-entropy loss of the training client, represents the dataset of the mini-batch noisy data of the i-th training client, represents the size of the mini-batch of noisy data, and i represents the number of the training client, , represents the total number of training clients;

[0075] ;

[0076] In the formula, represents the model parameters of the i-th training client after being updated by training with noisy data, represents the model parameters of the i-th training client in the current training round, represents the learning rate, represents the cross-entropy loss with respect to the model parameters gradient, and i represents the number of the training client, , represents the total number of training clients.

[0077] Further, to obtain the local updated models of each training client, the specific method is as follows: Obtain the original data of each training client and input it into the noisy local model of the corresponding training client to obtain the predicted probability distribution of the noisy prediction data of each training client. Input the predicted probability distribution of the noisy prediction data of each training client into the local model of the corresponding training client to obtain the predicted probability distribution of the original prediction data of each training client; Based on the predicted probability distribution of the noisy prediction data of each training client and the predicted probability distribution of the original prediction data of each training client, construct the cross-entropy loss by combining the predicted probability distribution of the noisy prediction data of each training client with the original data label as the meta-loss; Obtain the original data of each training client and input it into the global model to obtain the global data predicted probability distribution of each training client; Based on the predicted probability distribution of the noisy prediction data of each training client and the global data predicted probability distribution of each training client, analyze and obtain the KL divergence of each training client; Based on the coupling process of the KL divergence of each training client and the meta-loss, obtain the total meta-learning loss of each training client; Based on the total meta-learning loss of each training client and the basic loss of the local model training of each training client, jointly train the local model of each training client to obtain the local models of each training client in this round.

[0078] In this embodiment, meta-update of the local model is further performed based on the noisy local model of each training client. The noisy local model is used to guide the model training to obtain noise-resistant model parameters. Using the noisy local model, a correct prediction y is obtained for the original data x, thereby optimizing the model parameters. The specific method is as follows:

[0079] ;

[0080] In the formula, represents the meta-learning cross-entropy loss, i represents the number of the training client, , represents the total number of training clients, represents the mini-batch original data of the i-th training client, represents the cross-entropy loss of the training client, represents the output probability predicted by the noisy local model, represents the model parameters of the i-th training client after being trained and updated with the noisy data.

[0081] The global model is used to fine-tune the noisy local model to improve the problem of non-independent and identically distributed data. The loss of the global model and the noisy local model is:

[0082] ;

[0083] In the formula, denotes the knowledge distillation loss, denotes the mini-batch original data of the $i$-th training client, where $i$ represents the number of the training client, , denotes the total number of training clients, denotes the original data, denotes the global model, denotes the global model parameters, denotes the model parameters of the $i$-th training client after being trained and updated with noisy data, is the KL divergence between the probability distributions $p$ and $q$, denotes the global model with the global model parameters and the predicted class probability distribution vector obtained by the forward propagation of the global model on the original data $x$.

[0084] Therefore, the total loss of the meta-learning part is: ; where, denotes the total loss of the meta-learning part, denotes the meta-learning cross-entropy loss, denotes the knowledge distillation loss. Finally, the local model is jointly trained using the basic loss of the local model training and the total loss of the meta-learning part to obtain the local model of this round.

[0085] Furthermore, the server receives the local updated models of each training client and analyzes to obtain the global updated model. The specific method is as follows: identify the attacked clients and mark them as malicious clients, simulate the abnormal data generated by the malicious clients being attacked by adding noise to obtain the data sets of each malicious client, and perform data augmentation preprocessing to obtain the simulated and augmented data of each malicious client; each malicious client selected by the dynamic client screening strategy in each round obtains the global model sent by the server and performs cross-entropy loss training to obtain the local updated model of each malicious client; obtain the data set sizes of each training client and each malicious client selected in each round, and perform aggregation processing on the local updated models of each training client and each malicious client selected in each round received by the server through the FedAvg aggregation method to obtain the global updated model.

[0086] In this embodiment, the server receives the local updated models of each training client and analyzes to obtain the global updated model. Specifically, the server receives the local updated models of each training client participating in this round, and uses the FedAvg aggregation method to aggregate the local updated models of each training client, the local updated models of each malicious client, and the data sets of each malicious client to obtain the global model;

[0087] ;

[0088] where, denote the global model parameters of the round, denote the global model parameters of the round, where \(t\) is the current round number, is the dataset of the \(z\)-th client, is the sum of the datasets of each training client and each malicious client selected in each round, , denote the total number of clients.

[0089] It should be noted that the clients include each malicious client and each training client.

[0090] It should also be noted that the analysis method of the cross-entropy loss of each malicious client is exactly the same as that of the cross-entropy loss of each training client.

[0091] Furthermore, the globally updated model is obtained by repeating the operations of S2 to S6 until the globally updated model converges. The specific method is as follows: repeat the operations of S2 to S6. After each round of aggregation, evaluate the classification accuracy of the global model on the validation set; if the accuracy fluctuation for consecutive preset rounds is lower than the preset fluctuation ratio, or reaches the preset maximum number of rounds, it is determined that the globally updated model converges.

[0092] In this embodiment, the method proposed in the present invention is compared with several baseline models on three datasets (CIFAR-10, CIFAR-100, and Street View House Numbers). The performance of the present invention under data imbalance and the presence of data anomalies, as well as the model robustness under different degrees of data non-independent and identically distributed and data anomaly, are mainly analyzed. Among them, the degree of data anomaly mainly includes two factors: the proportion of attacked clients and the proportion of contaminated samples. Accuracy is used as the evaluation index.

[0093] The experimental results show that the method proposed in the present invention has improved model performance compared with the comparative methods under different degrees of non-iid and data anomaly on the three datasets. As Figures 2 - 4 shown, under the default experimental settings (the proportion of attacked clients \(a_r = 0.2\), the proportion of contaminated samples \(n_r = 0.8\), non-iid (\(\beta = 0.5\))), the accuracies of each method are compared and analyzed to evaluate the performance of the method in this paper. As Figure 2 and Figure 3As shown, on the CIFAR-10 and CIFAR-100 datasets, the model accuracy of the FedCM method is higher than that of all comparison methods, with increases of 8.2% and 6.1% respectively (the specific values can be seen in Table 1, when β = 0.5), so it shows that the FedCM method has good accuracy under non-IID and the existence of data anomalies. On the SVHN dataset, although the highest accuracy of FedCM is similar to that of the comparison methods, it can be seen from the first 10 rounds that the accuracy improvement speed of the FedCM method is faster than that of other methods, and it is basically close to convergence at the 15th round, while other methods are close to convergence at the 23rd round. It can be seen that the FedCM method improves the convergence speed of the model. Since the contrastive learning part based on improved features of the FedCM method makes the features of similar samples more concentrated, the model's prediction of categories is more accurate; and from Figure 4 it can be seen that FedCM fluctuates less in the first twenty rounds. Because in contrastive learning, the local model of the previous round is used to improve the features, reducing the interference of noise information, so the model is less affected by noise; it shows that the model performance of the FedCM method is better than that of other methods, and further shows that the contrastive learning part based on improved features is effective and maintains stability. In short, under non-IID and the existence of data anomalies, FedCM has good accuracy and the convergence speed is improved.

[0094] The accuracy for different degrees of non-IID. As shown in Table 1, FedCM has the best model accuracy under different degrees of non-IID (β = 0.25, 0.5, 1, 10), indicating that FedCM still maintains good accuracy for data with a higher degree of non-IID. Overall, when β changes from 0.5 to 0.25, the degree of decrease in the accuracy of FedCM is lower than that of the comparison methods, indicating that FedCM is less sensitive to the degree of non-IID. Due to the meta-learning part of the FedCM based on the Gaussian noise model parameters, the global model is used to adjust the update direction of the noise-resistant model, which can reduce the deviation of the local model from the global model during non-IID local model updates, so that a local model that is more in line with the global model update direction can be trained, improving generalization. As the degree of non-IID decreases, the accuracy of the FedCM method also increases, indicating that the method of this application is suitable for both non-IID and IID. On the CIFAR-10, CIFAR-100, and SVHN datasets, FedCM is at least 2.7%, 4.5%, and 0.58% higher than the comparison methods respectively, indicating that FedCM has good model accuracy under different degrees of non-IID.

[0095] Table 1: Comparison of the accuracy of each method under different degrees of non-IID:

[0096]

[0097] Robustness to the proportion of attacked clients. As Figures 5 - 7 shown, compared with the comparison methods, FedCM has the best average model accuracy in all cases of the proportion of attacked clients (ar = 0.2, 0.5, 0.8). On cifar-10 in the non-iid (β = 0.5) case, when the proportion of attacked clients is 0.2, 0.5, and 0.8, FedCM improves the average accuracy of the comparison methods by 8.2%, 7.9%, and 4.6% respectively; at different proportions of attacked clients, on the cifar-10, cifar-100, and svhn datasets, FedCM improves the average accuracy of the comparison methods by at least 4.6%, 5.2%, and 0.45% respectively, indicating that FedCM has better robustness to the proportion of attacked clients and better model accuracy. Although the amount of malicious update information in the model increases as the number of attacked clients increases and the model update direction shifts; while some methods aggregate by extracting partial updates, since in some rounds, among the participating clients, the number of attacked clients is relatively large and most of the aggregated model updates are noisy, resulting in model deviation; however, because the meta-learning part based on Gaussian noise model parameters in FedCM extracts anti-noise model parameter information, which corrects the model direction and improves the model performance, it further demonstrates the effectiveness of the meta-learning part based on Gaussian noise model parameters.

[0098] Robustness to the proportion of contaminated samples. As shown in Table 2, FedCM has higher accuracy than the comparison methods under different datasets and different proportions of contaminated samples (nr = 0.25, 0.5, 0.8, 1). For example, on cifar100, FedCM improves the average accuracy of the Median method by at least 22.2%; on the cifar-10, cifar-100, and svhn datasets, the FedCM method improves the rfa method by at least 8.4%, 5.2%, and 0.98% respectively. As the proportion of contaminated data increases, the characteristic information of the contaminated data increases, and the impact of the contaminated data on the model increases, making the model prone to deviation. Due to the randomness of the contaminated data and the non-iid data distribution (β = 0.5), as the amount of contaminated sample data increases, there will be certain fluctuations in the model accuracy of each method. Although the FedCM method fluctuates, its model accuracy is still higher than that of the comparison methods. Because the meta-learning part based on Gaussian noise in the FedCM method introduces noise for learning, enabling the model to predict contaminated data well, and using the global model to correct reduces the possibility and degree of the introduced noise causing the model to deviate, so the model update direction is more accurate, thus demonstrating that FedCM has good robustness to the proportion of contaminated samples.

[0099] Table 2: Performance comparison of each method under different sample contamination ratios

[0100]

[0101] In summary, the FedCM method has better model accuracy and robustness under different attacked client ratios and contaminated sample ratios.

[0102] Under the default experimental settings, ablation experiments were conducted by splitting and combining components. The following four cases were mainly compared: (1) Without any components: Only use fedavg for aggregation. (2) Only meta-learning: Only use meta-learning based on Gaussian noise model parameters on the client side. (3) Only contrastive learning: Only use contrastive learning based on improved data features on the client side. (4) All components: Use the entire fedCM on the client side and use FedAvg aggregation on the server side.

[0103] As Figure 8 shown, the accuracies of the above four cases are 75.39, 79.59, 79.23, and 81.61 respectively. When only meta-learning is performed, compared with without any components, the overall fluctuation is smaller and the accuracy is improved, indicating that meta-learning based on Gaussian noise model parameters maintains the stability of the model performance and reduces the sensitivity of the model to abnormal data. When only contrastive learning is performed, compared with without any components, the model accuracy is improved, but the fluctuation is larger, indicating that contrastive learning based on improved data features can improve the accuracy of the model, but is affected by noisy data. When all components are present, compared with only meta-learning, the overall performance is improved, indicating that contrastive learning based on improved features can improve the component performance; compared with only contrastive learning, the model accuracy is improved and the fluctuation is smaller, indicating that meta-learning can effectively improve the robustness of the model to data anomalies, and the combination of meta-learning and contrastive learning can improve the accuracy of the model in abnormal data. Since directly using local features for contrastive learning will make the abnormal information more prominent, while contrastive learning based on improved features uses the previous round of denoising model to fine-tune the features and slow down the extraction of noise features, the amplitude of performance improvement will be smoothed, making the model more stable. In short, when non-iid and there are data anomalies, the two components of FedCM proposed in this paper can improve the accuracy of the model to a certain extent and make the model more stable.

[0104] In summary, based on the corrected feature vectors of each training client, the present embodiment performs adaptive contrast learning, analyzes and obtains the contrast loss of each training client, thereby obtaining the cross-entropy loss of each training client, and jointly analyzes it with the contrast loss to obtain the basic loss of local model training, thus achieving the improvement of the classification accuracy and feature extraction quality of the local model in the federated learning environment, and effectively solving the problem of the decrease in model accuracy caused by data anomalies in the prior art under the condition of uneven data distribution.

[0105] Those skilled in the art should understand that the embodiments of the present invention can be provided as a method, a system, or a computer program product. Therefore, the present invention can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Moreover, the present invention can take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0106] The present invention is described with reference to the flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to the embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, and the combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to the processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing devices to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing devices generate means for implementing the specified functions in one process Figure 1 one process or multiple processes and / or blocks Figure 1 or multiple blocks.

[0107] These computer program instructions can also be stored in a computer-readable memory that can direct a computer or other programmable data processing device to work in a specific manner, so that the instructions stored in the computer-readable memory generate a manufactured article including instruction means, and the instruction means implements the specified functions in one process Figure 1 one process or multiple processes and / or blocks Figure 1 or multiple blocks.

[0108] These computer program instructions can also be loaded onto a computer or other programmable data processing device, so that a series of operation steps are performed on the computer or other programmable device to generate a computer-implemented process, and thus the instructions executed on the computer or other programmable device provide steps for implementing the specified functions in one process Figure 1 one process or multiple processes and / or blocks Figure 1 or multiple blocks.

[0109] Although the preferred embodiments of the present invention have been described, additional changes and modifications can be made by those skilled in the art once they learn the basic creative concept. Therefore, the appended claims are intended to be construed to include the preferred embodiments as well as all changes and modifications that fall within the scope of the present invention.

[0110] Obviously, those skilled in the art can make various changes and modifications to the present invention without departing from the spirit and scope of the present invention. Thus, if these modifications and variations of the present invention fall within the scope of the claims of the present invention and their equivalent technologies, the present invention is also intended to include these modifications and variations.

Claims

1. A robust federated learning method for image classification, characterized in that It includes the following steps: S1. The server filters out the clients participating in the training through a dynamic client screening strategy and marks them as each training client. The server initializes the global model to obtain an initialized global model and distributes it to each training client; S2. Obtain the local image data of each training client, and perform noise simulation processing and data augmentation preprocessing to obtain the simulated augmented data of each training client; S3. Train the local models of each training client based on the simulated augmented data of each training client to obtain the corrected feature vectors of each training client; S4. Perform adaptive contrast learning based on the corrected feature vectors of each training client, analyze to obtain the contrast loss of each training client, obtain the cross-entropy loss of each training client, and jointly analyze with the contrast loss to obtain the basic loss of the local model training, and thus perform retraining on the local models of each training client; S5. Introduce random Gaussian noise into the simulated augmented data of each training client to obtain the contaminated data of each training client, obtain the noisy local models of each training client based on the contaminated data of each training client, and thus analyze to obtain the local updated models of each training client and upload them to the server; S6. The server receives the local updated models of each training client and analyzes to obtain the global updated model; Repeat the operations of S2 to S6 until the global updated model converges; The server filters out the clients participating in the training through a dynamic client screening strategy and marks them as each training client. The specific steps are as follows: Obtain the computing resource parameters, data quality parameters of each client and the historical contribution coefficients of each client, and analyze to obtain the contribution ideal effect value of each client; Obtain the preset contribution ideal effect threshold in the database and compare it with the contribution ideal effect value of each client. If the contribution ideal effect value of a certain client is above the contribution ideal effect threshold, then use this client as the training client; Allocate the global model of the server to each training client, and train the model of the global model sent by the server based on the image sample data set of each training client to obtain the local models of each training client; The computing resource parameters include the average task execution duration and the maximum number of concurrent tasks; The data quality parameters include the abnormal data proportion, data update frequency and available data sample size; The contribution ideal effect value of each client is obtained by differentiating the computing resource parameters and data quality parameters of each client from the average level of the client respectively, then introducing the corresponding influence weights, and finally performing coupling processing with the historical contribution coefficients of each client.

2. The robust federated learning method for image classification according to claim 1, wherein: The step of training the local models of each training client based on the simulated augmented data of each training client to obtain the corrected feature vectors of each training client specifically includes: Respectively obtain the current round feature vectors and the feature vectors of the previous round of each training client at the predetermined intermediate layer, and analyze to obtain the cosine similarity of adjacent rounds of each training client; The dynamic feature weights of each training client are obtained based on the cosine similarity analysis of adjacent rounds of each training client; Based on the dynamic feature weights of each training client, the current-round feature vectors obtained by each training client are corrected using the feature vectors of the previous round to obtain the corrected feature vectors of each training client.

3. The robust federated learning method for image classification according to claim 1, characterized in that The adaptive contrast learning is performed based on the corrected feature vectors of each training client, and the contrast loss of each training client is analyzed. The specific method is as follows: Based on the corrected feature vectors of each training client, the adaptive contrast learning of each training client is respectively performed, and the cosine similarity between the sample and all its positive samples is calculated; Calculate the cosine similarity between the sample and all its contrast samples; For each sample, calculate its similarity with the positive samples, and normalize it through the Softmax distribution to convert the similarities of all positive and negative samples into a probability distribution; Calculate the negative log-likelihood, and maximize the similarity of the positive samples in the Softmax distribution to obtain the contrast loss of each training client.

4. The robust federated learning method for image classification according to claim 1, wherein: The cross-entropy loss of each training client is obtained, and the basic loss of local model training is jointly analyzed with the contrast loss. The specific method is as follows: The cross-entropy loss is used to measure the matching degree between the predicted values of the local models of each training client and the true labels; Calculate the predicted probability of each image sample data in the image sample dataset of each training client, and calculate the cross-entropy loss between its distribution and the true label to obtain the cross-entropy loss of each training client; Based on the cross-entropy loss of each training client, the contrast loss is increased, and the cross-entropy loss and the contrast loss of each training client are minimized to obtain the basic loss of local model training.

5. The robust federated learning method for image classification according to claim 1, wherein: Random Gaussian noise is introduced into the simulated augmented data of each training client to obtain the contaminated data of each training client. The specific steps include: Obtain part of the data in the simulated augmented data of each training client, and obtain the sample data information and its corresponding sample label in this part of the data; Set the Gaussian distribution condition, where the Gaussian distribution condition is that the mean of the Gaussian distribution is set to zero, and the random number of the Gaussian distribution is set to the standard deviation of the Gaussian distribution; Based on the simulated augmented data of each training client and the Gaussian distribution condition, random Gaussian noise is introduced to obtain the contaminated data of each training client.

6. The robust federated learning method for image classification according to claim 1, wherein: The noisy local models of each training client are obtained based on the contaminated data of each training client. The specific method is as follows: Based on the contaminated data of each training client, the local models of the current training clients are trained, and the local model parameters of each training client are updated using the stochastic gradient descent method to obtain the noisy local models of each training client; Output the noisy local models of each training client.

7. The robust federated learning method for image classification according to claim 1, wherein: The local updated models of each training client are obtained. The specific method is as follows: Obtain the original data of each training client, and input it into the noisy local model of the corresponding training client to obtain the predicted probability distribution of the noisy prediction data of each training client. Construct the cross-entropy loss with the predicted probability distribution of the noisy prediction data of each training client and the original data label as the meta-loss; Obtain the original data of each training client and input it into the global model to obtain the global data prediction probability distribution of each training client; Based on the prediction probability distribution of the noise prediction data of each training client and the global data prediction probability distribution of each training client, analyze and obtain the KL divergence of each training client; Based on the KL divergence of each training client and the meta-loss coupling processing, obtain the total meta-learning loss of each training client; Based on the total meta-learning loss of each training client and the basic loss of the local model training of each training client, jointly train the local model of each training client to obtain the local model of each training client in this round.

8. The robust federated learning method for image classification according to claim 1, wherein: The server receives the local updated models of each training client and analyzes to obtain the global updated model. The specific method is as follows: Identify the attacked clients and mark them as malicious clients. Simulate the abnormal data generated by the malicious clients under attack by adding noise to obtain the dataset of each malicious client, and perform data augmentation preprocessing to obtain the simulated augmented data of each malicious client; Each malicious client selected by the dynamic client screening strategy in each round obtains the global model sent by the server and performs cross-entropy loss training to obtain the local updated model of each malicious client; Obtain the dataset sizes of each training client and each malicious client selected in each round, and perform aggregation processing on the local updated models of each training client and each malicious client selected in each round received by the server through the FedAvg aggregation method to obtain the global updated model.

9. The robust federated learning method for image classification according to claim 1, wherein: For the analysis to obtain the global updated model, repeat the operations of S2 to S6 until the global updated model converges. The specific method is as follows: Repeat the operations of S2 to S6. After each round of aggregation, evaluate the classification accuracy of the global model on the validation set; If the accuracy fluctuation in consecutive preset rounds is lower than the preset fluctuation ratio, or reaches the preset maximum number of rounds, it is determined that the global updated model converges.

Images