Robust federated learning architecture for image classification

Abstract

The invention discloses an image classification-oriented robust federated learning architecture, which belongs to the technical field of machine vision, and comprises the following steps: S1, screening training clients, and initializing global model parameters; s2, obtaining simulation enhancement data of each training client; s3, obtaining the corrected feature vector of each training client; s4, obtaining basic loss of local model training, and performing local model retraining of each training client; s5, obtaining a noisy local model of each training client, thereby obtaining a local update model of each training client, and uploading the local update model to the server; and S6, the server receives the local update model of each training client to obtain a global update model, and the operations from S2 to S6 are repeated until the global update model converges. The problem that in the prior art, under the condition of uneven data distribution, the accuracy of the model is reduced due to data exception is solved.

Description

Technical Field

[0001] The present invention relates to the field of machine vision technology, and in particular to a robust federated learning architecture for image classification. Background Art

[0002] With the development of the big data era, machine learning technology is widely used, and federated learning is often used to ensure the privacy and security of its data. Federated learning mainly implements data privacy protection through the steps of sending model updates from the server to the client, the client receiving model updates and performing local model training, and the server aggregating the client's local model updates to obtain a new global model.

[0003] In real applications, the data characteristics and distribution of each client vary greatly, showing an unbalanced data distribution state (non-iid). Unbalanced data distribution will affect the accuracy of the model. Some existing methods use shared mapping space, global feature dictionary, and data distribution statistics to adjust local training or global aggregation to solve the non-iid problem, but ignore the data statistics deviation caused by data anomalies, which will affect model performance and update direction.

[0004] In the federated learning framework, the client may be attacked by attackers, resulting in abnormal data, which will affect the accuracy of the model and even cause the model to seriously deviate from the normal update direction. Some other existing methods reduce the impact of updates obtained by data anomalies on the global model by eliminating abnormal updates or adjusting abnormal update weights, but this may cause data information of some categories to be lost or unsuitable for non-iid situations, thus affecting the accuracy of the model. Therefore, in the case of uneven data distribution, there is a problem of reduced model accuracy due to data anomalies.

[0005] For example, the invention patent with announcement number: CN114494771B announces a federated learning image classification method that can defend against backdoor attacks, including: using matrix dimensionality reduction and clustering algorithms to process the gradients submitted by the Worker nodes, and finally selecting the gradients submitted by normal Worker nodes to participate in aggregation, thereby completely avoiding the possibility of backdoors being implanted into the global model. Summary of the invention

[0006] The embodiments of the present application provide a robust federated learning architecture for image classification, thereby solving the problem in the prior art of decreased model accuracy due to data anomalies when data distribution is uneven, and achieving improved classification accuracy and feature extraction quality of local models in a federated learning environment.

[0007] The embodiment of the present application provides a robust federated learning architecture for image classification, including the following steps: S1, the server obtains the clients participating in the training through a dynamic client screening strategy, and marks them as training clients. The server initializes the global model to obtain the initialized global model, and distributes it to each training client; S2, obtains the local image data of each training client, and performs noise simulation processing and data enhancement preprocessing to obtain the simulated enhanced data of each training client; S3, trains the local model of each training client based on the simulated enhanced data of each training client, and obtains the corrected feature vector of each training client; S4, based on the corrected feature vector of each training client Perform adaptive contrast learning, analyze and obtain the contrast loss of each training client, obtain the cross entropy loss of each training client, and jointly analyze with the contrast loss to obtain the basic loss for local model training, thereby retraining the local model of each training client; S5, introduce random Gaussian noise into the simulated enhanced data of each training client to obtain the contaminated data of each training client, and obtain the noisy local model of each training client based on the contaminated data of each training client, thereby analyzing and obtaining the local update model of each training client, and upload it to the server; S6, the server receives the local update model of each training client, analyzes and obtains the global update model, and repeats the operations from S2 to S6 until the global update model converges.

[0008] One or more technical solutions provided in the embodiments of the present application have at least the following technical effects or advantages: 1. The robust federated learning architecture for image classification provided by the present invention performs adaptive contrast learning based on the corrected feature vectors of each training client, analyzes the contrast loss of each training client, and thus obtains the cross entropy loss of each training client, and jointly analyzes with the contrast loss to obtain the basic loss of local model training, thereby achieving the improvement of classification accuracy and feature extraction quality of the local model in a federated learning environment, and effectively solves the problem of decreased model accuracy due to data anomalies in the prior art when data distribution is uneven; 2. The present invention obtains the computing resource parameters, data quality parameters and historical contribution coefficients of each client, analyzes and obtains the ideal contribution effect value of each client, and selects clients whose ideal contribution effect value is higher than the threshold to participate in the training, thereby improving the training efficiency and model performance, and effectively solving the problem in the prior art that the model accuracy decreases due to data anomalies when the data is unevenly distributed; 3. The present invention obtains the feature vector of the current round and the feature vector of the previous round of each training client in the predetermined intermediate layer respectively, analyzes the cosine similarity of adjacent rounds of each training client, and then obtains the dynamic feature weight of each training client based on the cosine similarity of adjacent rounds of each training client, thereby optimizing the quality of model feature extraction and improving the classification accuracy and robustness of the model; 4. The present invention obtains the noisy local model of each training client based on the contaminated data of each training client, and then performs meta-learning strategy training based on the noisy local model of each training client, thereby enhancing the adaptability and noise resistance of the model to noisy data, and improving the stability and performance of the model in a complex data environment; 5. The present invention introduces random Gaussian noise into the simulated enhanced data of each training client to obtain the contaminated data of each training client, and then obtains the noisy local model of each training client based on the contaminated data of each training client. The local updated model of each training client is obtained by analysis and uploaded to the server, thereby improving the noise resistance of the model by constructing a noisy data environment and reducing the impact of data anomalies on the global model update. BRIEF DESCRIPTION OF THE DRAWINGS

[0009] Figure 1 A flowchart of a robust federated learning architecture for image classification provided in an embodiment of the present application; Figure 2 A schematic diagram showing the effect of the model accuracy on the CIFAR-10 dataset provided in an embodiment of the present application; Figure 3 A schematic diagram showing the effect of the model accuracy on the CIFAR-100 dataset provided in an embodiment of the present application; Figure 4 A schematic diagram showing the effect of the model accuracy on the Street View House Numbers (SVHN) dataset provided in an embodiment of the present application; Figure 5 A schematic diagram showing the effect of the CIFAR-10 dataset model accuracy under different proportions of attacked clients provided in an embodiment of the present application; Figure 6 A schematic diagram showing the effect of the CIFAR-100 dataset model accuracy under different proportions of attacked clients provided in an embodiment of the present application; Figure 7 A schematic diagram showing the effect of the Street View House Numbers (SVHN) dataset model accuracy under different proportions of attacked clients provided in an embodiment of the present application; Figure 8 A schematic diagram of the effects of different components on model performance provided in an embodiment of the present application. DETAILED DESCRIPTION

[0010] The embodiments of the present application provide a robust federated learning architecture for image classification, thereby solving the problem in the prior art of decreased model accuracy due to data anomalies when data is unevenly distributed, thereby achieving the goal of improving the classification accuracy and feature extraction quality of the local model in a federated learning environment.

[0011] In order to better understand the above technical solution, the above technical solution will be described in detail below in conjunction with the accompanying drawings and specific implementation methods.

[0012] like Figure 1 As shown, it is a flowchart of a robust federated learning architecture for image classification provided by an embodiment of the present application, and the method includes the following steps: S1, the server obtains the clients participating in the training through a dynamic client screening strategy, and marks them as training clients, and the server initializes the global model to obtain the initialized global model, and distributes it to each training client; S2, obtains the local image data of each training client, and performs noise simulation processing and data enhancement preprocessing to obtain the simulated enhanced data of each training client; S3, trains the local model of each training client based on the simulated enhanced data of each training client, and obtains the corrected feature vector of each training client; S4, based on the corrected feature vector of each training client The feature vector of each training client is adaptively contrasted and learned, and the contrast loss of each training client is obtained by analysis. The cross entropy loss of each training client is obtained, and the basic loss of local model training is obtained by joint analysis with the contrast loss, thereby retraining the local model of each training client; S5, random Gaussian noise is introduced into the simulated enhanced data of each training client to obtain the contaminated data of each training client, and the noisy local model of each training client is obtained based on the contaminated data of each training client, thereby analyzing and obtaining the local update model of each training client, and uploading it to the server; S6, the server receives the local update model of each training client, analyzes and obtains the global update model, and repeats the operations from S2 to S6 until the global update model converges.

[0013] In this embodiment, it should be noted that the clients marked as training clients are benign clients, wherein only benign clients (clients that have not been attacked, i.e., clients without data anomalies) can execute steps S2 to S5; clients that have been attacked are malicious clients.

[0014] The operation of the contrastive learning part is mainly to calculate the cosine similarity of the data features of the previous round and the current round, use the similarity to adjust the local data features, reduce the impact of noise on the model, use the contrastive learning method to aggregate similar data features, and improve the model accuracy and convergence speed. The operation of the meta-learning part is mainly to randomly add noise to the original data, train to obtain the noise model, and use meta-learning to learn the knowledge of the noise model into the normal model to improve the model accuracy under data imbalance and data anomalies. It should be noted that the original data refers to the local image data of each training client.

[0015] Through experimental testing, the framework of this application improves the model accuracy and model convergence speed compared with existing methods in the presence of non-iid and data anomalies. For example, in the case of non-independent and identically distributed parameters of 0.5 on CIFAR-10, when the attack client ratio is 0.2, 0.5 and 0.8, the average accuracy of FedCM (Robust Federated Learning Architecture) is improved by 8.2%, 7.9% and 4.6% respectively compared with the baseline method; it shows obvious advantages on the SVHN (Street View House Numbers) dataset, and the number of training rounds required for convergence is reduced by about 34.78% compared with other methods.

[0016] It should be noted that each training client receives the global model and generates a noisy data set by simulating the noise environment after the client is attacked through Gaussian noise, shooting noise, impulse noise, glass blur, motion model, zoom blur, fog effect, frost effect, brightness adjustment, contrast adjustment, pixelation, elastic transformation, and distortion effect methods. At the same time, through data enhancement strategies such as random flipping, cropping, and brightness adjustment, the data diversity is improved, the generalization ability of the model is enhanced, and stable classification performance can be maintained in complex environments.

[0017] It should also be noted that the generated noisy data set is used to simulate the client being attacked and generate abnormal data.

[0018] Furthermore, the server obtains the clients participating in the training through the dynamic client screening strategy and marks them as training clients. The specific steps are as follows: obtaining the computing resource parameters, data quality parameters and historical contribution coefficients of each client, and analyzing to obtain the ideal contribution effect value of each client; obtaining the ideal contribution effect threshold preset by the database, and comparing it with the ideal contribution effect value of each client. If there is a client whose ideal contribution effect value is above the ideal contribution effect threshold, the client is used as the training client; allocating the global model of the server to each training client, and training the model of the global model sent by the server based on the image sample data set of each training client to obtain the local model of each training client; the computing resource parameters include the average task execution time and the maximum number of concurrent tasks; the data quality parameters include the proportion of abnormal data, the frequency of data update and the amount of available data samples; the ideal contribution effect value of each client is to perform a differential analysis of the computing resource parameters and data quality parameters of each client with the average level of the client, and then introduce the corresponding influence weight, and finally couple it with the historical contribution coefficient of each client to obtain the ideal contribution effect value of each client.

[0019] In this embodiment, the ideal contribution effect value of each client is obtained by: ; ; ; In the formula, represents the ideal contribution effect value of the mth client, represents the historical contribution coefficient of the mth client, where m represents the client number. , Indicates the total number of clients. represents the average execution time of the mth client's task, Indicates the maximum number of concurrent tasks for the mth client. Indicates the proportion of abnormal data of the mth client, Indicates the data update frequency of the mth client. represents the available sample size of the data for the mth client, Indicates the weight of task execution time. Indicates that the number of concurrent tasks affects the weight, Indicates the influence weight of abnormal data proportion, Indicates that the data update frequency affects the weight, Indicates that the available sample size of the data affects the weight.

[0020] It should be noted that by analyzing the computing resource parameters, data quality parameters and historical contribution coefficients of each client, the ideal contribution effect value of each client is obtained by analysis, taking into account the mutual influence relationship between these parameters. For example, clients with shorter average task execution time usually have more sufficient computing resources and can handle more tasks at the same time, thereby supporting a larger maximum number of concurrent tasks. Clients with high data update frequency usually have more timely and accurate data, so the proportion of abnormal data is lower. Clients with high data update frequency can often accumulate more data samples, thus having a larger amount of available data samples. Clients with high historical contribution coefficients usually have better task execution capabilities and stability in the past, so they will have shorter average task execution time and a larger maximum number of concurrent tasks.

[0021] A shorter average task execution time means that the client can complete the task more efficiently, thus getting closer to the ideal contribution effect value. A larger maximum number of concurrent tasks enables the client to handle more tasks at the same time, improve resource utilization and task processing efficiency, and help get closer to the ideal contribution effect value. A lower proportion of abnormal data means that the client's data quality is higher, the model training effect is better, and it is closer to the ideal contribution effect value. A higher data update frequency means that the client's data is more timely and accurate, and the model can be trained and optimized based on the latest data, which is closer to the ideal contribution effect value. A larger available data sample size provides a richer data foundation for model training, which helps to improve the generalization ability and accuracy of the model and is closer to the ideal contribution effect value. A higher historical contribution coefficient reflects the client's positive contribution to model training in the past, indicating that it has performed well in terms of computing resources, data quality, and task execution capabilities.

[0022] Furthermore, the local model of each training client is trained based on the simulated enhanced data of each training client to obtain the corrected feature vector of each training client. The specific steps include: respectively obtaining the current round feature vector and the previous round feature vector of each training client in the predetermined middle layer, and analyzing to obtain the cosine similarity of adjacent rounds of each training client; obtaining the dynamic feature weight of each training client based on the cosine similarity analysis of adjacent rounds of each training client; based on the dynamic feature weight of each training client, using the feature vector of the previous round, correcting the current round feature vector obtained by each training client to obtain the corrected features of each training client; the dynamic feature weight of each training client is to perform differential analysis on the current round feature vector and the previous round feature vector of each training client to obtain the dynamic feature weight of each training client.

[0023] In this embodiment, it should be noted that the predetermined intermediate layer is, for example, the last convolutional layer or the fully connected layer.

[0024] The cosine similarity of adjacent rounds of training clients is obtained by: ; In the formula, represents the cosine similarity of adjacent rounds of training clients, Indicates the similarity between the feature vector of the current round and the feature vector of the previous round, represents the feature vector of the current round, represents the feature vector of the previous round, represents the matrix norm of the eigenvector of the current round, Represents the matrix norm of the eigenvector of the previous round.

[0025] The cosine similarities of adjacent rounds of training clients can be obtained by summing up the cosine similarities of adjacent rounds of training clients.

[0026] Get the dynamic feature weights of the training client. The specific method is: ; In the formula, represents the dynamic feature weights of the training client, Represents the cosine similarity of adjacent rounds of training clients.

[0027] It should be noted that, When , it means that the feature similarity is very low. This situation rarely occurs, and it means that the direction of the feature after the local model is updated seriously deviates from the direction of the previous round of model features. Therefore, only the features of the local model are considered at this time; when When , although the similarity is low, the current local model is mainly considered, so a larger proportional weight should be assigned to the local model; when , the similarity is high, but because the current local model is more affected by noise, the previous round of local model is still needed for correction.

[0028] The dynamic feature weights of the training clients can be aggregated to obtain the dynamic feature weights of each training client.

[0029] Get the corrected features of the training client. The specific method is: ; In the formula, represents the corrected features of the training client, represents the dynamic feature weights of the training client, represents the feature vector of the current round, Represents the feature vector of the previous round.

[0030] The corrected features of the training client are summarized to obtain the corrected features of the training client.

[0031] Furthermore, adaptive contrastive learning is performed based on the corrected feature vectors of each training client, and the contrast loss of each training client is obtained by analysis. The specific method is: based on the corrected feature vectors of each training client, adaptive contrastive learning is performed on each training client, and the cosine similarity between the sample and all its positive samples is calculated; the cosine similarity between the sample and all its contrast samples is calculated; for each sample, its similarity with the positive sample is calculated, and the similarities of all positive and negative samples are converted into probability distributions through Softmax distribution normalization; the negative log-likelihood is calculated to maximize the similarity of the positive sample in the Softmax distribution to obtain the contrast loss of each training client.

[0032] In this embodiment, adaptive contrast learning is performed based on the corrected feature vectors of each training client, and the contrast loss of each training client is obtained by analysis. The specific method is: the corrected feature vectors of the training client are obtained for adaptive contrast learning, and the contrast loss of the training client is obtained by analysis, and the contrast loss of the training client is summarized to obtain the contrast loss of each training client; Get the contrast loss of the training client. The specific method is: ; In the formula, represents the contrastive loss of the training client, represents the sample set of the i-th training client, i represents the number of the training client, , represents the total number of training clients, j represents the comparison sample, represents the set of positive samples with the same category as the comparison sample j, represents the number of samples of the same category as sample j in the sample set of the i-th client, represents all local samples except sample j, represents the corrected feature vector of the comparison sample, k represents the same type of sample, represents the corrected feature vector of the same sample, a represents any comparison sample except sample j, represents the corrected feature vector of any comparison sample, Represents the temperature parameter, which is used to adjust the smoothness of the similarity distribution. express and The cosine similarity between express and The cosine similarity between .

[0033] It should be noted that the comparison samples include positive samples and negative samples, and the positive samples are samples of the same category. The cosine similarity between the sample and all its comparison samples is calculated, where all comparison samples are other samples in the same batch. The temperature parameters can be obtained by retrieving the temperature parameters after the expert statistical test in the database.

[0034] It should be noted that ; .

[0035] Furthermore, the cross entropy loss of each training client is obtained, and is jointly analyzed with the contrast loss to obtain the basic loss of local model training. The specific method is: use the cross entropy loss to measure the matching degree between the local model prediction value of each training client and the true label; calculate the prediction probability of each image sample data in the image sample data set of each training client, and calculate the cross entropy loss of its distribution and the true label to obtain the cross entropy loss of each training client; based on the cross entropy loss of each training client, increase the contrast loss, and minimize the cross entropy loss and contrast loss of each training client to obtain the basic loss of local model training.

[0036] In this example, the cross entropy loss is: ; In the formula, represents the cross entropy loss, represents the dataset size of the i-th training client, i represents the number of the training client, , represents the total number of training clients, represents the input sample data, represents the true label corresponding to the sample, represents the predicted probability distribution generated by the model for the input sample x, Represents the model parameters of each training client; ; In the formula, represents the total loss of the client's local model, represents the contrastive loss of the training client, represents the cross entropy loss.

[0037] Furthermore, random Gaussian noise is introduced into the simulated enhanced data of each training client to obtain contaminated data of each training client. The specific steps include: obtaining part of the data in the simulated enhanced data of each training client, obtaining sample data information in the part of the data and its corresponding sample label; setting a Gaussian distribution condition, where the Gaussian distribution condition is to set the mean of the Gaussian distribution to zero and to set the random number of the Gaussian distribution to the standard deviation of the Gaussian distribution; based on the simulated enhanced data of each training client and the Gaussian distribution condition, introducing random Gaussian noise to obtain the contaminated data of each training client.

[0038] In this embodiment, random Gaussian noise is first used to add noise to part of the simulated enhanced data of each training client to obtain the contaminated data of each training client. The specific method is: ; In the formula, represents the contaminated data of each training client, x is the sample data, y is the true label, is a Gaussian distribution, represents the input sample after Gaussian noise perturbation, m represents the mean of Gaussian distribution, Represents the standard deviation of the Gaussian distribution.

[0039] It should be noted that the local data set of each training client is marked as Di, i represents the number of the training client, , Represents the total number of training clients, selects a small batch of data from them, and marks it as Bi, Bi={(x,y)}, assumes that the Gaussian distribution is g(m,std), m is the mean of the Gaussian distribution, set to 0, to avoid excessive deviation of the data, causing the model to be too biased towards the noise direction and resulting in performance degradation, std is the standard deviation of the Gaussian distribution, set to a random number.

[0040] Furthermore, a noisy local model of each training client is obtained based on the contaminated data of each training client. The specific method is: based on the contaminated data of each training client, the local model of each current training client is trained, and the local model parameters of each training client are updated using the stochastic gradient descent method to obtain the noisy local model of each training client; and the noisy local model of each training client is output.

[0041] In this embodiment, the noisy small batch data is used to train the original local model (the local model of each training client) using the traditional cross entropy loss, and the model parameters are updated using stochastic gradient descent: ; In the formula, represents the cross entropy loss of noisy data for each training client, represents the input sample after Gaussian noise perturbation, represents the true label, represents the model parameters of the i-th training client in the current training round, represents the cross entropy loss of the training client, represents a dataset of small batches of noisy data for the i-th training client, represents the size of the noisy small batch data, i represents the number of the training client, , Indicates the total number of training clients; ; In the formula, represents the model parameters of the i-th training client after being updated by noisy data training, represents the model parameters of the i-th training client in the current training round, represents the learning rate, Represents the cross entropy loss on model parameters The gradient of , i represents the number of the training client, , Indicates the total number of training clients.

[0042] Furthermore, a local updated model of each training client is obtained. The specific method is as follows: the original data of each training client is obtained, and the data is input into the noisy local model of the corresponding training client to obtain the predicted probability distribution of the noise prediction data of each training client; the predicted probability distribution of the noise prediction data of each training client is input into the local model of the corresponding training client to obtain the predicted probability distribution of the original prediction data of each training client; based on the predicted probability distribution of the noise prediction data of each training client and the predicted probability distribution of the original prediction data of each training client, the predicted probability distribution of the noise prediction data of each training client is constructed together with the original data label. Construct cross entropy loss as meta-loss; obtain the original data of each training client and input it into the global model to obtain the global data prediction probability distribution of each training client; analyze and obtain the KL divergence of each training client based on the prediction probability distribution of the noise prediction data of each training client and the global data prediction probability distribution of each training client; obtain the total meta-learning loss of each training client based on the coupling processing of the KL divergence of each training client and the meta-loss; train the local model of each training client based on the total meta-learning loss of each training client and the basic loss of the local model training of each training client to obtain the local model of each training client in this round.

[0043] In this embodiment, the meta-update of the local model is performed based on the noisy local model of each training client. The noisy local model is used to guide the model training, so as to obtain the noise-resistant model parameters. The noisy local model is used to obtain a correct prediction y for the original data x, so as to optimize the model parameters. The specific method is as follows: ; In the formula, represents the meta-learning cross entropy loss, i represents the number of the training client, , represents the total number of training clients, represents the small batch original data of the i-th training client, represents the cross entropy loss of the training client, represents the output probability predicted by the noisy local model, Represents the model parameters of the i-th training client after being updated with noisy data training.

[0044] The global model is used to fine-tune the noisy local model to improve the problem of non-independent and identically distributed data. The loss of the global model and the noisy local model is: ; In the formula, represents the knowledge distillation loss, represents the small batch of original data of the i-th training client, i represents the number of the training client, , represents the total number of training clients, Represents the original data, represents the global model, represents the global model parameters, represents the model parameters of the i-th training client after being updated by noisy data training, is the KL divergence between probability distributions p and q, Represents a global model parameter The predicted category probability distribution vector is obtained by forward propagating the original data x through the global model.

[0045] Therefore, the total loss of the meta-learning part is: ; In the formula, represents the total loss of the meta-learning part, represents the meta-learning cross entropy loss, Represents the knowledge distillation loss. Finally, the basic loss of the local model training and the total loss of the meta-learning part are used to jointly train the local model to obtain the local model of this round.

[0046] Furthermore, the server receives the local update model of each training client, and analyzes it to obtain the global update model. The specific method is: identify the attacked client and mark it as a malicious client, simulate the abnormal data generated by the attack on the malicious client by adding noise to obtain the data set of each malicious client, and perform data enhancement preprocessing to obtain the simulated enhanced data of each malicious client; each malicious client screened out by the dynamic client screening strategy in each round obtains the global model sent by the server, and performs cross-entropy loss training to obtain the local update model of each malicious client; obtain the data set size of each training client and each screened malicious client in each round, and aggregate them with the local update model of each training client and each screened malicious client received by the server through the FedAvg aggregation method to obtain the global update model.

[0047] In this embodiment, the server receives the local update model of each training client and analyzes it to obtain the global update model. Specifically, the server receives the local update model of each training client participating in this round, and adopts the FedAvg aggregation method to aggregate the local update model of each training client, the local update model of each malicious client, and the data set of each malicious client to obtain the global model; ; In the formula, Indicates Wheel global model parameters, Indicates The global model parameters of the round, t is the round number, is the zth client dataset, is the sum of the data sets of each training client and each malicious client screened out in each round, is the local model parameter of the zth client in the tth round, where z represents the client number. , Indicates the total number of clients.

[0048] It should be noted that the clients include malicious clients and training clients.

[0049] It should also be noted that the analysis method of the cross entropy loss of each malicious client is exactly the same as the analysis method of the cross entropy loss of each training client.

[0050] Further, the global update model is obtained by analysis, and operations from S2 to S6 are repeated until the global update model converges. The specific method is: repeat operations from S2 to S6, and after each round of aggregation, evaluate the classification accuracy of the global model on the validation set; if the accuracy fluctuation of consecutive preset rounds is lower than the preset fluctuation ratio, or reaches the preset maximum round, it is determined that the global update model converges.

[0051] In this embodiment, the method proposed in the present invention is compared with several baseline models on three data sets (CIFAR-10, CIFAR-100 and Street View House Numbers), mainly analyzing the performance of the present invention under data imbalance and data anomaly, as well as the model robustness under different data non-independent and identically distributed degrees and data anomaly degrees. Among them, the data anomaly degree mainly includes two factors: the proportion of attacked clients and the proportion of contaminated samples. Accuracy is used as the evaluation indicator.

[0052] The experimental results show that the proposed method has improved model performance in three data sets under different non-iid and data anomaly levels compared with the comparison method. Figure 2-Figure 4 As shown in Figure 2, under the default experimental settings (attacked client ratio ar=0.2, contaminated sample ratio nr=0.8, non-iid (β=0.5)), the accuracy of each method is compared and analyzed to evaluate the performance of the proposed method. Figure 2 and Figure 3 As shown, on the cifar-10 and cifar-100 datasets, the model accuracy of the FedCM method is higher than that of all the comparison methods, with an increase of 8.2% and 6.1% respectively (the specific values ​​can be seen in Table 1, the data when β=0.5). This shows that the FedCM method has better accuracy under non-iid and data anomalies. On the svhn dataset, although the highest accuracy of FedCM and the comparison methods are similar, it can be seen from the first 10 rounds that the accuracy of the FedCM method is improved faster than other methods, and it is basically close to convergence in the 15th round, while other methods are close to convergence in the 23rd round. It can be seen that the FedCM method improves the convergence speed of the model. Because the comparative learning part of the FedCM method based on improved features makes the features of similar samples more clustered, the model's prediction of categories is more accurate; and from Figure 4 It can be seen that FedCM has less fluctuation in the first twenty rounds, because in contrastive learning, the local model of the previous round is used to improve the features, reducing the interference of noise information, so that the model is less affected by noise; therefore, the FedCM method has better model performance than other methods, and further illustrates that the contrastive learning part based on improved features is effective and maintains stability. In short, when non-iid and there are data anomalies, FedCM has better accuracy and improved convergence speed.

[0053] Accuracy for non-iid degree. As shown in Table 1, FedCM has the best model accuracy under different non-id degrees (β=0.25, 0.5, 1, 10), indicating that FedCM still maintains good accuracy for data with high non-iid degree. Overall, when β=0.5 changes to 0.25, the degree of decrease in FedCM's accuracy is lower than that of the comparison method, indicating that FedCM is less sensitive to the degree of non-iid. Since the meta-learning part of FedCM based on Gaussian noise model parameters uses a global model to adjust the update direction of the anti-noise model, it can reduce the deviation of the local model from the global model when updating in non-iid, so that a local model that is more in line with the update direction of the global model can be trained to improve generalization. As the degree of non-iid decreases, the accuracy of the FedCM method also increases, indicating that the method of this application is suitable for both non-iid and iid. On the cifar-10, cifar-100, and svhn datasets, FedCM improves by at least 2.7%, 4.5%, and 0.58% respectively over the comparison methods, indicating that FedCM has better model accuracy at different levels of non-iid.

[0054] Table 1: Comparison of the accuracy of various methods at different non-iid levels:

[0055] Robustness to the proportion of clients under attack. Figure 5-Figure 7As shown in the figure, compared with the comparison methods, FedCM has the best average model accuracy under all attacked client ratios (ar=0.2, 0.5, 0.8). In the non-iid case (β=0.5), on cifar-10, when the attacked client ratio is 0.2, 0.5, and 0.8, FedCM improves the average accuracy of the comparison methods by 8.2%, 7.9%, and 4.6%, respectively; under different attacked client ratios, on cifar-10, cifar-100, and svhn datasets, FedCM improves the average accuracy of the comparison methods by at least 4.6%, 5.2%, and 0.45%, respectively, indicating that FedCM has better robustness to the attacked client ratio and better model accuracy. Although as the number of attacked clients increases, the information of malicious updates in the model increases, and the direction of model updates shifts; and some methods aggregate by extracting partial updates. Since the number of attacked clients participating in certain rounds is large, most of the aggregated model updates are noisy, which leads to model shift; however, the meta-learning part based on Gaussian noise model parameters in FedCM extracts noise-resistant model parameter information, thereby correcting the direction of the model and improving model performance, which further illustrates the effectiveness of the meta-learning part based on Gaussian noise model parameters.

[0056] Robustness to the proportion of contaminated samples. As shown in Table 2, FedCM has higher accuracy than the comparison methods under different data sets and different proportions of contaminated samples (nr=0.25, 0.5, 0.8, 1). For example, under cifar100, FedCM has at least 22.2% higher average accuracy than the Median method; under cifar10, cifar100, and svhn data sets, the FedCM method has at least 8.4%, 5.2%, and 0.98% higher accuracy than the rfa method, respectively. As the proportion of contaminated data increases, the feature information of the contaminated data increases, the model is more affected by the contaminated data, and the model is prone to deviation. Due to the randomness of the contaminated data and the non-iid data distribution (β=0.5), the model accuracy of each method will fluctuate to a certain extent as the amount of contaminated sample data increases. Although the FedCM method has some fluctuations, its model accuracy is still higher than the comparison method. Since the Gaussian noise-based meta-learning part in the FedCM method introduces noise for learning, the model can predict the contaminated data well, and the use of the global model for correction reduces the possibility and degree of model deviation caused by the introduced noise, so that the model update direction is more accurate. This shows that FedCM has good robustness to the proportion of contaminated samples.

[0057] Table 2: Performance comparison of various methods under different sample contamination ratios

[0058] In summary, the FedCM method has good model accuracy and robustness under different proportions of attacked clients and contaminated samples.

[0059] Under the default experimental settings, ablation experiments are conducted by splitting and combining components, mainly comparing the following four scenarios: (1) No components: only fedavg is used for aggregation. (2) Meta-learning only: only meta-learning based on Gaussian noise model parameters is used on the client. (3) Contrastive learning only: only contrastive learning based on improved data features is used on the client. (4) All components: the entire fedCM is used on the client, and FedAvg aggregation is used on the server.

[0060] like Figure 8 As shown in the figure, the accuracies of the above four cases are 75.39, 79.59, 79.23, and 81.61, respectively. When only meta-learning is performed, the overall fluctuation is small and the accuracy is improved compared with no components, indicating that meta-learning based on Gaussian noise model parameters maintains the stability of model performance and reduces the sensitivity of the model to abnormal data. When only contrastive learning is performed, the model accuracy is improved compared with no components, but the fluctuation is large, indicating that contrastive learning based on improved data features can improve the accuracy of the model, but is affected by noise data. When all components are present, the overall performance is improved compared with meta-learning alone, indicating that contrastive learning based on improved features can improve component performance; compared with contrastive learning alone, the model accuracy is improved and the fluctuation is small, indicating that meta-learning can effectively improve the robustness of the model to data anomalies, and the combination of meta-learning and contrastive learning can improve the accuracy of the model in abnormal data. Since directly using local features for contrastive learning will make abnormal information more prominent, contrastive learning based on improved features uses the previous round of anti-noise model to fine-tune features and slow down the extraction of noise features, it will smooth the performance improvement and make the model more stable. In short, when non-iid and there are data anomalies, the two components of FedCM proposed in this paper improve the accuracy of the model to a certain extent and make the model more stable.

[0061] To summarize, this embodiment performs adaptive contrast learning based on the corrected feature vectors of each training client, analyzes the contrast loss of each training client, and thus obtains the cross entropy loss of each training client, and jointly analyzes the contrast loss to obtain the basic loss of local model training, thereby achieving the improvement of the classification accuracy and feature extraction quality of the local model in a federated learning environment, and effectively solves the problem of decreased model accuracy due to data anomalies in the prior art when data distribution is uneven.

[0062] It will be appreciated by those skilled in the art that embodiments of the present invention may be provided as methods, systems, or computer program products. Therefore, the present invention may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Furthermore, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0063] The present invention is described with reference to flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0064] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0065] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0066] Although the preferred embodiments of the present invention have been described, those skilled in the art may make other changes and modifications to these embodiments once they have learned the basic creative concept. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications that fall within the scope of the present invention.

[0067] Obviously, those skilled in the art can make various changes and modifications to the present invention without departing from the spirit and scope of the present invention. Thus, if these modifications and variations of the present invention fall within the scope of the claims of the present invention and their equivalents, the present invention is also intended to include these modifications and variations.

Claims

1. A robust federated learning architecture for image classification, characterized by: The following steps are involved: S1. The server selects the clients participating in the training through the dynamic client screening strategy and marks them as training clients. The server initializes the global model, obtains the initialized global model, and distributes it to each training client. S2, obtaining local image data of each training client, and performing noise simulation processing and data enhancement preprocessing to obtain simulated enhanced data of each training client; S3, training the local model of each training client based on the simulated enhanced data of each training client to obtain a corrected feature vector of each training client; S4, performing adaptive contrast learning based on the corrected feature vectors of each training client, analyzing and obtaining the contrast loss of each training client, obtaining the cross entropy loss of each training client, and jointly analyzing with the contrast loss to obtain the basic loss of local model training, thereby retraining the local model of each training client; S5. Introduce random Gaussian noise into the simulated enhanced data of each training client to obtain the contaminated data of each training client, obtain the noisy local model of each training client based on the contaminated data of each training client, thereby analyzing and obtaining the local updated model of each training client, and upload it to the server; S6. The server receives the local update models of each training client, analyzes them to obtain the global update model, and repeats the operations from S2 to S6 until the global update model converges.

2. The robust federated learning architecture for image classification as claimed in claim 1, characterized in that: The server obtains the clients participating in the training by screening through the dynamic client screening strategy and marks them as training clients. The specific steps are as follows: Obtain the computing resource parameters, data quality parameters and historical contribution coefficients of each client, and analyze and obtain the ideal contribution effect value of each client; Obtaining a contribution ideal effect threshold preset in the database, and comparing it with the contribution ideal effect value of each client, if there is a client whose contribution ideal effect value is above the contribution ideal effect threshold, then using the client as a training client; The global model of the server is distributed to each training client, and the model of the global model sent by the server is trained based on the image sample data set of each training client to obtain the local model of each training client; The computing resource parameters include the average execution time of tasks and the maximum number of concurrent tasks; The data quality parameters include the proportion of abnormal data, data update frequency and available data sample size; The ideal contribution effect value of each client is obtained by performing a differential analysis of the computing resource parameters and data quality parameters of each client with the average level of the clients, introducing the corresponding impact weights, and finally coupling them with the historical contribution coefficient of each client to obtain the ideal contribution effect value of each client.

3. The robust federated learning architecture for image classification as claimed in claim 1, characterized in that: The local model of each training client is trained based on the simulated enhanced data of each training client to obtain the corrected feature vector of each training client. The specific steps include: The current round feature vector and the previous round feature vector of each training client in the predetermined middle layer are respectively obtained, and the cosine similarity of adjacent rounds of each training client is analyzed; The dynamic feature weight of each training client is obtained based on the cosine similarity analysis of adjacent rounds of each training client; Based on the dynamic feature weight of each training client, the feature vector of the previous round is used to correct the feature vector of the current round obtained by each training client to obtain the corrected feature vector of each training client.

4. The robust federated learning architecture for image classification as claimed in claim 1, characterized in that: The adaptive contrast learning is performed based on the corrected feature vectors of each training client, and the contrast loss of each training client is obtained by analysis. The specific method is: Based on the corrected feature vectors of each training client, adaptive comparative learning is performed on each training client to calculate the cosine similarity between the sample and all its positive samples; Calculate the cosine similarity between the sample and all its comparison samples; For each sample, calculate its similarity with the positive sample, and normalize it through Softmax distribution to convert the similarities of all positive and negative samples into probability distribution; Calculate the negative log-likelihood and maximize the similarity of positive samples in the Softmax distribution to obtain the contrast loss of each training client.

5. The robust federated learning architecture for image classification as claimed in claim 1, characterized in that: The cross entropy loss of each training client is obtained, and the basic loss of local model training is obtained by joint analysis with the contrast loss. The specific method is: The cross entropy loss is used to measure the matching degree between the local model prediction value of each training client and the true label; Calculate the prediction probability of each image sample data in the image sample data set of each training client, and calculate the cross entropy loss between its distribution and the true label to obtain the cross entropy loss of each training client; The contrast loss is increased based on the cross entropy loss of each training client, and the cross entropy loss and contrast loss of each training client are minimized to obtain the basic loss for local model training.

6. The robust federated learning architecture for image classification as claimed in claim 1, characterized in that: The step of introducing random Gaussian noise into the simulated enhanced data of each training client to obtain the contaminated data of each training client specifically includes: Obtain part of the simulated enhanced data of each training client, and obtain sample data information and its corresponding sample label in the part of the data; Setting a Gaussian distribution condition, wherein the Gaussian distribution condition is to set the mean of the Gaussian distribution to zero and to set the random number of the Gaussian distribution to the standard deviation of the Gaussian distribution; Based on the simulated enhanced data and Gaussian distribution conditions of each training client, random Gaussian noise is introduced to obtain the contaminated data of each training client.

7. The robust federated learning architecture for image classification as claimed in claim 1, characterized in that: The noisy local model of each training client is obtained based on the contaminated data of each training client, and the specific method is: Based on the contaminated data of each training client, the local model of each current training client is trained, and the local model parameters of each training client are updated using the stochastic gradient descent method to obtain the noisy local model of each training client; Output the noisy local model of each training client.

8. The robust federated learning architecture for image classification as claimed in claim 1, characterized in that: The specific method of obtaining the local update model of each training client is as follows: The original data of each training client is obtained and input into the noisy local model of the corresponding training client to obtain the predicted probability distribution of the noisy predicted data of each training client. The predicted probability distribution of the noisy predicted data of each training client is used together with the original data label to construct a cross entropy loss as a meta-loss. Obtain the original data of each training client and input it into the global model to obtain the global data prediction probability distribution of each training client; Based on the prediction probability distribution of the noise prediction data of each training client and the global data prediction probability distribution of each training client, the KL divergence of each training client is analyzed; Based on the KL divergence and meta-loss coupling processing of each training client, the total meta-learning loss of each training client is obtained; Based on the total meta-learning loss of each training client and the basic loss of the local model training of each training client, the local model of each training client is trained together to obtain the local model of each training client in this round.

9. The robust federated learning architecture for image classification as claimed in claim 1, characterized in that: The server receives the local update model of each training client and analyzes it to obtain the global update model. The specific method is: Identify the attacked clients and mark them as malicious clients. Simulate the abnormal data generated by the attacks on malicious clients by adding noise to obtain the data sets of each malicious client. Perform data enhancement preprocessing to obtain the simulated enhanced data of each malicious client. In each round, each malicious client screened out by the dynamic client screening strategy obtains the global model sent by the server, and performs cross-entropy loss training to obtain the local updated model of each malicious client; The data set size of each training client and each malicious client screened in each round is obtained, and the local update model of each training client and each malicious client screened in each round received by the server is aggregated through the FedAvg aggregation method to obtain a global update model.

10. The robust federated learning architecture for image classification as claimed in claim 1, characterized in that: The analysis obtains a global update model, and the operations from S2 to S6 are repeated until the global update model converges. The specific method is: Repeat operations from S2 to S6. After each round of aggregation, evaluate the classification accuracy of the global model on the validation set. If the accuracy fluctuation of consecutive preset rounds is lower than the preset fluctuation ratio, or reaches the preset maximum round, it is determined that the global update model converges.

Images