A heterogeneous label noise robust federated learning method based on swarm dynamics

Abstract

The application belongs to the field of federated learning of artificial intelligence, and discloses a heterogeneous label noise robust federated learning method based on group dynamics, which includes the following steps in each training round: firstly, based on the prediction uncertainty value of the client local model, the client is divided into three categories of mild noise, moderate noise and extreme noise; secondly, differential aggregation is performed, the model of the mild noise client is weighted and averaged to generate a preliminary global model, and the client-aware negative distillation strategy is used to update the preliminary model, wherein different intensities of distillation penalty are applied to the moderate noise and extreme noise clients; finally, according to the inter-round change of the global model accuracy, the threshold used for client classification is dynamically adjusted. Through the combination of three-layer classification, differential aggregation and dynamic threshold adjustment, the application realizes the fine and adaptive management of clients with different noise levels, and significantly improves the robustness and generalization ability of the model in a complex noise environment.

Description

TECHNICAL FIELD

[0001] The present application belongs to the field of federated learning of artificial intelligence, and particularly relates to a robust model training method for coping with heterogeneous and dynamic label noise in distributed client data under the premise of data privacy protection. BACKGROUND

[0002] Federated learning (FL) as a new distributed machine learning paradigm allows multiple data holders (clients) to collaboratively train a shared global model without sharing local raw data, thus effectively protecting data privacy. Classic algorithms such as FedAvg have shown great application potential in data highly sensitive fields such as finance and medicine. However, existing federated learning methods are mostly based on an idealized assumption that the data quality of all clients is high and independent and identically distributed (IID).

[0003] In actual application scenarios, this assumption is often difficult to establish. Heterogeneous label noise exists in client data, which has complex and diverse causes, including but not limited to: non-independent and identically distributed (Non-IID) characteristics of data distribution of each client, uneven quality of artificial annotation, concept drift caused by device differences or environmental changes, etc. Such label noise not only shows significant heterogeneity (i.e. different noise levels) among different clients, but also may present dynamic changes during training, posing a serious challenge to the convergence speed, stability and final generalization performance of the global model.

[0004] To solve the problem of label noise in federated learning, existing technologies have made many explorations. One class of methods adjusts the aggregation weight to weaken the negative impact of low-quality clients. For example, by evaluating the quality of client data to assign a lower aggregation weight, or by introducing a proximal regularization term (such as FedProx) to limit the deviation of the local model from the global model, thereby mitigating the model drift caused by noisy data. Methods such as SCAFFOLD and FedDyn control the variation of client updates to improve aggregation. However, these methods mostly rely on static or quasi-static evaluation of client quality, which is difficult to adapt to the dynamic evolution of noise levels during training.

[0005] Another class of methods focuses on client selection or uncertainty estimation. For example, RoFL limits the contribution of untrusted clients from a security perspective; FedCorr designs a multi-stage error correction mechanism to suppress the propagation of mislabeled data; FedFixer estimates the uncertainty of client predictions using a Bayesian neural network and adjusts the aggregation weights accordingly. These methods can more accurately identify and handle low-quality clients, but often adopt a "one-size-fits-all" approach, directly excluding or heavily suppressing their weights, which may result in the loss of valuable information in these client data.

[0006] In recent years, mechanisms based on knowledge distillation have also been introduced into robust federated learning. For example, FedNed proposes negative knowledge distillation, aiming to make the global model actively "move away" from the erroneous predictions of high-noise clients when aggregating. These methods perform well in extreme noise conditions, but their core idea is still based on a binary division of clients, i.e., simply dividing clients into "clean" and "noisy" categories. This coarse-grained division ignores the existence of moderately noisy clients, which may contain misleading information or potentially useful knowledge.

[0007] In summary, existing technologies have the following shortcomings when dealing with heterogeneous and dynamic label noise:

[0008] 1. Coarse noise modeling granularity: Most methods use binary classification, which cannot accurately depict the multi-level and gradual noise patterns in the real world, especially for moderately noisy clients.

[0009] 2. Lack of adaptability in classification criteria: Fixed thresholds are often used to divide clients, which cannot be adjusted dynamically according to the training of the global model, leading to misclassification at different training stages and affecting model performance.

[0010] 3. Single strategy for high-noise client utilization: Usually, direct exclusion or strong suppression is used, which fails to explore the potential effective information in high-noise clients while suppressing their negative impact.

[0011] Therefore, there is an urgent need for a robust federated learning method that can model client noise with finer granularity, have dynamic adaptive ability, and handle different noise level clients differently. SUMMARY

[0012] The present application aims to overcome the shortcomings of coarse noise modeling granularity, static classification criteria, and single handling strategy for different noise level clients in existing technologies, and proposes a heterogeneous label noise robust federated learning method based on group dynamics.

[0013] In a first aspect, the present application provides a heterogeneous label noise robust federated learning method based on group dynamics, executed by a server computer, comprising the following steps in each federated learning training round:

[0014] a client classification step: based on the prediction uncertainty values of the local models received from the plurality of clients, and according to a dynamic classification threshold, the plurality of clients are divided into at least three preset categories, including a mild noise category, a moderate noise category and an extreme noise category;

[0015] a model aggregation step: receiving the updated local models uploaded from the plurality of clients, and performing differential aggregation to generate an updated global model, the differential aggregation comprising:

[0016] generating a preliminary global model by weighted average based on the updated local models of the clients in the mild noise category;

[0017] updating the preliminary global model using a client-aware negative distillation strategy based on the updated local models of the clients in the moderate noise category and the extreme noise category, wherein the preliminary global model is used as a student model, and the local models of the clients in the moderate noise and extreme noise categories are used as teacher models, a first distillation strength is applied to the clients in the moderate noise category to guide the student model to moderately deviate from its prediction, and a second distillation strength is applied to the clients in the extreme noise category to guide the student model to strongly deviate from its prediction, and the second distillation strength is greater than the first distillation strength;

[0018] a threshold adjustment step: based on the accuracy change of the updated global model and the last round global model on the validation set, the dynamic classification threshold used in the client classification step is dynamically adjusted.

[0019] As an optional implementation manner of the first aspect of the present application, in the client classification step, the process of obtaining the prediction uncertainty value comprises: for the local model of each client, multiple forward inferences are performed using the Monte Carlo dropout method to obtain the prediction probability distribution for each local data sample; and based on the prediction probability distribution, the average cross-entropy on all local data samples is calculated, and the average cross-entropy is taken as the prediction uncertainty value of the client.

[0020] As an optional implementation of the first aspect of the present application, the step of updating the preliminary global model using the client-aware negative distillation strategy comprises: initializing the preliminary global model as a student model; calculating the negative distillation loss of each client in the moderate noise category and the extreme noise category respectively, the negative distillation loss being defined by minimizing the difference between the inverse output of the student model and the teacher model; and updating the student model in combination with the negative distillation loss of the moderate noise category and the extreme noise category to obtain the updated global model.

[0021] As an optional implementation of the first aspect of the present application, the step of calculating the negative distillation loss of each client in the moderate noise category and the extreme noise category respectively comprises: calculating the final client-aware negative distillation loss, the value of which is the sum of the negative distillation loss of the client in the moderate noise category and the weighted negative distillation loss of the client in the extreme noise category; wherein the weighting is achieved by multiplying the negative distillation loss of the client in the extreme noise category by a preset weight factor.

[0022] As an optional implementation of the first aspect of the present application, the threshold adjustment step specifically comprises: calculating the accuracy change value between the updated global model and the last round global model; increasing the dynamic classification threshold when the accuracy change value is greater than a preset positive threshold; and decreasing the dynamic classification threshold when the accuracy change value is less than a preset negative threshold.

[0023] As an optional implementation of the first aspect of the present application, the process of increasing or decreasing the dynamic classification threshold is achieved by updating the current dynamic classification threshold to the combination of the specific quantile of the prediction uncertainty value of the client in the mild noise category, the moderate noise category and the extreme noise category and an adjustment factor; wherein the specific quantile comprises: a first classification threshold for determining the boundary between the mild noise category and the moderate noise category, the update of which is based on the 50% quantile of the prediction uncertainty value of the client in the mild noise category; and a second classification threshold for determining the boundary between the moderate noise category and the extreme noise category, the update of which is based on the 75% quantile of the prediction uncertainty value of the client in the moderate noise category and the 90% quantile of the prediction uncertainty value of the client in the extreme noise category.

[0024] As an optional implementation of the first aspect of the application, the method further comprises, before the model aggregation step, a step of performing local training by the plurality of clients, wherein the local training of the clients in the moderate noise category adopts a confidence-based gradient adjustment mechanism, including: for each training sample in the local data set, calculating the maximum softmax probability of the predicted category as the confidence; comparing the confidence with a dynamic confidence threshold to divide the training samples into high-confidence samples and low-confidence samples; and when calculating the loss function of the local training, reducing the gradient weight of the high-confidence samples by an indicator function, and completely retaining the gradient weight of the low-confidence samples.

[0025] As an optional implementation of the first aspect of the application, the local training of the clients in the extreme noise category adopts a double-model training mechanism, including: training a first model: using the original label to perform standard supervised learning training on the complete local data set; and training a second model: discarding the original label, performing unsupervised learning training based on the pseudo label generated by the global model for the local data in the last round; wherein the combination or one of the first model and the second model uploaded to the server for negative distillation.

[0026] In a second aspect, the embodiments of the present application provide an electronic device, which comprises a processor, a memory, and a program or instructions stored on the memory and executable on the processor, and the program or instructions are executed by the processor to implement the steps of the method according to the first aspect.

[0027] In a third aspect, the embodiments of the present application provide a readable storage medium, which stores a program or instructions, and the program or instructions are executed by a processor to implement the steps of the method according to the first aspect.

[0028] Compared with the prior art, the purpose of the present application is to solve the problems of rough classification and lack of dynamic adaptability in the existing federal learning method for dealing with label noise, and a heterogeneous label noise robust federated learning method based on group dynamics theory is proposed, which improves the robustness and generalization ability of the model under high noise and dynamic noise conditions through three-layer client classification mechanism, client-aware negative distillation and dynamic threshold adjustment based on accuracy. Specifically, in the federated learning process, first, the clients are classified into three levels, i.e., mild noise, moderate noise and extreme noise, to finely depict the noise characteristics of the clients; then, different aggregation strategies are adopted for different categories of clients, among which the moderate noise clients are used for potential information guidance, and the extreme noise clients are suppressed to affect the global model; at the same time, the dynamic threshold adjustment mechanism based on the accuracy change of the global model is adopted to realize the adaptive update of the client role, so as to continuously optimize the client classification and aggregation effect in the training process, and improve the robustness and generalization ability of the model under high noise and dynamic noise environment. BRIEF DESCRIPTION OF DRAWINGS

[0029] Figure 1 is the overall architecture and training process schematic diagram of the FedGD method proposed in the present application;

[0030] Figure 2 is a flowchart of a heterogeneous label noise robust federated learning method based on group dynamics theory proposed in the present application;

[0031] Figure 3 is a comparison chart of classification accuracy of the method and other baseline models in the present application on CIFAR-10 and CIFAR-100 data sets;

[0032] Figure 4 is a box plot comparison of classification accuracy of the method and other baseline models in the present application on CIFAR-10 and CIFAR-100 data sets;

[0033] Figure 5 is a paired ANOVA test result table of the performance of the method and the baseline model in the present application on CIFAR-10 and CIFAR-100 data sets;

[0034] Figure 6 is a comparison chart of ablation experiment results of key components in the method of the present application;

[0035] Figure 7 is a confusion matrix visualization chart of the effectiveness of the group dynamics driven noise stratification mechanism (GD-NS) in the method of the present application;

[0036] Figure 8is a t-SNE visualization comparison chart of the influence of the client-aware negative distillation mechanism (CA-ND) in the method of the application on feature embedding;

[0037] Figure 9 is a comparison chart of the accuracy rate difference between the threshold value dynamic adjustment mechanism (ATA) in the method of the application and the baseline method;

[0038] Figure 10 is a Top-1 accuracy rate convergence curve comparison chart of the ATA mechanism in the method of the application and the baseline method in the training process. DETAILED DESCRIPTION

[0039] The technical solutions in the embodiments of the application will be clearly and completely described below with reference to the drawings in the embodiments of the application. Obviously, the described embodiments are only part of the embodiments of the application, rather than all the embodiments of the application. Based on the embodiments in the application, all other embodiments obtained by those skilled in the art without creative labor fall within the scope of protection of the application.

[0040] The terms "first", "second", and the like in the specification and claims of the application are used to distinguish similar objects, and are not used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged under appropriate circumstances, so that the embodiments of the application can be implemented in an order other than those illustrated or described herein. In addition, "and / or" in the specification and claims indicates at least one of the connected objects, and the character " / ", generally indicates that the front and rear associated objects are in an "or" relationship. In the description of the application, the meaning of "multiple" is two or more, unless otherwise explicitly specified.

[0041] Embodiment 1

[0042] Please refer to Figure 1 , which shows the overall architecture of the FedGD method proposed in the application. In each training round t, the server first classifies the participating clients according to the historical information. After receiving the global model, the client adopts the corresponding local training strategy according to its own category, and uploads the updated local model to the server. After receiving the local model, the server uses the uncertainty index and the dynamic threshold to accurately classify the client, and then performs client-aware differentiated aggregation. At the same time, the server adjusts the classification threshold according to the change of the global model accuracy rate, and prepares for the next round.

[0043] In order to facilitate understanding, first, the problem of the embodiment is formally defined:

[0044] In the context of federated learning, the application defines a training problem with heterogeneous label noise. It is assumed that there is a large-scale data set , comprising samples and class labels, and the dataset is distributed among clients. A general dataset can be formally represented as , where denotes the local dataset of client , comprising samples, i.e. . Each sample denotes the th data point of client , where the index set is defined as . These data points are independently drawn from a local distribution . This local distribution can be either independent and identically distributed (IID) or non-independent and identically distributed (Non-IID). Consistent with typical FL scenarios, the present invention assumes a Non-IID setting where each client’s label space constitutes a subset of the global label set .

[0045] Existing federated learning methods usually assume that the label noise rate is bounded and relatively low, but real-world federated learning systems exhibit heterogeneous label noise across clients, which varies widely, from low levels (e.g., 5%) to extreme levels (e.g., 100%). To address this challenge, the present invention proposes a framework that can handle the full spectrum of heterogeneous label noise rates, thereby improving robustness in various client noise environments.

[0046] Next, as shown in Figure 2 , in each federated learning training round, the method of the present invention includes the following steps:

[0047] S1, a client classification step: based on the prediction uncertainty values of respective local models received from a plurality of clients, and according to a dynamic classification threshold, the plurality of clients are divided into at least three preset categories, including a mild noise category, a moderate noise category and an extreme noise category.

[0048] The behavior of clients in federated learning systems is inherently dynamic and influenced by a multitude of factors, including data heterogeneity, training stability, and global aggregation feedback. Traditional approaches often adopt a fixed or binary classification of clients (e.g., clean vs. noisy), which fails to capture the transitional and evolving nature of clients in the real world. To address this limitation, inspired by the concept of "individual role dynamics" in group dynamics theory, which posits that individuals in a group continually adjust their roles in response to internal and external interactions. Based on this perspective, the invention models each client as a developing individual whose role (reflected by its contribution quality) can change over time. This enables a more nuanced and adaptive classification of clients beyond static binary distinctions.

[0049] Specifically, according to the uncertainty threshold, clients are divided into three groups: Mildly noisy (MN, denoted as ), Moderately noisy (MD, denoted as ), and Extremely noisy (EN, denoted as ). In each communication round, the server randomly selects a subset of clients and calculates their uncertainty values using the Monte Carlo dropout method (Gal and Ghahramani 2016), which links prediction confidence to label noise—clients with higher noise tend to produce higher entropy. Let denote the local model parameters of client in the th round. The prediction probability of client for class is given by the following equation (1):

[0050]

[0051] where denotes the input data with label , is the public unlabeled dataset, denotes the parameters of the local client model, and denotes the distribution of these parameters, which are trained on local data . The approximate form obtained by Monte Carlo sampling is as follows:

[0052]

[0053] where is the number of times inference is performed for each sample, and denotes the prediction probability of class The parameters of the sampling are next computed. Then, the uncertainty of each client is computed to measure the uncertainty of the model, defined as the average cross-entropy over all samples:

[0054]

[0055] Based on the computed uncertainty values , the clients are classified into three categories:

[0056]

[0057] where and are dynamic thresholds, corresponding to the boundaries of light and moderate noise, respectively. Unlike binary classification, the three-layer stratification of the present invention explicitly captures clients with moderate noise, which are often misclassified by simpler classification methods. This distinction allows for customized treatment: fully exploiting reliable clients, carefully guiding moderate clients, and effectively isolating those with extreme noise.

[0058] Further, the clients employ corresponding training strategies and upload the model.

[0059] Inspired by the theory of group dynamics, which states that "the whole is greater than the sum of its parts", the present invention customizes training for the MN, MD, and EN clients determined in the tthround. These strategies improve data utilization and the robustness of the model. The specific methods for the t+1thround are as follows:

[0060] (1) MN clients

[0061] For MN clients, their local training follows the standard federated learning procedure, where the local model is updated using the complete local dataset and then uploaded to the server. The update rule is given by:

[0062]

[0063] where denotes the model parameters of client k in the tthround, is the learning rate, denotes the loss function computed on the dataset , is the gradient operator with respect to the parameters used to compute the loss function with respect to .

[0064] (2) MD clients

[0065] While previous methods (Lu et al. 2024; Ji et al. 2024) performed well with extremely noisy clients, they often failed to provide effective guidance when dealing with moderately labeled clients who had both clean and noisy labels. Drawing on group dynamics theory, which emphasizes the importance of transitional individuals in balancing group behavior, this study proposes a confidence-based gradient adjustment mechanism to better utilize uncertain but information-rich data. Specifically, during local training, this invention uses the maximum softmax probability of the predicted class to evaluate the confidence of each sample, denoted as... Dynamic thresholds are used. The samples were divided into high-confidence and low-confidence groups. If If the sample is clean or only slightly noisy, this invention reduces its gradient weights to mitigate the risk of overfitting to label noise. If If the sample is uncertain or heavily noisy, its gradient is fully preserved. This adaptive strategy allows the model to focus more on uncertain samples, thereby improving its robustness and generalization ability under label noise. The enhanced loss function is defined as follows:

[0066]

[0067] in It is a sample Cross-entropy loss, It is a sample The maximum softmax probability, where τ is the dynamic confidence threshold. It is an indicator function used to select samples with low confidence.

[0068] The MD client's local model is updated as follows:

[0069]

[0070] Through this confidence modulation strategy, the MD client can achieve guided optimization, gradually transforming into an MN client and unlocking its potential value. This optimization not only improves the model's learning performance but also enhances the robustness and generalization ability of the global model.

[0071] (3) EN client

[0072] Because the EN client contains a high proportion of noisy labels, training directly on its raw data may lead to learning incorrect patterns. To address this issue, this study employs a dual-model training mechanism. Specifically, the first model uses supervised learning on the complete local dataset. The second model is trained using an unsupervised method that discards noisy labels and updates parameters based on pseudo-labels generated by the global model in the previous round. This design aims to reduce the adverse effects of noisy supervision. The update rule for the second model is given by:

[0073]

[0074] where denotes the local data assigned pseudo-labels by the global model During global aggregation, the models of EN clients are uploaded and processed separately by the server, which helps the negative distillation process to further improve the generalization ability of the global model.

[0075] S2, a model aggregation step: receiving the updated local models uploaded from the plurality of clients, and performing differential aggregation to generate an updated global model, the differential aggregation comprising: generating a preliminary global model by weighted average based on the updated local models of the clients of the light noise category; updating the preliminary global model based on the updated local models of the clients of the moderate noise category and the extreme noise category using a client-aware negative distillation strategy, wherein the preliminary global model is taken as a student model, the local models of the clients of the moderate noise and extreme noise categories are taken as teacher models, a first distillation strength is applied to the clients of the moderate noise category to guide the student model to moderately deviate from its prediction, a second distillation strength is applied to the clients of the extreme noise category to guide the student model to strongly deviate from its prediction, and the second distillation strength is greater than the first distillation strength.

[0076] In federated learning, due to training on high-noise data, extreme noise clients (EN) often produce severely erroneous predictions. Directly including these clients in model aggregation can seriously mislead global model updates, leading to performance degradation. While completely excluding these clients can improve global performance, this contradicts the principle of federated learning "utilizing all available data", and some error node clients may still contain valuable local information. To solve this problem, the invention proposes a client-aware negative distillation strategy, which is inspired by the negative distillation idea in FedNed. Based on the theory of group dynamics, MD clients are considered "directable teachers", whose predictions, although biased, still have some reference value. In contrast, EN clients are considered "bad teachers" because their predictions are significantly misleading. The student model should moderately deviate from the predictions of MD clients, while strongly deviating from the erroneous predictions of EN clients. The server first aggregates only the models of MN clients to initialize the student model:

[0077]

[0078] where is the global student model parameter obtained after the t+1th round of aggregation, is the number of training samples in client k, and N is the total number of training samples of the MN clients selected in the tth round. For MD clients, the student model should deviate moderately from their predictions. The negative distillation loss of MD clients is defined as:

[0079]

[0080] For EN clients, if their predictions are highly misleading, the present invention will increase the distillation penalty to force the student model to strongly deviate from its output:

[0081]

[0082] where is the negative distillation loss of EN clients, is the softmax function. The output vector of the model represents the prediction probability of the input . The inverse output is used to guide the student model, encouraging it to deviate from incorrect predictions. The final client-aware negative distillation loss is:

[0083]

[0084] where µ is a weight factor used to control the strength of deviation from the EN client's prediction, enabling the student model to adjust its learning direction according to the client noise level. This design enhances the robustness and generalization ability of the global model. Finally, the updated student model serves as the new global model broadcasted back to each client.

[0085] S3, threshold adjustment step: based on the accuracy change of the updated global model and the last round global model on the validation set, dynamically adjust the dynamic classification threshold used in the client classification step.

[0086] The client noise level in federated learning often fluctuates during training, leading to insufficient classification methods based on fixed thresholds. Misclassification due to static thresholds can degrade the performance and stability of the global model. To overcome this problem, the invention proposes a dynamic threshold adjustment mechanism driven by accuracy feedback, which adaptively adjusts the client classification boundary to better reflect the changing noise characteristics. The mechanism is inspired by the theory of group dynamics and adaptive optimization strategies. Group dynamics emphasizes the continuous evolution of individual states, indicating that static classification thresholds are not suitable for dynamic environments. Similarly, adaptive optimization techniques dynamically adjust hyperparameters to improve convergence and stability. The invention adjusts the classification threshold based on the observed change in client accuracy and to unify these ideas.

[0087] The invention defines the accuracy change between two consecutive rounds as: where and represent the global model accuracy of the th and th round, respectively. When exceeds a predefined threshold δ, it indicates that the model performance has changed significantly, prompting the adjustment of the classification threshold. Specifically, if , the current threshold may be too strict, excluding beneficial clients; therefore, relax these thresholds to enhance generalization ability. Conversely, if , tighten the threshold to mitigate the impact of unreliable clients. When the change is within the tolerance range ( ), the threshold remains unchanged.

[0088] The update rule for the threshold is:

[0089]

[0090]

[0091] where , is the adjustment step size coefficient, used to control the magnitude of threshold update, is the adjustment factor, , and represent the uncertainty quantiles of MN, MD, and EN clients, respectively. Since MN has low uncertainty, the invention uses a quantile of 50%; for MD, to reflect moderate uncertainty, the invention uses a quantile of 75%; for EN, to capture high uncertainty, the invention uses a quantile of 90%. This ensures robust threshold updates.

[0092] Adjustment factor is defined as:

[0093]

[0094] This accuracy-based threshold adjustment mechanism allows the classification criteria to evolve with changes in client behavior, thereby enhancing the system's ability to distinguish between clients with different noise levels and improving the robustness and generalization ability of the global model.

[0095] It should be noted that the method of the present application mainly includes the following technologies:

[0096] (1) Three-tier client classification mechanism (GD-NS): The present application first divides the participating clients into three levels based on client prediction uncertainty, and labels them as mild noise, moderate noise and extreme noise clients. Unlike traditional binary classification methods, this mechanism can distinguish moderate noise clients with transitional noise levels, so that different types of clients can be treated differently during the global model aggregation process, preserving valuable information while reducing the negative impact of high-noise updates on the model.

[0097] (2) Client-aware negative distillation strategy (CA-ND): For moderate noise clients, the present application uses a mild negative distillation strategy to make the global model deviate from its potentially misleading updates during aggregation, achieving guidance and utilization of its potential learning value; for extreme noise clients, a stronger negative distillation mechanism is used to significantly deviate the global model from its updates, maximizing the suppression of its negative impact on model performance. This strategy applies asymmetric influence weights to different noise levels, achieving fine-tuned regulation of the global model for noisy clients.

[0098] (3) Accuracy-based dynamic threshold adjustment (ATA): The present application further introduces a dynamic threshold adjustment mechanism to automatically adjust the client classification threshold based on the accuracy of the global model during training, achieving dynamic reallocation of client roles. This mechanism can adapt to noise fluctuations and changes in client behavior during training, avoiding misjudgment problems that may arise from static thresholds, thereby improving the stability and generalization ability of the global model under dynamic noise conditions.

[0099] Through the above technologies, the present application can effectively deal with federated learning problems in a heterogeneous noise environment while maintaining the accuracy of the global model, achieving differentiated treatment and information utilization for clients with different noise levels. Specifically, it fully extracts information from mild noise clients, implements potential guidance on moderate noise clients, and effectively suppresses extreme noise clients, significantly enhancing the robustness and generalization ability of the global model.

[0100] In summary, on the one hand, the present application proposes FedGD, a new federated learning framework inspired by group dynamics theory, which solves the limitations of binary client classification by modeling the evolving client roles under heterogeneous and dynamic label noise in a more fine-grained manner. On the other hand, the present application develops a three-level classification mechanism based on prediction uncertainty and designs a noise level-aware training strategy, including confidence-guided optimization for moderately noisy clients and client-aware negative distillation for extremely noisy clients. In addition, a dynamic threshold adjustment strategy is introduced to adapt to the evolving characteristics of client noise during training.

[0101] The method proposed by the present application also achieves higher classification accuracy and statistical significance. As shown in Figure 3 and Figure 4 , the classification accuracy of the method (FedGD) of the present application is significantly better than that of the baseline and advanced methods such as FedAvg, FedProx, FedNed, etc. under various heterogeneous noise settings on CIFAR-10 and CIFAR-100 datasets. The accuracy is improved by 0.8%-1.4% on CIFAR-10 and 2.1%-6.9% on CIFAR-100, and the performance variance is smaller, proving its superior robustness and generalization ability. To further verify the statistical significance of this superiority, as shown in Figure 5 , the present application performs paired ANOVA test on the performance of FedGD and baseline methods. The results show that in most comparisons, the p-value is less than 0.05, which means that the method of the present application is statistically significantly better than most baseline methods including FedAvg, FedProx, SCAFFOLD, FedNoRo, thus firmly proving its effectiveness at the data level. Ablation experiments ( Figure 6 ) also confirm that each core component (GD-NS, CA-ND, ATA) proposed by the present application makes a key and complementary contribution to performance improvement.

[0102] To further confirm the independent contribution of each core component of the present application, a series of ablation experiments were conducted, and the results are as follows:

[0103] Verification of the effectiveness of the group dynamics-driven noise stratification mechanism (GD-NS): As shown in Figure 7 , the effectiveness of the mechanism is verified by comparing the confusion matrices before and after introducing the GD-NS mechanism. Figure 7 (a) is the result of the baseline model, Figure 7 (b) is the result after introducing GD-NS, Figure 7(c) is the difference matrix of the two. Experiments show that the overall accuracy of the model is greatly improved from 59.1% to 80.9% by introducing only the GD-NS mechanism. From the difference matrix, it can be seen that the positive values on the diagonal line (such as car + 282, bird + 237, and frog + 417) represent a significant increase in the number of correctly classified samples, while the values on the non-diagonal line are generally reduced, indicating that misclassification is effectively inhibited. Specifically, the classification accuracy of 9 out of 10 classes is improved, among which the accuracy of frogs, cars, and birds is increased by 41.7%, 27.2%, and 23.7%, respectively. This fully demonstrates the great advantage of the three-layer client classification mechanism in identifying and distinguishing different noise level clients.

[0104] Validation of the effectiveness of the client-aware negative distillation mechanism (CA-ND): As shown in Figure 8 , the t-SNE visualization technology shows the impact of introducing the CA-ND mechanism on the feature embedding space learned by the model. Figure 8 (a) is the baseline model without applying CA-ND, and its feature distribution presents poor intra-class compactness and significant inter-class overlap. In contrast, Figure 8 (b) is the model after introducing CA-ND, and the feature clusters learned by it are significantly more compact, and the inter-class boundaries are also clearer, such as the separation degree between class 0 (Class 0) and class 9 (Class 9) is significantly improved. This shows that by applying differential negative distillation to moderate and extreme noise clients, CA-ND can effectively guide the global model to learn more discriminative feature representations, thereby enhancing the model's intra-class aggregation and inter-class separation capabilities and improving the overall robustness.

[0105] Validation of the effectiveness of the threshold dynamic adjustment mechanism (ATA) based on accuracy: Figure 9 and Figure 10 together show the superiority of the ATA mechanism compared to the baseline method using a static threshold. As shown in Figure 9 , after the training enters the stable period (after about the 20th round), the model using the ATA mechanism always maintains a positive accuracy margin over the baseline model, and its advantage ratio (black curve) steadily rises and eventually exceeds 70%, indicating that ATA is superior to the static threshold method in most of the training time. More importantly, ATA significantly reduces the performance instability in the early stages of training caused by noisy clients. Figure 10From the perspective of convergence dynamics, it further confirms the advantage of ATA. The model using ATA shows faster convergence speed (more than 80% accuracy in the 40th round, while the baseline reaches 78% in the 70th round) and achieves higher final accuracy (86%). In addition, the accuracy curve of ATA shows an almost monotonic upward trend with small fluctuations in the later period, while the baseline model has at least three significant performance backfalls. This proves that ATA not only improves the final performance of the model, but also enhances the stability and convergence efficiency of the training process, enabling the system to more effectively extract and retain knowledge.

[0106] In summary, the ablation experiments have proved the effectiveness of the three core components GD-NS, CA-ND and ATA proposed in the present application and their complementary interaction, which together constitute the outstanding performance of the method of the present application in dealing with heterogeneous label noise.

[0107] Embodiment 2

[0108] Alternatively, the embodiments of the present application also provide an electronic device, which comprises a processor, a memory, a program or instructions stored in the memory and executable on the processor, the program or instructions being executed by the processor to implement the processes of each of the above-mentioned embodiments of the group dynamics-based robust federated learning method for heterogeneous label noise and achieve the same technical effects. To avoid repetition, details are not repeated here.

[0109] Embodiment 3

[0110] The embodiments of the present application also provide a readable storage medium having a program or instructions stored thereon, the program or instructions being executed by a processor to implement the processes of each of the above-mentioned embodiments of the group dynamics-based robust federated learning method for heterogeneous label noise and achieve the same technical effects. To avoid repetition, details are not repeated here.

[0111] The processor is the processor of the electronic device described in the above embodiments. The readable storage medium includes a computer readable storage medium, such as a computer readable memory (Read-Only Memory, ROM), a random access memory (Random Access Memory, RAM), a magnetic disk or an optical disk, etc.

[0112] It should be noted that, in the present document, the terms "comprises", "comprising", or any other variations thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but can include other elements not expressly listed or inherent to such process, method, article, or apparatus. An element proceeded by "comprises... a" does not, without more constraints, exclude the presence of additional identical elements in the process, method, article, or apparatus that comprises the element. Furthermore, it is to be understood that the method and apparatus of the present application can be carried out by more than one process, method, article, or apparatus either simultaneously, concurrently, or with intervening action that are carried out at the same time, either in a simultaneous fashion or in a fashion that is interleaved in time. For example, the described methods can be performed in a different order from that described, and / or various steps can be combined or omitted, and / or additional steps can be added, without departing from the scope of the present application. Also, features described with respect to certain examples can be combined in other examples.

[0113] From the above description of the embodiments, it is apparent that the above-described method of the embodiments can be realized by means of software and general-purpose hardware platforms, and of course, can also be realized by hardware, but in many cases, the former is a better embodiment. Based on such an understanding, the technical solutions of the present application can be embodied in the form of a software product, which is stored in a storage medium (such as a ROM / RAM, a magnetic disk, or an optical disk) and includes a number of instructions for causing a terminal (which can be a mobile phone, a computer, a server, an air conditioner, or a network device) to execute the methods described in the various embodiments of the present application.

[0114] The embodiments of the present application are described above with reference to the accompanying drawings, but the present application is not limited to the above-described specific embodiments, and the above-described specific embodiments are merely illustrative, rather than limiting, and a person of ordinary skill in the art can make many forms under the inspiration of the present application without departing from the scope of the present application and the protection scope of the claims, and all of them belong to the protection scope of the present application.

Claims

1. A heterogeneous label noise robust federated learning method based on group dynamics, characterized in that, executed by the server computer, in each federated learning training round, comprises the following steps: a client classification step: based on the prediction uncertainty values of the respective local models received from the plurality of clients, and according to a dynamic classification threshold, the plurality of clients are divided into at least three preset categories, including a mild noise category, a moderate noise category and an extreme noise category; a model aggregation step: receiving the updated local models uploaded from the plurality of clients, and performing differential aggregation to generate an updated global model, the differential aggregation comprising: based on the updated local models of the clients in the mild noise category, a preliminary global model is generated by weighted average; based on the updated local models of the clients in the moderate noise category and the extreme noise category, a client-aware negative distillation strategy is used to update the preliminary global model, wherein the preliminary global model is taken as a student model, the local models of the clients in the moderate noise category and the extreme noise category are taken as teacher models, a first distillation intensity is applied to the clients in the moderate noise category to guide the student model to moderately deviate from its prediction, a second distillation intensity is applied to the clients in the extreme noise category to guide the student model to strongly deviate from its prediction, and the second distillation intensity is greater than the first distillation intensity; a threshold adjustment step: based on the accuracy change of the updated global model and the last round global model on the validation set, the dynamic classification threshold used in the client classification step is dynamically adjusted, and the process of increasing or decreasing the dynamic classification threshold is by updating the current dynamic classification threshold to the combination of a certain quantile of the prediction uncertainty values of the clients in the mild noise category, the moderate noise category and the extreme noise category and an adjustment factor; wherein the certain quantile includes: a first classification threshold for determining the boundary between the mild noise category and the moderate noise category, which is updated based on the 50% quantile of the prediction uncertainty values of the clients in the mild noise category; and a second classification threshold for determining the boundary between the moderate noise category and the extreme noise category, which is updated based on the 75% quantile of the prediction uncertainty values of the clients in the moderate noise category and the 90% quantile of the prediction uncertainty values of the clients in the extreme noise category.

2. The heterogeneous label noise robust federated learning method based on group dynamics according to claim 1, wherein, In the client classification step, the process of obtaining the prediction uncertainty value comprises: for the local model of each client, a Monte Carlo dropout method is used for multiple forward inferences to obtain the prediction probability distribution for each local data sample; and based on the prediction probability distribution, the average cross-entropy on all local data samples is calculated, and the average cross-entropy is taken as the prediction uncertainty value of the client.

3. The heterogeneous label noise robust federated learning method based on group dynamics according to claim 1, wherein, The step of updating the preliminary global model using the client-aware negative distillation strategy comprises: initializing the preliminary global model as a student model; calculating the negative distillation loss of each client in the moderate noise category and the extreme noise category respectively, the negative distillation loss is defined by minimizing the difference between the inverse output of the student model and the teacher model; and, The student model is updated based on the negative distillation loss of the moderate noise category and the extreme noise category to obtain an updated global model.

4. The heterogeneous label noise robust federated learning method based on group dynamics according to claim 3, wherein, The negative distillation loss of each client in the moderate noise category and the extreme noise category is calculated respectively, specifically as follows: A final client-perceived negative distillation loss is calculated, which is the sum of the negative distillation loss of the clients in the moderate noise category and the weighted negative distillation loss of the clients in the extreme noise category. The weighting is achieved by multiplying the negative distillation loss of the clients in the extreme noise category by a preset weight factor.

5. The heterogeneous label noise robust federated learning method based on group dynamics according to claim 1, wherein, The threshold adjustment step specifically includes: An accuracy change value between the updated global model and the global model of the previous round is calculated. When the accuracy change value is greater than a preset positive threshold, the dynamic classification threshold is increased. When the accuracy change value is less than a preset negative threshold, the dynamic classification threshold is decreased.

6. The heterogeneous label noise robust federated learning method based on group dynamics according to claim 1, wherein, The method further includes a step of performing local training by the plurality of clients before the model aggregation step, wherein the local training of the clients in the moderate noise category adopts a confidence-based gradient adjustment mechanism, including: For each training sample in the local data set, the maximum softmax probability of its predicted category is calculated as the confidence. The confidence is compared with a dynamic confidence threshold to divide the training samples into high-confidence samples and low-confidence samples; and, When calculating the loss function of the local training, the gradient weight of the high-confidence samples is reduced by an indicator function, and the gradient weight of the low-confidence samples is completely retained.

7. The heterogeneous label noise robust federated learning method based on group dynamics according to claim 1 or 6, characterized in that, The local training of the clients in the extreme noise category adopts a double-model training mechanism, including: Training a first model: using the original label for standard supervised learning training on the complete local data set; and Training a second model: discarding the original label, generating pseudo-labels for the local data based on the global model of the previous round, and performing unsupervised learning training; Wherein, the combination or one of the first model and the second model uploaded to the server for negative distillation.

8. An electronic device, comprising: A processor, a memory, and a program or instructions stored on the memory and executable on the processor, wherein the program or instructions are executed by the processor to implement the steps of the method according to any one of claims 1-7.

9. A readable storage medium, characterized by, A readable storage medium stores a program or instructions, wherein the program or instructions are executed by a processor to implement the steps of the method according to any one of claims 1-7.

Images