Robust federated learning method based on space-time collaborative optimization
By co-optimizing on the client and server sides, and employing an adaptive training mechanism and momentum adjustment, the model convergence problem caused by data heterogeneity and sparse participation in federated learning is solved, achieving efficient training and stable convergence in resource-constrained environments.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- ANHUI UNIVERSITY OF TECHNOLOGY
- Filing Date
- 2026-01-21
- Publication Date
- 2026-04-28
AI Technical Summary
Existing federated learning techniques suffer from high computational resource requirements and improper momentum adjustment when dealing with model convergence problems caused by statistical heterogeneity of data and sparse client participation, making it difficult to achieve effective convergence in complex heterogeneous environments.
A robust federated learning method based on spatiotemporal co-optimization is adopted. By combining the adaptive training mechanism on the client side and the dynamic momentum aggregation strategy on the server side, the model update process is optimized, the computational resource requirements are reduced, and the optimization trajectory oscillation is suppressed.
It improves the robustness and performance of federated learning in complex and heterogeneous environments, enhances the convergence speed and training stability of the model, and is suitable for resource-constrained edge devices.
Smart Images
Figure CN121936629A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the fields of distributed machine learning and Internet of Things (IoT) technology, specifically a robust federated learning method based on spatiotemporal collaborative optimization. Background Technology
[0002] With the widespread adoption of IoT devices, massive amounts of data are generated at the edge. To leverage this dispersed data for model training while protecting user privacy, federated learning has emerged. Federated learning allows clients to upload local model parameters without sharing the original data, which are then aggregated by a central server to collaboratively train the global model. However, in complex real-world scenarios, existing federated learning technologies still face two core challenges: first, statistical heterogeneity of the data leads to discrepancies between the local training objectives of different clients and the global objective, causing client drift and hindering effective model convergence in the spatial dimension; second, the sparse and dynamic participation of clients results in random variations in the subset of clients in each round of aggregation, causing oscillations in the global model's optimization trajectory in the temporal dimension, making stable convergence difficult.
[0003] To address these challenges, existing technologies typically perform optimizations in isolation, which has significant limitations. On the client side, methods such as sharpness-aware minimization, introduced to improve the model's generalization ability to heterogeneous data, often require high computational overhead and are difficult to apply to resource-constrained edge devices. On the server side, momentum mechanisms used to smooth the optimization trajectory typically have fixed momentum coefficients, which cannot adaptively adjust according to the degree of conflict in client updates during training. This can lead to convergence lag when updates are consistent, and an inability to effectively suppress oscillations when updates conflict. These methods fail to coordinate the coupling between spatial inconsistency and temporal instability, thus limiting the convergence efficiency and final performance of federated learning in heterogeneous environments. Summary of the Invention
[0004] In order to solve the technical problems mentioned in the background art, the present invention is proposed.
[0005] Therefore, the technical solution adopted by the present invention is as follows:
[0006] A robust federated learning method based on spatiotemporal collaborative optimization, comprising at least one server and multiple clients, including:
[0007] Step 1: The server initializes or obtains the model dataset from the previous communication round, including model parameters, global momentum, and global sharpness beliefs; in each communication round, a certain number of client subsets are selected from all the clients based on preset filtering conditions, and the model parameters are distributed; the clients perform local training of the global model based on a preset adaptive training mechanism.
[0008] Step 2: After completing local training, the client uploads the local data to the server; the local data includes the update amount and sharpness value of the global model; the server aggregates all local data and calculates the statistical characteristics of the update amount and the average sharpness value.
[0009] Step 3: Update the global sharpness belief based on the average sharpness value to obtain a new global sharpness belief; calculate the adaptive momentum coefficient based on the statistical features and the new global sharpness belief; fuse the average update amount in the statistical features and the global momentum based on the adaptive momentum coefficient to obtain a new global momentum; update the model parameters based on the new global momentum.
[0010] Step 4: Iterate through Step 1, Step 2, and Step 3 until the preset termination condition is met to obtain the final global model.
[0011] Furthermore, the screening criteria include basic availability criteria and random sampling;
[0012] The basic availability conditions refer to the availability conditions set according to the actual application scenario;
[0013] The random sampling refers to randomly or selectively choosing a certain number of clients from those that meet the basic availability conditions to form a subset of clients participating in the training.
[0014] Furthermore, the local training process includes K iteration steps;
[0015] Before local training, fast weights and slow weights are set, and the fast weights and slow weights are equal to the model parameters; the fast weights will continuously change during local training, while the slow weights remain unchanged during local training.
[0016] Furthermore, the local training process of the adaptive training mechanism is as follows:
[0017] The first step is to randomly sample a batch of data from the local dataset stored in the client.
[0018] The second step is to calculate the Euclidean distance between the fast weights and the slow weights.
[0019] The third step is to determine the training mode of the global model based on the Euclidean distance and the preset perturbation radius threshold.
[0020] If the Euclidean distance is greater than or equal to the perturbation radius threshold, the fast weights are updated using a computationally inefficient optimizer.
[0021] If the Euclidean distance is less than the perturbation radius threshold, the fast weights are updated using a sharpness-aware minimization optimizer.
[0022] The second and third steps are executed only in the first iteration step, and subsequent iteration steps update the fast weights based on the optimizer determined in the third step;
[0023] Step 4, complete. After one iteration, the client generates model parameters from the obtained fast and slow weights using linear interpolation, with the following interpolation formula:
[0024]
[0025] in, For the t-th communication round The model parameters for each client, where t is greater than 1. These are the interpolation coefficients. For slow weights, For the t-th communication round Fast weighting for each client.
[0026] Furthermore, the formula for calculating the update amount is as follows:
[0027]
[0028] in, For the first The update amount of each client in the t-th communication round;
[0029] The formula for calculating the sharpness value is as follows:
[0030]
[0031] in, For the first The sharpness value of a client in the t-th communication round. Let be the perturbation vector. For loss function, The threshold value for the disturbance radius. Let L2 be the norm of the perturbation vector.
[0032] Furthermore, the statistical characteristics of the update quantity include the average update quantity and the update quantity variance;
[0033] The formula for calculating the average sharpness value is as follows:
[0034]
[0035] in, Let be the average sharpness value in the t-th communication round. This represents the number of clients in the client subset.
[0036] Furthermore, the server uses an exponential moving average mechanism to smoothly update the global sharpness belief, with the following update formula:
[0037]
[0038] in, For the new global sharpness belief in the t-th communication round, For the sake of overall sharpness belief, is the smoothing coefficient for the global sharpness belief.
[0039] Furthermore, the calculation process for the adaptive momentum coefficient is as follows:
[0040] Dynamic sensitivity is calculated based on the new global sharpness belief. The formula is as follows:
[0041]
[0042] in, The preset base sensitivity;
[0043] Calculate the time window by combining the statistical characteristics of dynamic sensitivity and update volume. The formula is as follows:
[0044]
[0045] in, This is the maximum upper limit of the preset time window. It is a preset minimum constant. To update the variance, This is the average update amount. The square of the L2 norm of the average update amount;
[0046] The adaptive momentum coefficient is calculated using the time window, as shown in the following formula:
[0047]
[0048] in, For the first The adaptive momentum coefficient for each communication round.
[0049] Furthermore, the formula for obtaining the new global momentum is as follows:
[0050]
[0051] in, For new global momentum, For global momentum.
[0052] The update formula for the model parameters is as follows:
[0053]
[0054] in, For the updated model parameters, This represents the global learning rate.
[0055] Furthermore, the termination conditions include a preset number of communication rounds and the model convergence state.
[0056] Compared with the prior art, the advantages of the present invention are as follows:
[0057] 1. This invention solves the problem that existing sharpness perception methods have excessively high requirements for edge device computing resources, making them difficult to deploy in practice, by using an adaptive training mechanism on the client side and enabling optimization strategies with high generalization performance only when the model updates into a stable neighborhood, while adopting conventional methods with low computational overhead during the exploration phase.
[0058] 2. This invention dynamically adjusts the smoothing intensity during aggregation by sensing the degree of conflict in client updates and the geometric characteristics of the global model. This adaptively suppresses optimization trajectory oscillations caused by data heterogeneity and sparse participation, overcoming the blindness of traditional fixed momentum strategies.
[0059] 3. This invention combines efficient local optimization by the client for data heterogeneity with intelligent global aggregation by the server for dynamic participation. The two form a positive feedback loop, breaking the vicious cycle of mutually reinforcing spatial inconsistency and temporal instability, thereby improving the overall robustness and final performance of the federated learning system in complex heterogeneous environments. Attached Figure Description
[0060] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.
[0061] Fig. 1 This is a schematic diagram of the method flow of the present invention;
[0062] Fig. 2 This is a schematic diagram illustrating the interaction between the server and client in this invention;
[0063] Fig. 3 This is a schematic diagram of the adaptive training mechanism of the present invention. Detailed Implementation
[0064] To achieve the above objectives, this invention provides a robust federated learning method based on spatiotemporal collaborative optimization. Please refer to [link to relevant documentation]. Figs. 1-2 ,include:
[0065] Step 1: The server initializes or obtains the model dataset from the previous communication round, including model parameters, global momentum, and global sharpness beliefs; in each communication round, a certain number of client subsets are selected from all the clients based on preset filtering conditions, and the model parameters are distributed; the clients perform local training of the global model based on a preset adaptive training mechanism.
[0066] If it is the first communication round, the model parameters, global momentum, and global sharpness belief are initially assigned values. The model parameters are randomly initialized, the global momentum is initially set to a 0 vector, and the global sharpness belief is initially set to 0.
[0067] If it is not the first communication round, the model parameters, global momentum, and global sharpness beliefs retained after the end of the previous communication round are directly obtained.
[0068] The screening criteria include basic availability criteria and random sampling;
[0069] Basic availability conditions refer to setting availability conditions based on actual application scenarios, such as the client's current network status, remaining battery power, historical participation, and other indicators, and selecting clients with stable status and training capabilities.
[0070] Random sampling refers to randomly or selectively choosing a certain number of clients from those that meet the basic availability conditions to form a subset of clients participating in training. For example, typically 5%-10% of the total number of clients is selected, and the specific proportion can be flexibly adjusted according to the actual resource allocation.
[0071] Finally, the server will broadcast the initialized or stored model parameters to each client in the client subset via a secure communication protocol, providing a basic model basis for the client's local training.
[0072] After receiving the model parameters from the server, each client in the client subset starts the adaptive training mechanism and conducts local training.
[0073] The local training process consists of K iteration steps. The number of iteration steps can be set according to the size of the local dataset stored on the client and the device's computing power. For example, the value can be an integer between 10 and 50.
[0074] Set fast weights and slow weights, and set the fast weights and slow weights equal to the model parameters; the fast weights will change continuously during local training, while the slow weights remain unchanged during local training.
[0075] The adaptive training mechanism adopts different local training mechanisms based on whether the current communication round is the first communication round;
[0076] The local training process for non-first communication rounds is as follows. Please participate. Fig. 3 :
[0077] The first step is for the client to randomly sample a batch of data from the stored local dataset. The size of the batch of data can be flexibly set according to the device's memory, for example, the value can be 32-128.
[0078] The second step involves the client calculating the Euclidean distance between the fast and slow weights, using the following formula:
[0079]
[0080] in, For the first Euclidean distance for each client, For slow weights, For the index of the communication round, For the first The client number Fast weights for each iteration step, where k is 1. The symbol for Euclidean distance calculation;
[0081] The third step is to determine the training mode of the global model based on the Euclidean distance and the preset perturbation radius threshold ρ.
[0082] if If the value is greater than or equal to ρ, it indicates that the local model is still in the rapid exploration phase and deviates significantly from the model parameters. In this case, a computationally inefficient optimizer is used to update the weights quickly, such as the stochastic gradient descent (SGD) optimizer. The update formula is as follows:
[0083]
[0084] in, For the first The client number Fast weights for each iteration step For loss function, For batch data, This represents the gradient update amount for the stochastic gradient descent optimizer.
[0085] if The value <ρ indicates that the global model has initially converged to the neighborhood of the initial baseline model, and the optimization trajectory tends to stabilize. To prevent the model from getting trapped in sharp local optima, a sharpness-aware minimization optimizer (SAM) is used to update the fast weights. The update formula is as follows:
[0086]
[0087] in, To minimize the gradient update amount of the sharpness-aware optimizer;
[0088] The second and third steps are executed only in the first iteration step, that is... When =1, subsequent iterations update the fast weights based on the optimizer determined in the third step;
[0089] Step 4, complete. After one iteration, the client will obtain the fast weights. and slow weights Model parameters are generated through linear interpolation. The interpolation formula is as follows:
[0090]
[0091] in, For the t-th communication round Model parameters for each client, These are the interpolation coefficients;
[0092] In this embodiment, The range of values is For example, a value of 0.3-0.7 is used to balance the exploratory nature of local training with the inheritance of the global model. The larger the value, the more significantly the global model is affected by the fast weights, and the stronger the exploratory nature. The smaller the value, the stronger the inheritance.
[0093] In this embodiment, the perturbation radius threshold ranges from 0.001 to 0.1, and the specific setting is based on the following:
[0094] It needs to match the normal numerical range of the model parameters. For example, the parameters of a neural network model are usually distributed in the range of [-1,1] or [-0.1,0.1]. The perturbation radius threshold needs to be set to a small value within this range, such as 0.01~0.1. If it is set to 1 or larger, it will cause the deviation judgment to fail; if it is set to below 0.0001, it will be oversensitive and lose the significance of stage differentiation.
[0095] The computational overhead on the client side needs to be balanced, and the perturbation radius threshold directly determines the frequency at which the client switches to SAM; the larger the perturbation radius threshold, the easier it is to satisfy the Euclidean distance requirement. <ρ, the earlier you switch to SAM, the stronger the generalization ability may be, but the computational burden on edge devices will increase; the smaller the perturbation radius threshold, the more difficult it is to satisfy the Euclidean distance. <ρ, staying in the SGD stage for a longer time results in lower computational overhead, but may delay the improvement of generalization ability; therefore, resource-constrained edge devices need to be set with a larger perturbation radius threshold, while devices with strong computing power can be set with a smaller perturbation radius threshold.
[0096] The degree of heterogeneity of the data needs to be matched. Highly heterogeneous data requires a larger perturbation radius threshold, such as 0.05~0.1, to avoid prematurely switching to SAM due to misjudgment of convergence; lowly heterogeneous data requires a smaller perturbation radius threshold, such as 0.001~0.01, to ensure that SAM can be switched in time to optimize generalization ability after the model converges.
[0097] The local training mechanism for the first communication round is as follows:
[0098] At this point, the global model parameters are randomly initialized, and the client has no historical training trajectory or data adaptation memory. This is the first exploration starting from scratch. Training is started directly using a low-overhead optimizer, and subsequent iterations are executed according to the logic of the local training mechanism mentioned above for non-first communication rounds.
[0099] Step 2: After completing local training, the client uploads the local data to the server; the local data includes the update amount and sharpness value of the global model; the server aggregates all local data and calculates the statistical characteristics of the update amount and the average sharpness value.
[0100] The formula for calculating the update amount is as follows:
[0101]
[0102] in, For the first The update amount of each client in the t-th communication round.
[0103] The formula for calculating the sharpness value is as follows:
[0104]
[0105] in, For the first The sharpness value of a client in the t-th communication round. The perturbation vector generated for the client, Let L2 norm be the perturbation vector;
[0106] The client packages the update amount and sharpness value and uploads them to the central server through an encrypted communication channel;
[0107] During the upload process, it is necessary to ensure the integrity and security of data transmission to avoid data leakage or tampering. After receiving the data, the server will verify it and store it in the training data cache pool for this round after confirming that it is correct, waiting for all participating clients to complete the data upload.
[0108] The server performs integrity checks on all received data, checking for missing data, incorrect formatting, or transmission anomalies. If a client's data is found to be problematic, the server will notify the client to re-upload. If multiple uploads fail, the client will be removed from the aggregation queue for this round, ensuring the reliability of the aggregated data.
[0109] The statistical characteristics of update volume include average update volume and update volume variance;
[0110] The formula for calculating the average update volume is as follows:
[0111]
[0112] in, For the client subset in the t-th communication round, The number of clients in the client subset;
[0113] The formula for calculating the variance of the update quantity is as follows:
[0114]
[0115] in, Let be the variance of the update amount in the t-th communication round.
[0116] The formula for calculating the average sharpness value is as follows:
[0117]
[0118] in, Let be the average sharpness value in the t-th communication round.
[0119] Step 3: Update the global sharpness belief based on the average sharpness value to obtain a new global sharpness belief; calculate the adaptive momentum coefficient based on the statistical features and the new global sharpness belief; fuse the average update amount in the statistical features and the global momentum based on the adaptive momentum coefficient to obtain a new global momentum; update the model parameters based on the new global momentum.
[0120] The server uses an exponential moving average (EMA) mechanism to smoothly update the global sharpness belief, thereby obtaining a stable new global sharpness belief that reflects the current global model geometry. The update formula is as follows:
[0121]
[0122] in, For the new global sharpness belief in the t-th communication round, For the sake of overall sharpness belief, Smoothing coefficient for global sharpness belief
[0123] In this embodiment, the smoothing coefficient ranges from 0.9 to 0.99, for example, it is 0.95;
[0124] The smoothing coefficient serves to balance the weights of historical and current sharpness information, preventing single sharpness fluctuations from having an excessive impact on global beliefs and ensuring the stability of the perception of the flatness of the lost landscape in which the global model is located.
[0125] The calculation process for the adaptive momentum coefficient is as follows:
[0126] Calculating dynamic sensitivity based on the new global sharpness belief The formula is as follows:
[0127]
[0128] in, This is the preset baseline sensitivity, with a value range of 0.1 to 1.0. Its function is to set the baseline level of sensitivity.
[0129] Dynamic sensitivity Will follow the overall sharpness belief Dynamic adjustment is made when the global model is in a sharp region. The corresponding increase provides a basis for subsequent enhancements to the smoothing effect.
[0130] Calculate the time window by combining the statistical characteristics of dynamic sensitivity and update volume. The formula is as follows:
[0131]
[0132] in, This is the maximum upper limit of the preset time window, with a value ranging from 5 to 20, used to avoid convergence lag caused by an excessively large time window. For example, a preset minimum constant, with a value of 10 ... This is used to prevent calculation errors caused by a denominator of zero. The square of the L2 norm of the average update amount;
[0133] The adaptive momentum coefficient is calculated using a time window, as shown in the following formula:
[0134]
[0135] in, For the first The adaptive momentum coefficient for each communication round.
[0136] The formula for obtaining the new global momentum is as follows:
[0137]
[0138] in, For new global momentum, For global momentum.
[0139] The formula for updating the model parameters is as follows:
[0140]
[0141] in, For the updated model parameters, The global learning rate ranges from 0.001 to 0.01, for example, 0.005. It is used to control the step size of model parameter updates, avoiding model oscillation caused by an excessively large step size or slow convergence caused by an excessively small step size.
[0142] Step 4: Iterate through Step 1, Step 2, and Step 3 until the preset termination condition is met to obtain the final global model.
[0143] The iteration process is as follows:
[0144] After each round of communication is completed, i.e. after step three, the server first confirms the validity of the updated global model parameters, new global momentum, and new global sharpness beliefs to avoid abnormal situations such as numerical overflow and parameter dimension mismatch.
[0145] If the verification passes, the server stores the core data of that round in a dedicated cache space as the initial dataset for the next iteration.
[0146] If the verification finds an anomaly, such as parameter values exceeding a reasonable range, discard the update results of the current round, revert to the initial model dataset of the round, and re-execute steps two to three of the current round, or appropriately adjust hyperparameters such as the global learning rate, sharpness belief smoothing coefficient, etc. to reduce the probability of anomalies.
[0147] Termination conditions include the preset number of communication rounds and the model convergence status;
[0148] The number of communication rounds can be flexibly set based on the data scale, model complexity, and client resource availability in the actual application scenario. A typical range is 50-200 rounds. For example, for simple models and small-scale heterogeneous data, 50-100 rounds can be used; for complex, deep models and highly heterogeneous data, 100-200 rounds can be used. When the current number of communication rounds reaches the preset number, the server directly terminates the iteration process and confirms that the model parameters of the global model updated in the current round are the final model parameters.
[0149] The model convergence status is evaluated based on the global model convergence status. The evaluation metrics include test accuracy and the global loss function value. The specific determination logic is as follows:
[0150] The server pre-stores a public test dataset that is independent of all local data on the clients, for objectively evaluating the generalization performance of the global model;
[0151] For the global model updated in the current communication round, two core metrics are calculated using the test dataset: the classification accuracy of the global model on the test set and the global loss function value of the global model on the test set.
[0152] Two convergence thresholds are set: an accuracy stabilization threshold and a loss function decrease threshold. The typical value for the accuracy stabilization threshold is 0.1%-0.5%, and the typical value for the loss function decrease threshold is 0.001-0.005.
[0153] Evaluation results of N consecutive communication rounds: If, in N consecutive communication rounds, the improvement in test accuracy of each communication round compared to the previous communication round is less than the accuracy stabilization threshold, and the decrease in global loss function value of each communication round compared to the previous communication round is less than the loss function decrease threshold, then the model is determined to have reached stable convergence, the server terminates the iteration process, and the model parameters of the current communication round are the final model parameters.
[0154] This invention proposes a robust federated learning method based on spatiotemporal co-optimization. By combining a low-overhead training mechanism on the client side that adapts to the updated state with a dynamic momentum aggregation strategy on the server side that integrates the updated variance and model sharpness, this method collaboratively solves the spatial inconsistency caused by data heterogeneity and the temporal instability caused by sparse client participation in federated learning. Thus, while ensuring the feasibility of edge devices, it significantly improves the convergence speed, training stability, and generalization performance of the global model.
[0155] In summary, this invention effectively suppresses training oscillations and accelerates convergence through a server-side intelligent adaptive aggregation strategy. Its core lies in the collaborative optimization mechanism between the client and the server, which systematically solves the spatiotemporal inconsistency problem caused by data heterogeneity and sparse participation, thereby improving the robustness and performance of the federated learning system as a whole.
[0156] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.
Claims
1. A robust federated learning method based on spatiotemporal collaborative optimization, comprising at least one server and multiple clients, characterized in that, include: Step 1: The server initializes or obtains the model dataset from the previous communication round, including model parameters, global momentum, and global sharpness beliefs; in each communication round, a certain number of client subsets are selected from all the clients based on preset filtering conditions, and the model parameters are distributed; the clients perform local training of the global model based on a preset adaptive training mechanism. Step 2: After completing local training, the client uploads the local data to the server; the local data includes the update amount and sharpness value of the global model; the server aggregates all local data and calculates the statistical characteristics of the update amount and the average sharpness value. Step 3: Update the global sharpness belief based on the average sharpness value to obtain a new global sharpness belief; calculate the adaptive momentum coefficient based on the statistical features and the new global sharpness belief; fuse the average update amount in the statistical features and the global momentum based on the adaptive momentum coefficient to obtain a new global momentum; update the model parameters based on the new global momentum. Step 4: Iterate through Step 1, Step 2, and Step 3 until the preset termination condition is met to obtain the final global model.
2. The method according to claim 1, characterized in that, The screening criteria include basic availability criteria and random sampling; The basic availability conditions refer to the availability conditions set according to the actual application scenario; The random sampling refers to randomly or selectively choosing a certain number of clients from those that meet the basic availability conditions to form a subset of clients participating in the training.
3. The method according to claim 1, characterized in that, The local training process consists of K iteration steps; Before local training, fast weights and slow weights are set, and the fast weights and slow weights are equal to the model parameters; the fast weights will continuously change during local training, while the slow weights remain unchanged during local training.
4. The method according to claim 3, characterized in that, The local training process of the adaptive training mechanism is as follows: The first step is to randomly sample a batch of data from the local dataset stored in the client. The second step is to calculate the Euclidean distance between the fast weights and the slow weights. The third step is to determine the training mode of the global model based on the Euclidean distance and the preset perturbation radius threshold. If the Euclidean distance is greater than or equal to the perturbation radius threshold, the fast weights are updated using a computationally inefficient optimizer. If the Euclidean distance is less than the perturbation radius threshold, the fast weights are updated using a sharpness-aware minimization optimizer. The second and third steps are executed only in the first iteration step, and subsequent iteration steps update the fast weights based on the optimizer determined in the third step; Step 4, complete. After one iteration, the client generates model parameters from the obtained fast and slow weights using linear interpolation, with the following interpolation formula: in, For the t-th communication round The model parameters for each client, where t is greater than 1. These are the interpolation coefficients. For slow weights, For the t-th communication round Fast weighting for each client.
5. The method according to claim 4, characterized in that, The formula for calculating the update amount is as follows: in, For the first The update amount of each client in the t-th communication round; The formula for calculating the sharpness value is as follows: in, For the first The sharpness value of a client in the t-th communication round. Let be the perturbation vector. For loss function, for , Let L2 be the norm of the perturbation vector.
6. The method according to claim 5, characterized in that, The statistical characteristics of the update volume include the average update volume and the update volume variance; The formula for calculating the average sharpness value is as follows: in, Let be the average sharpness value in the t-th communication round. This represents the number of clients in the client subset.
7. The method according to claim 6, characterized in that, The server uses an exponential moving average mechanism to smoothly update the global sharpness belief, with the following update formula: in, For the new global sharpness belief in the t-th communication round, For the sake of overall sharpness belief, is the smoothing coefficient for the global sharpness belief.
8. The method according to claim 7, characterized in that, The calculation process for the adaptive momentum coefficient is as follows: Dynamic sensitivity is calculated based on the new global sharpness belief. The formula is as follows: in, The preset base sensitivity; Calculate the time window by combining the statistical characteristics of dynamic sensitivity and update volume. The formula is as follows: in, This is the maximum upper limit of the preset time window. It is a preset minimum constant. To update the variance, This is the average update amount. The square of the L2 norm of the average update amount; The adaptive momentum coefficient is calculated using the time window, as shown in the following formula: in, For the first The adaptive momentum coefficient for each communication round.
9. The method according to claim 8, characterized in that, The formula for obtaining the new global momentum is as follows: in, For new global momentum, For global momentum; The update formula for the model parameters is as follows: in, For the updated model parameters, This represents the global learning rate.
10. The method according to claim 1, characterized in that, The termination conditions include a preset number of communication rounds and the model convergence status.