Gradient correction-based federated learning adaptive optimization method and gradient corrector

Abstract

The application provides a gradient correction-based federated learning adaptive optimization method and a gradient corrector, and relates to the field of distributed machine learning.The application obtains gradient vectors uploaded by each client, logically decouples to obtain shallow feature gradient and deep semantic gradient, performs conflict detection to generate a hierarchical conflict mask matrix, performs elastic projection correction, simultaneously introduces inertia anchor point constraint to correct the direction, clusters the corrected gradient into several gradient clusters, calculates a signal-to-noise ratio, allocates a nonlinear aggregation weight according to the signal-to-noise ratio, generates a global aggregation gradient, uses the gradient conflict ratio of the current round and the global loss change rate to form an error signal, dynamically calculates a gain weight and a global aggregation step through a PID control law, updates the global aggregation gradient and sends it to each client to complete the iteration of the current round of federated learning.The application can effectively alleviate the gradient conflict under non-independent and identically distributed data, improve the convergence stability and model precision, and realize Pareto stable convergence.

Description

Technical Field

[0001] This invention relates to the field of distributed machine learning technology, and more specifically, to a federated learning adaptive optimization method and gradient corrector based on gradient correction. Background Technology

[0002] With the rapid development of edge computing and IoT technologies, the data generated by massive numbers of terminal devices exhibits highly distributed, heterogeneous, and privacy-sensitive characteristics. Traditional centralized machine learning paradigms face severe challenges in terms of communication overhead, data privacy, and compliance. Against this backdrop, Federated Learning (FL), as a distributed collaborative training framework where "the model moves while the data remains stationary," has become a research hotspot in the field of privacy-preserving machine learning because it can achieve multi-party joint modeling without sharing the original data. In a typical federated learning architecture, multiple clients independently train models based on their local private data and periodically upload gradients or model parameters to a central server. The server then aggregates these updates, generates a new global model, and distributes it, thus completing collaborative optimization. This mechanism effectively alleviates the data silo problem, but it also introduces gradient conflict caused by the non-independent and identically distributed (Non-IID) distribution of client data—that is, there is a significant or even negatively correlated cosine relationship between the gradient directions uploaded by different clients, causing the global update direction to deviate from the optimal path, leading to model oscillations, decreased convergence speed, and even eventual accuracy degradation.

[0003] To address the aforementioned challenges, existing federated optimization methods primarily explore three technical routes. The first category, such as FedProx and SCAFFOLD, introduces global regularization terms or control variables into the local objective function to constrain client update magnitudes and suppress divergence. While this improves stability to some extent, it doesn't directly address gradient direction conflicts and lacks geometric consistency guarantees. The second category, such as FedNova and FedDyn, focuses on improving aggregation strategies by adjusting weights or introducing momentum mechanisms to compensate for differences in local update steps or historical biases. However, their aggregation logic largely relies on scalar metrics (such as sample size or loss value) and cannot perceive the structured directional relationships between gradient vectors. The third category draws on the projection correction idea from multi-task learning, such as FedMGDA+, GradNorm, and PCGrad, attempting to alleviate conflicts by detecting negative correlations between gradients and performing orthogonalization or culling operations, achieving some success in specific scenarios. However, these methods generally suffer from three major drawbacks: First, they lack structure awareness, treating the entire neural network gradient as a flat vector for uniform processing. This ignores the difference in sensitivity to data heterogeneity between shallow (general feature extraction) and deep (task semantic decision-making) layers, making it easy to inadvertently damage shallow common features with generalization value when correcting deep conflicts, resulting in "overcorrection." Second, they neglect temporal consistency, judging and correcting conflicts only based on the gradient snapshot of the current round, without considering the inertial information of historical optimization trajectories. This may lead to short-term conflict elimination but long-term deviation from the main convergence path, inducing "catastrophic forgetting." Third, the aggregation weight adjustment mechanism is static or linear, lacking the ability to control the closed-loop feedback of the optimization dynamic process, making it difficult to achieve rapid response and steady-state balance on non-convex, high-dimensional, and violently oscillating loss surfaces.

[0004] In view of the above, this application is hereby submitted. Summary of the Invention

[0005] This invention aims to provide a gradient correction-based adaptive optimization method and gradient corrector for federated learning, in order to address the technical shortcomings of existing federated learning methods in non-independent and identically distributed (Non-IID) data scenarios, such as severe gradient conflicts, global optimization path oscillations, unstable long-term convergence trajectories, and lack of structure awareness and dynamic adjustment capabilities.

[0006] To solve the above-mentioned technical problems, the present invention is achieved through the following technical solution:

[0007] A gradient-correction-based federated learning adaptive optimization method, applied to the server side, includes:

[0008] S1, obtain the gradient vectors uploaded by each client, and logically decouple them to obtain the shallow feature gradient and the deep semantic gradient;

[0009] S2, perform conflict detection on the shallow feature gradient and the deep semantic gradient respectively, and generate a hierarchical conflict mask matrix;

[0010] S3. Based on the hierarchical conflict mask matrix, perform elastic projection correction on the gradient components with conflict, and introduce the exponential moving average of the global aggregated gradient as an inertial anchor point to constrain the correction direction from deviating from the historical optimization trajectory.

[0011] S4. The corrected gradient is clustered into several gradient clusters, and the signal-to-noise ratio is calculated for each cluster. Based on this, nonlinear aggregation weights are assigned to generate a global aggregated gradient.

[0012] S5. Based on the global aggregated gradient, the gradient conflict ratio of the current round and the global loss change rate are used to form an error signal. The gain weight and global aggregated step size are dynamically calculated through the PID control law, the global aggregated gradient is updated, and it is sent to each client to complete the iteration of this round of federated learning.

[0013] The shallow feature gradient is extracted through the front-end convolutional layer of the neural network or the bottom coding layer of the Transformer.

[0014] The deep semantic gradient is extracted through the fully connected layer of a neural network or the top decision layer of a Transformer.

[0015] Preferably, when performing conflict detection, the relaxed cosine similarity is used to measure the correlation of the shallow feature gradients of any two clients. By setting a tolerance threshold, excessive constraints that lead to a decrease in generalization ability are avoided. The expression is:

[0016] ;

[0017] in, This represents the shallow gradient similarity between clients i and j. , These are the shallow feature gradients of clients i and j, respectively; The relaxation coefficient is used to relax the similarity judgment criteria. Represents the magnitude of the vector;

[0018] when Greater than the set shallow tolerance threshold When the shallow gradient is determined to be conflict-free;

[0019] Furthermore, class prototype orthogonality detection is performed on the deep semantic gradients between clients, and the correlation of deep gradient directions is determined by cosine similarity, expressed as:

[0020] ;

[0021] in, This represents the deep gradient similarity between clients i and j; , These are the deep feature gradients of clients i and j, respectively;

[0022] when Less than the set deep strict threshold At that time, it is determined that there is a semantic conflict in the deep gradient;

[0023] Then, based on the hierarchical collision detection results, a hierarchical collision mask matrix is ​​constructed; assuming the total number of clients is K, the hierarchical collision mask matrix is... Then matrix elements for:

[0024] ;

[0025] when This indicates that there are no conflicts between the shallow and deep gradients of client i and j; when This indicates a semantic conflict in deep gradients; when This indicates that there is a conflict in the shallow gradient.

[0026] Preferably, based on the hierarchical conflict mask matrix, the expression for performing elastic projection correction on conflicting gradient components is as follows:

[0027] ;

[0028] ;

[0029] in, The gradient after elastic projection correction for client i; This indicates that there is a conflicting gradient at client i; , The dynamic elastic coefficient; For conflict resolution items; for exist Projection in the orthogonal direction; This is for inertial anchor point compensation. for Projection onto the inertial anchor point; , The current round The previous round's inertial anchor point, i.e., the exponential moving average vector of the global aggregated gradient, is used to prevent gradient updates from deviating from the optimal trajectory. The momentum coefficient is used to smooth historical trajectories. This is the global aggregate gradient from the previous round.

[0030] Preferably, S4 specifically comprises:

[0031] First, using the cosine similarity between gradient vectors as the distance metric, the corrected gradients are divided into C disjoint clusters;

[0032] Then, for each cluster, the signal-to-noise ratio (SNR) is calculated; the formula for calculating the SNR is:

[0033] ;

[0034] in, is the signal-to-noise ratio of the c-th cluster, used to distinguish between highly consistent clusters and outlier noise clusters; Let c be the modulus of the center of the c-th cluster; Let be the intra-cluster variance of the c-th cluster; It is a very small positive number, used to prevent the denominator from being 0;

[0035] Based on the calculated signal-to-noise ratio (SNR), a nonlinear weighted formula is constructed, using the SNR of each cluster as the weight, to automatically suppress the influence of low SNR clusters, amplify the signals of high SNR clusters, and generate a pure global aggregated gradient. The expression is as follows:

[0036] ;

[0037] in, For global aggregated gradients; This is the signal-to-noise ratio scaling factor; For the first The signal-to-noise ratio of each cluster; C is the total number of clusters.

[0038] Preferably, S5 specifically includes:

[0039] The gradient conflict ratio is calculated based on the hierarchical conflict mask matrix and is used to represent the intensity of the current client gradient conflict. The expression is:

[0040] ;

[0041] in, The conflict ratio for the current round t; Total number of clients; For indicator functions; These are the elements of the hierarchical collision mask matrix;

[0042] The formula for calculating the error signal is:

[0043] ;

[0044] ;

[0045] ;

[0046] in, This is the error signal for the current round t; , These are weighting coefficients used to balance the contributions of conflict bias and loss rate of change; This is due to the conflict ratio deviation. The ideal conflict ratio for the target; The global loss rate of change; , These represent the global losses for the current round t and the previous round, respectively. The interval between rounds;

[0047] Then, using the error signal as input, the formula for dynamically calculating the gain weight through the PID control law is as follows:

[0048] ;

[0049] in, The gain weight for the current round t; , , These are the proportional coefficient, integral coefficient, and differential coefficient, respectively. For history Error signal for each round; , They are respectively for , Integral;

[0050] Gain weight The mapping to the global aggregation step size required for actual optimization is given by the formula:

[0051] ;

[0052] in, This represents the global aggregation step size for the current round t. This is the initial base step size; This is the step size decay coefficient;

[0053] Based on the global aggregation step size With the gain weight Update the global aggregated gradient for each cluster to obtain the global aggregated gradient that integrates the signal-to-noise ratio and PID control. .

[0054] Preferably, it further includes:

[0055] S6 uses a sliding window to monitor the global update direction of the most recent round. If a sudden change in direction is detected that exceeds a preset safety threshold, the current global aggregate gradient is convexly fused with the previous round to generate a smoothed global aggregate gradient, which is then updated and sent to each client.

[0056] Preferably, S6 specifically includes:

[0057] A sliding window is used to calculate the global gradient of the current round and the update offset of the previous round in real time to determine whether there is a sudden change in direction; the formula for the update offset is:

[0058] ;

[0059] in, This is the update offset for the current round t; , These represent the global aggregate gradients before smoothing in the current round and the previous round, respectively.

[0060] A safety threshold is set. If the updated offset exceeds the safety threshold, it is determined that the global gradient has abruptly changed direction, and a smoothing correction is performed. The formula for the smoothing correction is:

[0061] ;

[0062] in, The global aggregated gradient is smoothed and corrected. This is the adjustment coefficient;

[0063] Otherwise, no correction is performed, and the current round's global aggregated gradient is used directly for updating.

[0064] The present invention also provides a gradient corrector, applied on the server side, comprising:

[0065] The feature extraction unit is used to obtain the gradient vectors uploaded by each client and logically decouple them to obtain the shallow feature gradient and the deep semantic gradient.

[0066] A multi-granularity conflict detection unit is used to perform conflict detection on the shallow feature gradient and the deep semantic gradient respectively, and generate a hierarchical conflict mask matrix.

[0067] The elastic correction unit is used to perform elastic projection correction on the gradient components with conflict based on the hierarchical conflict mask matrix, and at the same time introduces the exponential moving average of the global aggregated gradient as an inertial anchor point to constrain the correction direction from deviating from the historical optimization trajectory.

[0068] The signal-to-noise ratio hierarchical aggregation unit is used to cluster the corrected gradient into several gradient clusters, calculate the signal-to-noise ratio for each cluster, assign nonlinear aggregation weights accordingly, and generate a global aggregated gradient.

[0069] The PID gain scheduling unit is used to construct an error signal based on the global aggregated gradient, using the gradient conflict ratio of the current round and the global loss change rate together. It dynamically calculates the gain weight and global aggregated step size through the PID control law, updates the global aggregated gradient, and sends it to each client to complete the iteration of this round of federated learning.

[0070] Preferably, it further includes: a global smoothing unit, used to monitor the global update direction of the most recent round using a sliding window; if a sudden change in direction is detected that exceeds a preset safety threshold, the current global aggregated gradient is convexly fused with the previous round to generate a smoothed global aggregated gradient, which is then updated and sent to each client.

[0071] This invention also provides a gradient-correction-based federated learning adaptive optimization device, comprising several clients and a server; the clients are used to train based on local raw datasets and upload gradient vectors to the server; the server includes a memory and a processor, the memory storing a computer program that can be executed by the processor to implement the gradient-correction-based federated learning adaptive optimization method as described above.

[0072] The present invention also provides a computer-readable storage medium storing computer-readable instructions, which, when executed by a processor of the device in which the computer-readable storage medium resides, implement a gradient-correction-based federated learning adaptive optimization method as described above.

[0073] In summary, compared with the prior art, the present invention has the following beneficial effects:

[0074] This invention constructs a three-dimensional federated optimization system from microscopic structure perception to macroscopic closed-loop control, which significantly improves the model convergence speed, accuracy, and robustness in highly heterogeneous data environments, without requiring modifications to the client architecture. It possesses good engineering deployment feasibility and technical compatibility, and achieves refined identification and differentiated processing of gradient conflicts in federated learning, as detailed below:

[0075] By structurally decoupling, conflict patterns between general shallow features and task-related deep semantic features are distinguished, avoiding over-correction of shallow gradients. Then, the current inertial anchor constraints and historical gradient information are integrated to maintain the consistency of long-term optimization direction while eliminating negatively correlated components. Next, a signal-to-noise ratio quality assessment mechanism is introduced to effectively filter low-quality or abnormal gradients. Industrial PID control theory is introduced into the non-convex optimization process, and dynamic steady-state control is achieved through proportional response, integral correction, and derivative prediction. Finally, the Pareto stationary convergence condition is satisfied by forcing the optimization path to be smooth and continuous.

[0076] This invention achieves Pareto-stable convergence in a highly heterogeneous environment by realizing structural decoupling of heterogeneous gradients, elastic manifold correction guided by historical inertia, quality-aware aggregation driven by signal-to-noise ratio, and dynamic feedback adjustment based on industrial control theory without increasing the computational burden and communication overhead of the client. Attached Figure Description

[0077] To more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings used in the embodiments will be briefly introduced below. It should be understood that the following drawings only show some embodiments of the present invention and should not be regarded as a limitation of the scope. For those skilled in the art, other related drawings can be obtained from these drawings without creative effort.

[0078] Figure 1 This is a flowchart illustrating a gradient correction-based federated learning adaptive optimization method provided in Example 1.

[0079] Figure 2 This is a schematic diagram of a federated learning adaptive optimization method based on gradient correction, provided in Example 1.

[0080] Figure 3 This is a schematic diagram of a gradient corrector provided in Embodiment 2.

[0081] The present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments. Detailed Implementation

[0082] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of the present invention, not all of them. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without inventive effort are within the scope of protection of the present invention. Therefore, the following detailed description of the embodiments of the present invention provided in the accompanying drawings is not intended to limit the scope of the claimed invention, but merely represents selected embodiments of the invention. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without inventive effort are within the scope of protection of the present invention.

[0083] Example 1

[0084] Embodiment 1 of the present invention provides a gradient-corrected federated learning adaptive optimization method, which can be implemented by a gradient-corrected federated learning adaptive optimization device (hereinafter referred to as optimization device), specifically, executed by one or more processors within the optimization device.

[0085] In this embodiment, the optimization device may be an electronic device equipped with a processor, the processor having a computer program for the gradient correction-based federated learning adaptive optimization method and the computer program being executable, such as a computer, smartphone, smart tablet, workstation, etc., without limitation.

[0086] In this embodiment, Federated Learning (FL) is a distributed machine learning framework that coordinates multiple clients to collaboratively train a shared model without centralizing user data. It effectively protects data privacy while utilizing distributed computing power for large-scale training.

[0087] Gradient conflict refers to a situation in federated learning where the gradient directions uploaded by different clients have a negative cosine relationship, causing the global gradient direction to deviate from the optimal update path. This phenomenon can lead to model oscillations, slow convergence, and even accuracy degradation.

[0088] Adaptive aggregation refers to a mechanism in which the server dynamically adjusts the aggregation weights based on the gradient similarity, conflict level, and loss change rate of the client, thereby achieving a balance between directional consistency and global convergence.

[0089] Projection correction is a conflict mitigation method based on geometric constraints. It achieves non-negative cosine direction alignment by removing or orthogonalizing negative conflict components from the gradient.

[0090] like Figures 1-2 As shown, a federated learning adaptive optimization method based on gradient correction includes steps S1 to S6.

[0091] S1: Obtain the gradient vectors uploaded by each client and logically decouple them to obtain the shallow feature gradient and the deep semantic gradient.

[0092] The server receives the local training gradient vectors uploaded by each client, and then divides the gradients according to the feature extraction level based on the neural network topology.

[0093] Specifically, the shallow feature gradient is extracted through the front-end convolutional layer of the neural network or the bottom coding layer of the Transformer. It is mainly responsible for extracting general features such as texture and edge, and has strong task independence.

[0094] The deep semantic gradient is extracted through the fully connected layer of a neural network or the top decision layer of a Transformer. It is directly related to the semantics of task classification and is strongly correlated with the model output.

[0095] This step employs a conflict perception mechanism based on a structured, hierarchical perspective, aiming to overcome the information confusion caused by the traditional method's "one-size-fits-all" treatment of gradient vectors.

[0096] S2, perform conflict detection on the shallow feature gradient and the deep semantic gradient respectively, and generate a hierarchical conflict mask matrix.

[0097] This step uses a differentiated measurement method to detect gradient conflicts between clients based on the different characteristics of shallow and deep gradients, and generates a hierarchical conflict mask matrix.

[0098] When performing conflict detection, the correlation of the shallow feature gradients of any two clients is measured using relaxed cosine similarity (or canonical correlation analysis). By setting a tolerance threshold, excessive constraints that lead to a decrease in generalization ability are avoided. The expression is:

[0099] ;

[0100] in, This represents the shallow gradient similarity between clients i and j. , These are the shallow feature gradients of clients i and j, respectively; The relaxation coefficient is used to relax the similarity judgment criteria. It represents the magnitude of the vector.

[0101] when Greater than the set shallow tolerance threshold When this is the case, it is determined that there is no conflict in the shallow gradient.

[0102] Furthermore, class prototype orthogonality detection is performed on the deep semantic gradients between clients, and the correlation of deep gradient directions is determined by cosine similarity, expressed as:

[0103] ;

[0104] in, This represents the deep gradient similarity between clients i and j; , These are the deep feature gradients of clients i and j, respectively;

[0105] when Less than the set deep strict threshold At that time, it is determined that there is a semantic conflict in the deep gradient;

[0106] Then, based on the hierarchical collision detection results, a hierarchical collision mask matrix is ​​constructed; assuming the total number of clients is K, the hierarchical collision mask matrix is... Then matrix elements for:

[0107] ;

[0108] when This indicates that there are no conflicts between the shallow and deep gradients of client i and j; when This indicates a semantic conflict in deep gradients; when This indicates that there is a conflict in the shallow gradient.

[0109] S3. Based on the hierarchical conflict mask matrix, perform elastic projection correction on the gradient components with conflicts, and introduce the exponential moving average of the global aggregated gradient as an inertial anchor point to constrain the correction direction from deviating from the historical optimization trajectory.

[0110] This step introduces a global gradient exponential moving average (EMA) as an inertial anchor point for the conflict gradients marked by the hierarchical conflict mask matrix, and performs elastic correction to avoid losing the direction of gradient updates.

[0111] After the server updates the global gradient EMA vector (inertial anchor point) in real time, it corrects conflicting gradients.

[0112] Based on the hierarchical conflict mask matrix, the expression for elastic projection correction of conflicting gradient components is as follows:

[0113] ;

[0114] ;

[0115] in, The gradient after elastic projection correction for client i; This indicates that there is a conflicting gradient at client i; , The dynamic elastic coefficient; For conflict resolution, eliminate the immediate conflict component with conflicting client j; for exist Projection in the orthogonal direction; As an inertial anchor point compensation term, it moves closer to the historical inertial direction, constraining the updated trajectory; for Projection onto the inertial anchor point; , The current round The previous round's inertial anchor point, i.e., the exponential moving average vector of the global aggregated gradient, is used to prevent gradient updates from deviating from the optimal trajectory. The momentum coefficient is used to smooth historical trajectories. This is the global aggregate gradient from the previous round.

[0116] S4 clusters the corrected gradients into several gradient clusters, calculates the signal-to-noise ratio for each cluster, assigns nonlinear aggregation weights accordingly, and generates a global aggregated gradient.

[0117] This step involves quantizing the cluster quality of the corrected gradient clusters using signal-to-noise ratio (SNR), performing nonlinear weighted aggregation, and generating a pure global gradient.

[0118] Specifically, the corrected gradients are first divided into C disjoint clusters using the cosine similarity between gradient vectors as a distance metric.

[0119] Then, for each cluster, the signal-to-noise ratio (SNR) is calculated; the formula for calculating the SNR is:

[0120] ;

[0121] in, is the signal-to-noise ratio of the c-th cluster, used to distinguish between highly consistent clusters and outlier noise clusters; Let c be the modulus of the center of the c-th cluster; Let be the intra-cluster variance of the c-th cluster; It is a very small positive number, used to prevent the denominator from being 0;

[0122] Based on the calculated signal-to-noise ratio (SNR), a nonlinear weighted formula is constructed, using the SNR of each cluster as the weight, to automatically suppress the influence of low SNR clusters, amplify the signals of high SNR clusters, and generate a pure global aggregated gradient. The expression is as follows:

[0123] ;

[0124] in, For global aggregated gradients; This is the signal-to-noise ratio scaling factor; For the first The signal-to-noise ratio of each cluster; C is the total number of clusters.

[0125] S5. Based on the global aggregated gradient, the gradient conflict ratio of the current round and the global loss change rate are used to form an error signal. The gain weight and global aggregated step size are dynamically calculated through the PID control law, the global aggregated gradient is updated, and it is sent to each client to complete the iteration of this round of federated learning.

[0126] This step introduces industrial PID control theory to construct an error signal, dynamically adjust the aggregation step size and weights, and achieve non-convex optimization steady-state control.

[0127] Specifically, the gradient conflict ratio is calculated based on the hierarchical conflict mask matrix and is used to represent the intensity of the current client gradient conflict. The expression is as follows:

[0128] ;

[0129] in, The conflict ratio for the current round t; Total number of clients; This is an indicator function; its value is 1 if the condition is met, and 0 otherwise. These are the elements of the hierarchical collision mask matrix;

[0130] The error signal is composed of a weighted average of the collision ratio deviation and the global loss change rate, and the calculation formula is as follows:

[0131] ;

[0132] ;

[0133] ;

[0134] in, This is the error signal for the current round t; , These are weighting coefficients used to balance the contributions of conflict bias and loss rate of change; This is due to the conflict ratio deviation. The ideal conflict ratio for the target; The global loss rate of change; , These represent the global losses for the current round t and the previous round, respectively. The interval between rounds;

[0135] Then, using the error signal as input, the formula for dynamically calculating the gain weight through the PID control law is as follows:

[0136] ;

[0137] in, The gain weight for the current round t; , , These are the proportional coefficient, integral coefficient, and differential coefficient, respectively. For history Error signal for each round; , They are respectively for , Find the integral.

[0138] This is a proportional term used to respond to the current level of conflict intensity. When a severe conflict is detected, the aggregation step size is rapidly reduced to prevent oscillation. The integral term is used to record historical aggregation bias. When the model gets stuck in a local minimum or fails to converge for a long time, the integral term accumulates and increases, forcibly changing the weight distribution to escape the stagnation zone. As the differential term, it is responsible for predicting the trend of Loss changes. If it is detected that Loss is about to rebound (a precursor to oscillation), damping suppression is introduced in advance.

[0139] Gain weight The mapping to the global aggregation step size required for actual optimization is given by the formula:

[0140] ;

[0141] in, This represents the global aggregation step size for the current round t. This is the initial base step size; This is the step size decay coefficient;

[0142] Based on the global aggregation step size With the gain weight Update the global aggregated gradient for each cluster to obtain the global aggregated gradient that integrates the signal-to-noise ratio and PID control. .

[0143] S6 uses a sliding window to monitor the global update direction of the most recent round. If a sudden change in direction is detected that exceeds a preset safety threshold, the current global aggregate gradient is convexly fused with the previous round to generate a smoothed global aggregate gradient, which is then updated and sent to each client.

[0144] Although the PID module has output the optimized step size, direct updates may still cause drastic changes in model parameters due to the extreme complexity of the non-convex optimization surface. This step monitors gradient abrupt changes through a sliding window and performs trajectory smoothing correction to eliminate the "sawtooth" oscillations that may occur during multiple rounds of aggregation, ensuring that the parameter trajectory satisfies the Pareto stability constraint.

[0145] Specifically, a sliding window is used to calculate the global gradient of the current round and the update offset of the previous round in real time to determine whether there is a sudden change in direction; the formula for the update offset is:

[0146] ;

[0147] in, This is the update offset for the current round t; , These represent the global aggregate gradients before smoothing in the current round and the previous round, respectively. Let be the magnitude of the vector.

[0148] A safety threshold is set. If the updated offset exceeds the safety threshold, it is determined that the global gradient has abruptly changed direction, and a smoothing correction is performed. The formula for the smoothing correction is:

[0149] ;

[0150] in, The global aggregated gradient is smoothed and corrected. This is the adjustment coefficient.

[0151] Otherwise, no correction is performed, and the current round's global aggregated gradient is used directly for updating.

[0152] This operation preserves the current update magnitude (update strength) while forcibly fine-tuning the update direction towards the stable direction of the previous round. This "soft landing" mechanism effectively suppresses parameter space jitter caused by heterogeneous data impacts, ensuring the continuity and convergence stability of the federated learning process during long-term training.

[0153] This invention overcomes the limitation of traditional federated optimization methods that only process gradients in a single vector space by constructing a five-dimensional closed-loop collaborative mechanism consisting of "multi-granularity hierarchical conflict detection (M-GCD), historical inertia-guided manifold correction (H-EMR), signal-to-noise ratio hierarchical aggregation (SNR-LWA), PID gain scheduling feedback (PID-DAW), and global trajectory smoothing (GCS)." In an edge computing environment with highly skewed heterogeneous data (Non-IID), this scheme achieves end-to-end optimization from microscopic structure perception to macroscopic closed-loop control, significantly improving the Pareto convergence efficiency, noise robustness, and spatiotemporal consistency of the global model.

[0154] In practical applications, such as cross-hospital federated learning systems for intelligent healthcare, each hospital (as an edge client) possesses highly heterogeneous patient image data (Non-IID datasets), and locally deployed diagnostic models are used to train on local datasets. In each training round, each hospital client only uploads model parameters (i.e., gradient vectors) trained on its local dataset to the central server. The server extracts shallow gradients of common lesion features and deep gradients for specific disease classifications. Through multi-granularity hierarchical conflict detection, it is found that different hospitals have high similarity in shallow gradients (due to common lesion textures), but significant negative correlations exist in deep gradients (due to differences in disease category distribution). Using historical trajectories in the global gradient EMA memory, deep conflict gradients are elastically corrected, eliminating adversarial components while preserving long-term optimization directions. A hierarchical weighted aggregation step based on signal-to-noise ratio can identify hospitals with high gradient noise (low SNR) due to outdated equipment and automatically reduce their weights. In the early stages of training, the step size may be significantly reduced due to severe conflicts; later, when the loss stabilizes, residual errors are eliminated through integration terms. This problem can be solved through an adaptive feedback aggregation step with PID gain scheduling. To prevent drastic fluctuations in model parameters due to sudden data drift, a global consistency scheduling step is used. The entire process requires no modification to hospital-side equipment; upgrading the server-side algorithm alone significantly improves model convergence speed and diagnostic accuracy.

[0155] In summary, compared with the prior art, the present invention has the following beneficial effects:

[0156] This invention overcomes the blindness of traditional cosine detection by decoupling gradients into shallow feature manifolds and deep semantic manifolds. It can not only identify gradient direction conflicts, but also accurately locate whether the source of the conflict comes from perturbations of general features or semantic adversarial reactions in the decision layer. This provides a fine structured index for subsequent differential correction and effectively prevents the decline in model generalization ability caused by "false positives" on shallow features.

[0157] Building upon this foundation, a historical inertia anchor point and elastic projection mechanism are introduced to address the "short-sightedness" problem inherent in traditional projection methods. While eliminating conflict components in the current round, the update direction is forced to adhere to the historical long-term optimization trajectory, effectively suppressing the "forgetting" phenomenon of the model on non-convex optimization surfaces and achieving a dual balance between spatial geometric correction and temporal memory preservation. Subsequently, the signal-to-noise ratio (SNR) metric is used to replace simple similarity clustering, enabling the system to automatically identify and filter "toxic" gradients or low-quality updates. Nonlinear weighting suppresses high-variance noise clusters, ensuring extremely high signal fidelity in the aggregated gradient flow. Furthermore, industrial PID control theory is innovatively introduced into federated aggregation. Through the proportional response, integral correction, and derivative prediction of the PID controller, the system can rapidly increase convergence speed in the early stages of training and automatically increase damping in the later stages to eliminate steady-state errors, achieving rapid response and smooth transition without overshoot during the convergence process. Finally, by using sliding window monitoring and trajectory smoothing, the severe oscillations in the parameter space caused by excessive step size or heterogeneous impacts are eliminated, theoretically ensuring the continuity and smoothness of the global optimization path.

[0158] This invention's multi-layer adaptive system fundamentally solves the problems of "gradient drift" and "optimization oscillation" caused by data heterogeneity in federated learning. Theoretically, this scheme guarantees approximately monotonically convergent under non-convex objective functions through manifold correction and closed-loop control. In engineering, each module employs a lightweight tensor operation design, introducing no additional communication overhead and possessing extremely high deployment feasibility. The system exhibits excellent versatility and scalability, seamlessly embedding into existing federated learning frameworks (such as FedAvg and FedProx) and perfectly compatible with gradient compression, differential privacy, and other technical modules. Therefore, by constructing an adaptive optimization system based on control theory and manifold geometry, this invention achieves multidimensional Pareto optimality in communication efficiency, convergence accuracy, and system stability, demonstrating significant technological innovation and broad industrial application value.

[0159] Example 2

[0160] like Figure 3 As shown, the second embodiment of the present invention also provides a gradient corrector, comprising:

[0161] The feature extraction unit is used to obtain the gradient vectors uploaded by each client and logically decouple them to obtain the shallow feature gradient and the deep semantic gradient.

[0162] A multi-granularity conflict detection unit is used to perform conflict detection on the shallow feature gradient and the deep semantic gradient respectively, and generate a hierarchical conflict mask matrix.

[0163] The elastic correction unit is used to perform elastic projection correction on the gradient components with conflict based on the hierarchical conflict mask matrix, and at the same time introduces the exponential moving average of the global aggregated gradient as an inertial anchor point to constrain the correction direction from deviating from the historical optimization trajectory.

[0164] The signal-to-noise ratio hierarchical aggregation unit is used to cluster the corrected gradient into several gradient clusters, calculate the signal-to-noise ratio for each cluster, assign nonlinear aggregation weights accordingly, and generate a global aggregated gradient.

[0165] The PID gain scheduling unit is used to construct an error signal based on the global aggregated gradient, using the gradient conflict ratio of the current round and the global loss change rate together. It dynamically calculates the gain weight and global aggregated step size through the PID control law, updates the global aggregated gradient, and sends it to each client to complete the iteration of this round of federated learning.

[0166] The global smoothing unit is used to monitor the global update direction of the most recent round using a sliding window. If a sudden change in direction is detected that exceeds a preset safety threshold, the current global aggregated gradient is convexly fused with the previous round to generate a smoothed global aggregated gradient, which is then updated and sent to each client.

[0167] Example 3

[0168] The third embodiment of the present invention also provides a federated learning adaptive optimization device based on gradient correction, which includes several clients and a server; the clients are used to train based on the local original dataset and upload gradient vectors to the server; the server includes a memory and a processor, the memory stores a computer program, and the computer program can be executed by the processor to implement the federated learning adaptive optimization method based on gradient correction as described above.

[0169] Example 4

[0170] The fourth embodiment of the present invention also provides a computer-readable storage medium storing computer-readable instructions, which, when executed by a processor of the device where the computer-readable storage medium is located, implement the gradient-correction-based federated learning adaptive optimization method as described above.

[0171] The above description is merely a preferred embodiment of the present invention and is not intended to limit the invention. Various modifications and variations can be made to the present invention by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A federated learning adaptive optimization method based on gradient correction, applied to a server side, characterized in that, include: S1, obtain the gradient vectors uploaded by each client, and logically decouple them to obtain the shallow feature gradient and the deep semantic gradient; S2, perform conflict detection on the shallow feature gradient and the deep semantic gradient respectively, and generate a hierarchical conflict mask matrix; S3. Based on the hierarchical conflict mask matrix, perform elastic projection correction on the gradient components with conflict, and introduce the exponential moving average of the global aggregated gradient as an inertial anchor point to constrain the correction direction from deviating from the historical optimization trajectory. Based on the hierarchical conflict mask matrix, the expression for elastic projection correction of conflicting gradient components is as follows: ； ； wherein, is the elastic projection corrected gradient for the client i; represents the gradient of the client i with conflict; , is the dynamic elasticity coefficient; is the conflict elimination term; is the is the projection in the orthogonal direction; is the projection on the inertia anchor point; is the inertia anchor point compensation term; is the is the projection on the inertia anchor point; , is the inertia anchor point of the current round , the inertia anchor point of the last round, that is, the exponentially moving average vector of the global aggregated gradient, used to prevent the gradient update from deviating from the optimal trajectory; is the momentum coefficient, used to smooth the historical trajectory; is the global aggregated gradient of the last round; S4. The corrected gradient is clustered into several gradient clusters, and the signal-to-noise ratio is calculated for each cluster. The nonlinear aggregation weights are assigned based on the signal-to-noise ratio of each cluster to generate the global aggregated gradient. S5, based on the global aggregated gradient, the error signal is constructed by the gradient conflict ratio of the current round and the global loss change rate. The gain weight and global aggregated step size are dynamically calculated using a PID control law, the global aggregated gradient is updated, and then distributed to each client to complete the iteration of this round of federated learning. Specifically: The gradient conflict ratio is calculated based on the hierarchical conflict mask matrix and is used to represent the intensity of the current client gradient conflict. The expression is: ； wherein, is the conflict ratio for the current round t; is the total number of clients; is the indicator function; is an element of the hierarchical conflict mask matrix. The formula for calculating the error signal is: ； ； ； wherein, is the error signal for the current round t; , is a weighting factor balancing the contribution of conflict bias and loss rate of change; is the conflict rate bias; is the target ideal conflict rate; is the global loss rate of change; , are the global losses for the current round t, the previous round, respectively; is the round interval; Then, using the error signal as input, the formula for dynamically calculating the gain weight through the PID control law is as follows: ； wherein, is the gain weight for the current round t; , , are a proportional coefficient, an integral coefficient, a derivative coefficient, respectively; is the error signal for the historical round t; , are the proportional, integral and derivative of , , respectively; map the gain weights to the global aggregation step size required for the actual optimization, which is given by ； wherein, is the global aggregation step size for the current round t; is the initial base step size; is the step size decay coefficient; based on the global aggregation step size with the gain weight updating the global aggregation gradient of each cluster to obtain a global aggregation gradient fused with a signal-to-noise ratio and PID control .

2. The federated learning adaptive optimization method based on gradient correction according to claim 1, characterized in that The shallow feature gradient is extracted through the front-end convolutional layer of the neural network or the bottom coding layer of the Transformer. The deep semantic gradient is extracted through the fully connected layer of a neural network or the top decision layer of a Transformer.

3. The federated learning adaptive optimization method based on gradient correction according to claim 1, characterized in that When performing conflict detection, the relaxed cosine similarity is used to measure the correlation of the shallow feature gradients of any two clients. By setting a tolerance threshold, excessive constraints that lead to a decrease in generalization ability are avoided. The expression is: ； wherein, represents the shallow gradient similarity of clients i, j; , are the shallow feature gradients of clients i, j, respectively; is a relaxation coefficient, used to relax the similarity criterion; represents the module length of a vector; When greater than a set shallow tolerance threshold a shallow gradient is determined to be conflict free; Furthermore, class prototype orthogonality detection is performed on the deep semantic gradients between clients, and the correlation of deep gradient directions is determined by cosine similarity, expressed as: ； wherein, represents the deep gradient similarity of the clients i, j; , are the deep feature gradients of the clients i, j, respectively; When less than a set deep strict threshold a semantic conflict is determined to exist in the deep gradient; Then based on the hierarchical conflict detection results, a hierarchical conflict mask matrix is constructed; assuming that the total number of clients is K, the hierarchical conflict mask matrix is The matrix element is: ； When represents that there is no conflict in the shallow gradient and the deep gradient of the client i, j; when , it represents that there is semantic conflict in the deep gradient; when , it represents that there is conflict in the shallow gradient.

4. The federated learning adaptive optimization method based on gradient correction according to claim 3, characterized in that S4 specifically refers to: First, using the cosine similarity between gradient vectors as the distance metric, the corrected gradients are divided into C disjoint clusters; Then, for each cluster, the signal-to-noise ratio (SNR) is calculated; the formula for calculating the SNR is: ； wherein, S C is the signal-to-noise ratio of the cth cluster, used to distinguish high-consistency clusters from outlier noise clusters; is the length of the cth cluster center; is the intra-cluster variance of the cth cluster; is a very small positive number, used to prevent the denominator from being zero; Based on the calculated signal-to-noise ratio (SNR), a nonlinear weighted formula is constructed, using the SNR of each cluster as the weight, to automatically suppress the influence of low SNR clusters, amplify the signals of high SNR clusters, and generate a pure global aggregated gradient. The expression is as follows: ； wherein, is a global aggregated gradient; is a signal-to-noise ratio scaling factor; is a signal-to-noise ratio of the th cluster; C is the total number of clusters.

5. The federated learning adaptive optimization method based on gradient correction according to claim 4, characterized in that... It also includes: S6 uses a sliding window to monitor the global update direction of the most recent round. If a sudden change in direction is detected that exceeds a preset safety threshold, the current global aggregate gradient is convexly fused with the previous round to generate a smoothed global aggregate gradient, which is then updated and sent to each client.

6. The federated learning adaptive optimization method based on gradient correction according to claim 5, characterized in that... S6 specifically refers to: A sliding window is used to calculate the global gradient of the current round and the update offset of the previous round in real time to determine whether there is a sudden change in direction; the formula for the update offset is: ； in, This is the update offset for the current round t; , These represent the global aggregate gradients before smoothing in the current round and the previous round, respectively. A safety threshold is set. If the updated offset exceeds the safety threshold, it is determined that the global gradient has abruptly changed direction, and a smoothing correction is performed. The formula for the smoothing correction is: ； in, The global aggregated gradient is smoothed and corrected. This is the adjustment coefficient; Otherwise, no correction is performed, and the current round's global aggregated gradient is used directly for updating.

7. A gradient corrector, applied to the server side of federated learning, characterized in that, include: The feature extraction unit is used to obtain the gradient vectors uploaded by each client and logically decouple them to obtain the shallow feature gradient and the deep semantic gradient. A multi-granularity conflict detection unit is used to perform conflict detection on the shallow feature gradient and the deep semantic gradient respectively, and generate a hierarchical conflict mask matrix. The elastic correction unit is used to perform elastic projection correction on the gradient components with conflict based on the hierarchical conflict mask matrix, and at the same time introduces the exponential moving average of the global aggregated gradient as an inertial anchor point to constrain the correction direction from deviating from the historical optimization trajectory. Based on the hierarchical conflict mask matrix, the expression for elastic projection correction of conflicting gradient components is as follows: ； ； in, The gradient after elastic projection correction for client i; This indicates that there is a conflicting gradient at client i; , The dynamic elastic coefficient; For conflict resolution items; for exist Projection in the orthogonal direction; This is for inertial anchor point compensation. for Projection onto the inertial anchor point; , The current round The previous round's inertial anchor point, i.e., the exponential moving average vector of the global aggregated gradient, is used to prevent gradient updates from deviating from the optimal trajectory. The momentum coefficient is used to smooth historical trajectories. This is the global aggregate gradient from the previous round; The signal-to-noise ratio hierarchical aggregation unit is used to cluster the corrected gradient into several gradient clusters, calculate the signal-to-noise ratio for each cluster, and assign nonlinear aggregation weights based on the signal-to-noise ratio of each cluster to generate a global aggregated gradient. The PID gain scheduling unit is used to dynamically calculate the gain weight and global aggregation step size based on the global aggregated gradient, using the gradient conflict ratio of the current round and the global loss change rate to form an error signal. It then updates the global aggregated gradient and distributes it to each client to complete the iteration of this round of federated learning. Specifically: The gradient conflict ratio is calculated based on the hierarchical conflict mask matrix and is used to represent the intensity of the current client gradient conflict. The expression is: ； in, The conflict ratio for the current round t; Total number of clients; For indicator functions; These are the elements of the hierarchical collision mask matrix; The formula for calculating the error signal is: ； ； ； in, This is the error signal for the current round t; , These are weighting coefficients used to balance the contributions of conflict bias and loss rate of change; This is due to the conflict ratio deviation. The ideal conflict ratio for the target; The global loss rate of change; , These represent the global losses for the current round t and the previous round, respectively. The interval between rounds; Then, using the error signal as input, the formula for dynamically calculating the gain weight through the PID control law is as follows: ； in, The gain weight for the current round t; , , These are the proportional coefficient, integral coefficient, and differential coefficient, respectively. For history Error signal for each round; , They are respectively for , Integral; Gain weight The mapping to the global aggregation step size required for actual optimization is given by the formula: ； in, This represents the global aggregation step size for the current round t. This is the initial base step size; This is the step size decay coefficient; Based on the global aggregation step size With the gain weight Update the global aggregated gradient for each cluster to obtain the global aggregated gradient that integrates the signal-to-noise ratio and PID control. .

8. The gradient corrector according to claim 7, characterized in that, Also includes: The global smoothing unit is used to monitor the global update direction of the most recent round using a sliding window. If a sudden change in direction is detected that exceeds a preset safety threshold, the current global aggregated gradient is convexly fused with the previous round to generate a smoothed global aggregated gradient, which is then updated and sent to each client.

Images