Adaptive direction correction two-stage differential privacy federated learning method and system
By employing a two-stage differential privacy federated learning method with adaptive orientation correction, the problems of fixed noise scale and model drift are solved, achieving a more reasonable allocation of privacy budget and improved model stability, adapting to different training states and non-independent identically distributed scenarios.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- CHANGCHUN UNIV
- Filing Date
- 2026-06-15
- Publication Date
- 2026-07-14
AI Technical Summary
In existing federated learning systems, the fixed noise scale or simple decay leads to rigid allocation of privacy budgets, making it difficult to adapt to the early and late stages of training and the states of different clients. Furthermore, in non-independent and identically distributed scenarios, the model update direction varies greatly, resulting in model drift and performance loss.
A two-stage differential privacy federated learning method with adaptive orientation correction is adopted. The dynamic noise scaling factor is calculated by recording historical information on the client side, differential privacy noise is applied, and the model update orientation correction and classification head reweight training are performed on the server side to optimize the global model.
It achieves adaptive adjustment of noise intensity based on training state, reduces uneven allocation of privacy budget, suppresses model drift, improves convergence stability and performance, and reduces performance loss caused by classification head noise.
Smart Images
Figure CN122389087A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of federated learning technology, specifically to a two-stage differential privacy federated learning method and system with adaptive direction correction. Background Technology
[0002] With the continuous improvement of data security laws and regulations, multi-party collaborative data modeling is being increasingly widely used in fields such as healthcare, finance, industrial internet, and smart terminals. Traditional centralized training typically requires uploading the raw data from all participants to a central server for unified modeling. While this approach is direct, it is prone to unauthorized transfer of sensitive data and fails to meet requirements such as data localization, minimum necessary authorization, and cross-institutional compliance auditing. Federated learning, through the concept of "data doesn't move, model moves," allows clients to complete model training locally, exchanging only model parameters or gradient information. This has become an important technological path for solving data silos and collaborative intelligence.
[0003] However, simply training the model locally and uploading updates does not completely eliminate the risk of privacy breaches. Attackers can still perform membership inference, attribute inference, and gradient inversion attacks based on parameter updates, gradient directions, and intermediate representations. Therefore, existing federated learning systems typically incorporate differential privacy mechanisms, which statistically reduce the impact of a single sample on the model output by adding random noise to the update amount, gradient, or intermediate features.
[0004] Existing differential privacy federated learning schemes generally suffer from the following shortcomings: First, the noise scale often adopts a fixed value or a simple linear decay method, which is difficult to adapt to the early training stage, the later training stage, and the training state of different clients simultaneously, easily leading to rigid allocation of privacy budget; Second, in non-independent and identically distributed scenarios, the local data of each client is significantly skewed, and the update direction varies greatly. If directly averaged and aggregated, it is easy to cause model drift, affecting the convergence speed and final accuracy; Third, existing methods often uniformly add noise to the entire model without distinguishing the differences in privacy sensitivity and functional division between the feature extractor and the classification head. As a result, the classification head, which could be further optimized on the server side, is also forced to be subjected to noise perturbation, causing performance loss. Summary of the Invention
[0005] The purpose of this invention is to provide a two-stage differential privacy federated learning method and system with adaptive orientation correction, so as to solve the problems mentioned in the background art.
[0006] To achieve the above objectives, the present invention provides the following technical solution:
[0007] A two-stage differential privacy federated learning method with adaptive orientation correction, the method comprising:
[0008] The server distributes a global model containing a feature extractor and a classification head to selected clients, which then train the global model locally using local data and record historical information reflecting the training status.
[0009] The client determines a dynamic noise scaling factor based on the historical information, and applies differential privacy noise to the intermediate features generated during local training based on the dynamic noise scaling factor to obtain noisy features.
[0010] The client uploads the locally trained model update and the noisy features to the server. The server receives the model update uploaded by at least one client, and performs orientation correction on the model update of each client according to the consistency relationship between the historical model update and the currently received model update. Then, it aggregates all the corrected model updates to obtain the updated global model.
[0011] The server freezes the feature extractor in the updated global model and retrains the classification head based on the noisy features and corresponding labels from multiple clients.
[0012] The server compares the performance of the classification heads before and after retraining, and decides whether to retain the classification head parameters after retraining based on the comparison results.
[0013] As a further embodiment of the present invention, the client determines a dynamic noise scaling factor based on the historical information, and applies differential privacy noise to the intermediate features generated during local training based on the dynamic noise scaling factor to obtain noisy features, specifically including:
[0014] The client calculates the dynamic noise scaling factor and sensitivity based on historical information recorded during local training. The historical information includes the average gradient norm, local sample size, learning rate change information, current communication round, loss change rate, and current training progress.
[0015] The noise scale is calculated based on the dynamic noise scaling factor and sensitivity, and the differential privacy noise is determined based on the noise scale.
[0016] Differential privacy noise is applied to the intermediate features generated during local training to obtain noisy features.
[0017] As a further aspect of the present invention, the direction correction based on the consistency relationship between historical model updates and currently received model updates specifically includes:
[0018] The server constructs the historical average direction based on model updates within the historical update window;
[0019] Calculate the consistency metric between the current client's model update and the historical average direction;
[0020] The trend correction factor is determined based on the consistency metric.
[0021] The current client's model update is combined with the historical average direction using the trend correction factor to obtain the corrected model update.
[0022] As a further aspect of the present invention, the consistency metric is the cosine similarity between the current client's model update vector and the historical average direction vector.
[0023] As a further aspect of the present invention, the current combination of model updates and historical average direction on the client is as follows:
[0024] ;
[0025] in, As a trend correction factor, For the corrected client update, Update the local model. The historical average direction.
[0026] As a further aspect of the present invention, the retraining of the classification head specifically includes:
[0027] The server keeps the parameters of the feature extractor unchanged, takes the noisy features as input, uses the corresponding labels as supervision signals, and optimizes only the network parameters of the classification head.
[0028] As a further aspect of the present invention, the performance comparison of the classification head before and after retraining specifically includes:
[0029] The server calculates the prediction accuracy or loss value of the classifier before and after retraining on the local evaluation dataset.
[0030] When the performance of the retrained classification head is lower than that of the classification head before retraining, the server will revert the classification head parameters to the state before retraining.
[0031] This invention also provides a two-stage differential privacy federated learning system with adaptive orientation correction, the system comprising:
[0032] The local training module is used to distribute a global model containing a feature extractor and a classification head to selected clients. The clients use local data to train the global model locally and record historical information reflecting the training status.
[0033] The noise-adding module is used to determine a dynamic noise scaling factor based on the historical information, and to apply differential privacy noise to the intermediate features generated during local training based on the dynamic noise scaling factor to obtain the noise-adding features.
[0034] An aggregation module is used to upload the locally trained model update and the noisy features to the server. The server receives model updates uploaded by at least one client, and performs orientation correction on the model updates of each client according to the consistency relationship between historical model updates and currently received model updates. Then, it aggregates all the corrected model updates to obtain the updated global model.
[0035] The retraining module is used to freeze the feature extractor in the updated global model and retrain the classification head based on the noisy features and corresponding labels from multiple clients.
[0036] The comparison module is used to compare the performance of the classifier before and after retraining, and decide whether to retain the parameters of the retrained classifier based on the comparison results.
[0037] Compared with existing technologies, the beneficial effects of this invention are: adaptively adjusting noise intensity according to the training state, enabling more reasonable privacy budget allocation in the early and late stages of training and across different clients, thus improving noise utilization efficiency; suppressing update drift in non-independent and identically distributed scenarios through direction correction, enhancing global model convergence stability, and reducing oscillation risk; placing privacy protection focus on the feature extraction stage through model decoupling and server-side retraining of the classification head, reducing performance degradation caused by adding noise to the classification head while ensuring that the original data remains within its domain; and avoiding occasional degradation during server-side retraining through a performance fallback mechanism, thereby improving system usability and deployment reliability. Attached Figure Description
[0038] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention.
[0039] Figure 1 This is a diagram illustrating the overall architecture of a two-stage differential privacy federated learning system with adaptive orientation correction, as provided in an embodiment of the present invention.
[0040] Figure 2 This is a flowchart of a two-stage differential privacy federated learning method for adaptive orientation correction provided in an embodiment of the present invention.
[0041] Figure 3 This is a diagram of the decoupling and two-stage training structure provided in an embodiment of the present invention.
[0042] Figure 4 This is a schematic diagram illustrating the principle of dynamic noise scaling and orientation correction provided in an embodiment of the present invention.
[0043] Figure 5 This is a schematic diagram of server-side classification header training and performance rollback provided in an embodiment of the present invention. Detailed Implementation
[0044] To make the technical problems to be solved, the technical solutions, and the beneficial effects of the present invention clearer, the present invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the present invention and are not intended to limit the present invention.
[0045] like Figure 1 and Figure 2 As shown in the embodiment of the present invention, the two-stage differential privacy federated learning method with adaptive direction correction includes:
[0046] The server distributes a global model containing a feature extractor and a classification head to selected clients, which then train the global model locally using local data and record historical information reflecting the training status.
[0047] The client determines a dynamic noise scaling factor based on the historical information, and applies differential privacy noise to the intermediate features generated during local training based on the dynamic noise scaling factor to obtain noisy features.
[0048] The client uploads the locally trained model update and the noisy features to the server. The server receives the model update uploaded by at least one client, and performs orientation correction on the model update of each client according to the consistency relationship between the historical model update and the currently received model update. Then, it aggregates all the corrected model updates to obtain the updated global model.
[0049] The server freezes the feature extractor in the updated global model and retrains the classification head based on the noisy features and corresponding labels from multiple clients.
[0050] The server compares the performance of the classification heads before and after retraining, and decides whether to retain the classification head parameters after retraining based on the comparison results.
[0051] In this embodiment, the server constructs a global model including a feature extractor and a classification head, and randomly selects some clients to participate in training in each communication round. During local training, the clients record historical information such as gradient norm, loss change, and training progress, and calculate the dynamic noise scaling factor accordingly. Based on the learning rate, gradient clipping threshold, and local sample size, the sensitivity is calculated. Differential privacy noise is applied to intermediate features, and then uploaded along with the local model update. The server establishes a direction correction relationship based on the consistency between historical updates and current client updates, and performs correction and aggregation on client updates. After aggregation, the server freezes the feature extractor, retrains the classification head only based on the noisy features and labels, and sets a performance rollback mechanism.
[0052] The method of this invention is executed collaboratively by a server and multiple clients. The server is responsible for client selection, model aggregation, orientation correction, retraining of the classification head, and performance fallback; the clients are responsible for local training, recording historical information, adding noise to features, and uploading results. The global model consists of a feature extractor and a classification head, where the feature extractor is used to learn sample representations, and the classification head is used to complete the prediction output. For image tasks, the feature extractor can be composed of convolutional layers, activation layers, and pooling layers, and the classification head can be composed of one or more fully connected layers; for text, speech, time series, or multimodal tasks, other decoupled neural network structures can also be used.
[0053] In a preferred embodiment of the present invention, the client determines a dynamic noise scaling factor based on the historical information, and applies differential privacy noise to the intermediate features generated during local training based on the dynamic noise scaling factor to obtain noisy features, specifically including:
[0054] The client calculates the dynamic noise scaling factor and sensitivity based on historical information recorded during local training. The historical information includes the average gradient norm, local sample size, learning rate change information, current communication round, loss change rate, and current training progress.
[0055] The noise scale is calculated based on the dynamic noise scaling factor and sensitivity, and the differential privacy noise is determined based on the noise scale.
[0056] Differential privacy noise is applied to the intermediate features generated during local training to obtain noisy features.
[0057] like Figure 4 As shown, in this embodiment, the client-side local training process includes:
[0058] Let the global model parameters received by the i-th client in the r-th communication round be... , Given the initial parameters, what is actually optimized during training is... The local dataset is D i The client performs stochastic gradient descent training on local mini-batch samples. The local training objective can be written as:
[0059] ;
[0060] in, Indicates the first Model parameters to be optimized during local training on a single client; Indicates the first The global model parameters received at the start of round communication; This represents the set of mini-batch samples used by the i-th client in the r-th round of training. Represents the loss function. Indicates the input sample; This indicates the corresponding real label.
[0061] During training, the client prioritizes recording status information such as the average gradient norm, the loss values of the most recent rounds, the current communication round, learning rate changes, and local training stability, providing a basis for subsequent adaptive adjustment of noise intensity. If multiple mini-batches are used for local training, the client can also take the average or moving average of the gradient norms of multiple mini-batches, thereby reducing the impact of occasional abnormal batches on noise adjustment.
[0062] Sensitivity calculation and dynamic noise scaling: The client base is based on the learning rate. Gradient clipping threshold C and local sample size n i Calculate the current communication round sensitivity S:
[0063] ;
[0064] The client uses the historical average gradient norm. Loss change rate and training progress Calculate the dynamic noise scaling factor:
[0065] ;
[0066] in, Based on the scaling factor, P r For the training progress, This represents the total number of communication rounds.
[0067] To accommodate a lighter-weight native implementation, another implementation approach can employ a simplified scaling expression that monotonically increases with the number of communication rounds. .
[0068] The noise scale depends on the selected differential privacy mechanism and privacy budget. Privacy failure probability Sensitivity The dynamic noise scaling factor is determined; in one implementation of the Gaussian mechanism, it can be written as:
[0069] ;
[0070] in, This is the scaling term corresponding to the Gaussian mechanism. This represents the dynamic noise scaling factor for the i-th client in the r-th round; when using the Laplace mechanism, the corresponding scale parameter can be expressed as... This expression can be implemented using Gaussian mechanisms, Laplace mechanisms, and dynamic noise scaling strategies.
[0071] The noise scale is calculated based on the dynamic noise scaling factor and sensitivity. and utilize noise scale Generate differential privacy noise .
[0072] Feature noise addition and result upload: such as Figure 3 As shown, the client utilizes a feature extractor Perform forward computation on local samples to extract intermediate features. Differential privacy noise is applied to the feature space:
[0073] ;
[0074] in, Differential privacy noise can be obtained by sampling from a Gaussian or Laplace distribution. The client will then add the noisy features. With tags Write to the feature buffer and update it with the local model after local training is complete. These are uploaded to the server. Since the server receives noisy intermediate representations rather than the original samples, subsequent optimizations can be performed without violating data locality boundaries. The client can cache features from recent batches or selected according to a certain sampling ratio to balance communication overhead with the amount of information required for server-side retraining.
[0075] After receiving the noisy features uploaded by each client, the server can first perform dimensionality consistency checks, label integrity checks, and sample size statistics, and then construct the server-side feature training dataset. The server can load the feature training data in batches and update the classification head parameters using stochastic gradient descent, momentum optimization, or other first-order optimization methods.
[0076] In a preferred embodiment of the present invention, the direction correction based on the consistency relationship between historical model updates and currently received model updates specifically includes:
[0077] The server constructs the historical average direction based on model updates within the historical update window;
[0078] Calculate the consistency metric between the current client's model update and the historical average direction;
[0079] The trend correction factor is determined based on the consistency metric.
[0080] The current client's model update is combined with the historical average direction using the trend correction factor to obtain the corrected model update.
[0081] The consistency metric uses the cosine similarity between the current client's model update vector and the historical average direction vector.
[0082] In this embodiment, the server maintains a historical update window and constructs a historical average direction based on the previous H rounds of global updates. And calculate the consistency between the current client update and the historical average direction:
[0083] ;
[0084] In a preferred embodiment of the present invention, the combination of the current client's model update and the historical average direction is as follows:
[0085] ;
[0086] in, As a trend correction factor, For the corrected client update, Update the local model. The historical average direction.
[0087] In this embodiment, the server can continue to aggregate updates from each client using a sample size weighting method:
[0088] ;
[0089] In another implementation, an element-wise orientation correction matrix can be constructed based on parameter differences and applied to the local update. The correction result is then combined with the original local parameters. Regardless of whether similarity scaling, rotation correction, or element-wise orientation correction is used, the goal is to suppress update offset under Non-IID conditions and make the global model update more stable.
[0090] In a preferred embodiment of the present invention, the retraining of the classification head specifically includes:
[0091] The server keeps the parameters of the feature extractor unchanged, takes the noisy features as input, uses the corresponding labels as supervision signals, and optimizes only the network parameters of the classification head.
[0092] In this embodiment, as Figure 5 As shown, the server concatenates the noisy features and labels uploaded by each client to form a training set. And freeze the feature extractor parameters. Only optimize the classification header parameters :
[0093] ;
[0094] in, This refers to the training set built on the server side. This represents the classification head loss function computed on the training set F. This represents the m-th noisy feature. represents the corresponding label; m represents the sample index; M represents the total number of noisy features.
[0095] In a preferred embodiment of the present invention, the performance comparison of the classification head before and after retraining specifically includes:
[0096] The server calculates the prediction accuracy or loss value of the classifier before and after retraining on the local evaluation dataset.
[0097] When the performance of the retrained classification head is lower than that of the classification head before retraining, the server will revert the classification head parameters to the state before retraining.
[0098] In this embodiment, after the classifier retraining is completed, the server compares the performance of the new and old classifiers on the server-side evaluation data; if the performance of the new classifier is not higher than that of the original classifier, parameter rollback is performed.
[0099] ;
[0100] This mechanism avoids the negative impact of occasional overfitting, class skewness, or feature noise fluctuations on the classification head, allowing the server to focus on training and forming a controlled update path. For deployment scenarios requiring higher robustness, the server can also use validation set accuracy, loss value, or a combination of multiple metrics as performance comparison criteria.
[0101] In a preferred embodiment of the present invention, before retraining the classification head, the server further performs the following: counts the total number of received noisy features or the number of categories covered; if the total number is lower than a preset threshold or the number of categories does not meet a preset condition, the current round of classification head retraining is skipped.
[0102] In this embodiment, since the retraining input is the noisy features rather than the original samples, the server does not need to access the original training data to re-optimize the classification head. Before entering retraining, the server first detects the total number of noisy feature samples. If the total number of uploaded feature samples in a certain communication round is too small, or the categories covered by the uploaded features are insufficient, the server can choose to skip the retraining of the classification head in this round and only retain the global model after orientation correction to avoid model oscillation caused by invalid training.
[0103] Client-side feature sampling and cache management: Clients do not need to upload all intermediate features of local samples. Instead, they can select features to upload based on a set sampling ratio, a priority strategy for the most recent batches, or a balanced sampling strategy by class. For terminals with limited computing resources or network bandwidth, features from later batches that better represent the current model state can be uploaded first. For terminals with severely imbalanced class distributions, the retention ratio of features corresponding to minority classes can be appropriately increased so that the server-side classification head can obtain more stable decision boundary updates. Clients can also set a maximum feature buffer. When the cache reaches a threshold, feature samples can be managed using first-in-first-out (FIFO), last-in-first-out (LIFO), or replacement based on importance scores.
[0104] When different clients only hold local category data or local domain data, the local optimal directions of each client often differ significantly. Directly averaging these directions can easily cause the global model to oscillate between multiple directions. This invention constructs a trend correction factor by combining historical average direction and current update consistency. This can preserve effective individual differences while limiting updates that deviate too much from the global trend from entering the aggregation result. At the same time, server-side classification head-weighted training further utilizes noisy features uploaded by multiple clients to recalibrate the decision boundary, thereby making the overall model more adaptable to training environments with significant category skew and domain differences.
[0105] Experimental verification: To verify the effectiveness and superiority of the method of the present invention, the MNIST and CIFAR-10 datasets were selected, and the method of the present invention (AdaDirTwin) and the comparison method were analyzed in two scenarios: independent and identically distributed and non-independent and identically distributed.
[0106] Table 1. Test accuracy results (%) under different scenarios and method variations
[0107]
[0108] This invention also provides a two-stage differential privacy federated learning system with adaptive orientation correction, the system comprising:
[0109] The local training module is used to distribute a global model containing a feature extractor and a classification head to selected clients. The clients use local data to train the global model locally and record historical information reflecting the training status.
[0110] The noise-adding module is used to determine a dynamic noise scaling factor based on the historical information, and to apply differential privacy noise to the intermediate features generated during local training based on the dynamic noise scaling factor to obtain the noise-adding features.
[0111] An aggregation module is used to upload the locally trained model update and the noisy features to the server. The server receives model updates uploaded by at least one client, and performs orientation correction on the model updates of each client according to the consistency relationship between historical model updates and currently received model updates. Then, it aggregates all the corrected model updates to obtain the updated global model.
[0112] The retraining module is used to freeze the feature extractor in the updated global model and retrain the classification head based on the noisy features and corresponding labels from multiple clients.
[0113] The comparison module is used to compare the performance of the classifier before and after retraining, and decide whether to retain the parameters of the retrained classifier based on the comparison results.
[0114] The above description is merely a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention should be included within the protection scope of the present invention.
Claims
1. A two-stage differential privacy federated learning method with adaptive orientation correction, characterized in that, The method includes: The server distributes a global model containing a feature extractor and a classification head to selected clients, which then train the global model locally using local data and record historical information reflecting the training status. The client determines a dynamic noise scaling factor based on the historical information, and applies differential privacy noise to the intermediate features generated during local training based on the dynamic noise scaling factor to obtain noisy features. The client uploads the locally trained model update and the noisy features to the server. The server receives the model update uploaded by at least one client, and performs orientation correction on the model update of each client according to the consistency relationship between the historical model update and the currently received model update. Then, it aggregates all the corrected model updates to obtain the updated global model. The server freezes the feature extractor in the updated global model and retrains the classification head based on the noisy features and corresponding labels from multiple clients. The server compares the performance of the classification heads before and after retraining, and decides whether to retain the classification head parameters after retraining based on the comparison results.
2. The adaptive orientation correction two-stage differential privacy federated learning method according to claim 1, characterized in that, The client determines a dynamic noise scaling factor based on the historical information, and applies differential privacy noise to the intermediate features generated during local training based on the dynamic noise scaling factor to obtain noisy features, specifically including: The client calculates the dynamic noise scaling factor and sensitivity based on historical information recorded during local training. The historical information includes the average gradient norm, local sample size, learning rate change information, current communication round, loss change rate, and current training progress. The noise scale is calculated based on the dynamic noise scaling factor and sensitivity, and the differential privacy noise is determined based on the noise scale. Differential privacy noise is applied to the intermediate features generated during local training to obtain noisy features.
3. The adaptive orientation correction two-stage differential privacy federated learning method according to claim 1, characterized in that, The direction correction based on the consistency relationship between historical model updates and currently received model updates specifically includes: The server constructs the historical average direction based on model updates within the historical update window; Calculate the consistency metric between the current client's model update and the historical average direction; The trend correction factor is determined based on the consistency metric. The current client's model update is combined with the historical average direction using the trend correction factor to obtain the corrected model update.
4. The adaptive orientation correction two-stage differential privacy federated learning method according to claim 3, characterized in that, The consistency metric uses the cosine similarity between the current client's model update vector and the historical average direction vector.
5. The adaptive orientation correction two-stage differential privacy federated learning method according to claim 3, characterized in that, The current client's model update and historical average direction are combined as follows: ； in, As a trend correction factor, For the corrected client update, Update the local model. The historical average direction.
6. The adaptive orientation correction two-stage differential privacy federated learning method according to claim 1, characterized in that, The retraining of the classification head specifically includes: The server keeps the parameters of the feature extractor unchanged, takes the noisy features as input, uses the corresponding labels as supervision signals, and optimizes only the network parameters of the classification head.
7. The adaptive orientation correction two-stage differential privacy federated learning method according to claim 1, characterized in that, The performance of the classification heads before and after retraining is compared, specifically including: The server calculates the prediction accuracy or loss value of the classifier before and after retraining on the local evaluation dataset. When the performance of the retrained classification head is lower than that of the classification head before retraining, the server will revert the classification head parameters to the state before retraining.
8. A two-stage differential privacy federated learning system with adaptive orientation correction, used to implement the two-stage differential privacy federated learning method with adaptive orientation correction as described in any one of claims 1-7, characterized in that, The system includes: The local training module is used to distribute a global model containing a feature extractor and a classification head to selected clients. The clients use local data to train the global model locally and record historical information reflecting the training status. The noise-adding module is used to determine a dynamic noise scaling factor based on the historical information, and to apply differential privacy noise to the intermediate features generated during local training based on the dynamic noise scaling factor to obtain the noise-adding features. An aggregation module is used to upload the locally trained model update and the noisy features to the server. The server receives model updates uploaded by at least one client, and performs orientation correction on the model updates of each client according to the consistency relationship between historical model updates and currently received model updates. Then, it aggregates all the corrected model updates to obtain the updated global model. The retraining module is used to freeze the feature extractor in the updated global model and retrain the classification head based on the noisy features and corresponding labels from multiple clients. The comparison module is used to compare the performance of the classifier before and after retraining, and decide whether to retain the parameters of the retrained classifier based on the comparison results.