Federal active data labeling method based on inconsistency between models and inconsistency in models
By fusing the inconsistency scores between models and within models to select samples for labeling in federated learning, the problem of insufficient stability of sample selection strategy in federated active learning is solved, the model performance and generalization ability are improved, and the labeling cost is reduced.
Patent Information
- Application Number
- CN202510829926.X
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-06-20
- Publication Date
- 2025-10-10
AI Technical Summary
Existing federated active learning methods fail to fully utilize the differences between the models of multiple participants in a federated environment, resulting in insufficient stability and robustness of the sample selection strategy. In addition, it is difficult to select the most informative samples for labeling when the data distribution is uneven and the noise interference is large.
By introducing a federated active data labeling method for inter-model inconsistency and intra-model inconsistency, the prediction probabilities of the server model and the local distillation model are used to calculate the inconsistency scores between models and within models. The samples with the highest fusion scores are selected for labeling, and the model parameters are updated through the aggregation algorithm of federated learning.
It effectively eliminates the prediction bias between local and server models, selects the most informative samples for annotation, improves model performance and generalization ability, and reduces annotation costs.
Smart Images
Figure CN120764631A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of computer technology applications, and primarily relates to a federated active data labeling method based on inter-model inconsistency and intra-model inconsistency for distributed machine learning scenarios where data privacy is limited and labeling costs are high. Background Art
[0002] With the rapid development of artificial intelligence (AI), data-driven model training has become a mainstream learning paradigm. Deep learning models have achieved remarkable results in tasks such as image recognition and natural language processing. However, training high-performance models typically relies on large amounts of labeled data, which in practice results in high manual labeling costs. To address this, active learning methods have been proposed. Their core idea is to proactively select and label samples that are most valuable for improving model performance from a large number of unlabeled samples, thereby improving model training efficiency.
[0003] Traditional centralized active learning methods are difficult to directly apply in scenarios where data distribution requires privacy protection. Federated learning, as a distributed learning framework, has garnered widespread attention in recent years. It allows multiple participants to retain data locally and perform joint modeling by sharing model parameters or updates, avoiding the centralized transmission of raw data and effectively protecting data privacy. In this context, federated active learning has emerged as a research direction that combines the advantages of federated and active learning. Its goal is to effectively identify data samples that require priority labeling while ensuring data privacy.
[0004] Most existing federated active learning methods focus on the model's uncertainty score for samples, selecting difficult-to-classify samples for labeling. However, these methods often rely on a single model or local uncertainty estimates, failing to fully exploit the diversity among the models of multiple participants in a federated environment. Furthermore, some methods ignore variations in the perception of samples within the same model at different training stages or from different perspectives, resulting in insufficient stability and robustness in the sample selection strategy.
[0005] Therefore, how to integrate the inconsistencies between models and within models in the federated learning framework to build a more accurate and more generalizable sample selection mechanism has become an urgent problem to be solved. Summary of the Invention
[0006] The purpose of the present invention is to provide a federated active data labeling method based on inter-model inconsistency and intra-model inconsistency to solve the problem in the prior art that in federated learning, the data distribution of the local client may be significantly different from the data distribution of the server model, resulting in a large deviation in the prediction of the local model and the server model on rare local categories, and this deviation will interfere with the effectiveness of active sample selection.
[0007] To achieve the above object, the present invention adopts the following technical solutions:
[0008] A federated active data labeling method based on inter-model inconsistency and intra-model inconsistency includes the following steps:
[0009] Step S1: Initialize the federated learning active data labeling task:
[0010] Configure a federated learning active data annotation scenario that includes 1 server model, N local models, and N local distillation models. Each client maintains a pool of labeled and unlabeled samples and sets the total number of active annotation rounds Q.
[0011] Step S2, active data labeling:
[0012] Initial round: Each client randomly selects unlabeled samples to label and update the sample pool;
[0013] Subsequent rounds: Use the server model and local labeled samples to distill and train the local distillation model; parallelize the learning-based prediction probabilities of the server / local / distillation models for unlabeled samples, and the data-driven prediction probabilities of the three models for the same samples; calculate the inter-model inconsistency score based on the data-driven probabilities; calculate the intra-model inconsistency score based on the two probabilities of the local model; fuse the scores and select the highest-scoring sample annotation to update the sample pool;
[0014] Step S3, federated learning training:
[0015] Perform T rounds of communication training. Each round includes: the client downloads the server model to initialize the local model; trains the local model based on the local labeled sample pool; the client uploads the local model parameters to the server, and updates the server model through the aggregation algorithm;
[0016] Step S4: If Q rounds of active data labeling have been performed, then the process ends; otherwise, the process proceeds to step S2.
[0017] Furthermore, in step S1,
[0018] The federated learning active data annotation setting scenario includes 1 server model f G , N local models and N local distillation models Each local client k maintains a labeled sample pool and an unlabeled sample pool and have one or more pools of unlabeled samples The annotator provides labels for samples in the dataset; the number of active data annotation rounds is set to Q rounds.
[0019] Furthermore, in step S1,
[0020] Marked sample pool on client k It satisfies: x i Represents the input feature corresponding to the i-th sample, y i ∈Y k ={1,…,C} represents the sample label corresponding to the i-th sample, Y k Representation dataset The corresponding label set, C is the label set Y k The number of categories contained in Indicates the labeled sample pool The number of samples included in ;
[0021] Unlabeled sample pool on client k It satisfies: x i represents the input feature corresponding to the i-th sample, Indicates the unlabeled sample pool The number of samples included in ;
[0022] 1 server model f G , N local models and N local distillation models All are C classification models, among which the server model f G and N local models The structure is identical, G refers to the server model f G The parameter θ k Refers to the local model on client k Parameters, Refers to the local distillation model on client k Parameters.
[0023] Furthermore, in step S2,
[0024] If it is the initial round, then k random unlabeled samples are collected from the pool on each client. Select a batch of samples and give them to local annotators for annotation, and add them to the local labeled sample pool accordingly The sample pool has never been marked Removed;
[0025] Otherwise, perform the following steps:
[0026] (a) First, at each client k, use the server model f G and a local pool of labeled samples Local distillation model using knowledge distillation Conduct training;
[0027] (b) After training, the server model f is used on each client. G , local model and local distillation models Unlabeled sample pool The samples in are evaluated to obtain the learning-based prediction probability and data-driven prediction probability of each sample;
[0028] (c) Using the data-driven prediction probabilities obtained by the sample from the three models, the corresponding inter-model inconsistency scores are calculated; using the learning-based prediction probabilities and data-driven prediction probabilities obtained by the sample from the local model, the corresponding intra-model inconsistency scores are calculated;
[0029] (d) Finally, the two inconsistency scores of the samples are fused, and the samples with the highest total inconsistency scores are queried and handed over to the local annotator for annotation, and the corresponding samples are added to the local labeled sample pool. The sample pool has never been marked Removed.
[0030] Furthermore, in step S2,
[0031] Local distillation model About the Marker Sample Pool A sample label pair (x i ,y i )’s distillation training loss is expressed as:
[0032]
[0033] in represents the specific loss function, Is the first loss term used to measure the model prediction output With the given label y i The difference in previous predictions, The second loss term is used to measure the model prediction output And the server model predicts the output f G (x i ) between the predicted differences;
[0034] For any sample x i , the learning-based prediction probability of any model f on label v mentioned in the above steps is defined as:
[0035]
[0036] Where f(v|x) represents the model f for sample x i The corresponding output on label v;
[0037] For the unlabeled sample pool on client k Any sample x in i , the prediction probability of model f based on data driving is calculated as follows:
[0038] (1) Using model f G Unlabeled sample pool Predict each sample in the dataset, calculate the prediction probability based on learning, and regard the label with the highest prediction probability as a pseudo label to obtain the updated unlabeled sample pool in Represents the pseudo label corresponding to the i-th sample;
[0039] (2) Use model f to extract the labeled sample pool and unlabeled sample pool The features of each sample in the pool are updated to obtain the labeled sample pool. and unlabeled sample pool where z i Indicates the feature corresponding to the i-th sample;
[0040] (3) Based on the extracted features, for each sample from the labeled sample pool and unlabeled sample pool The samples in the pool are calculated separately for them and the unlabeled samples The feature similarity between all samples in and the K samples with the highest feature similarity are obtained;
[0041] (4) For any sample x i , the data-driven prediction probability of any model f mentioned in the above steps on the label v is defined as:
[0042]
[0043] in Represents all labeled samples from the labeled sample pool and unlabeled sample pool The sample in the sample x i is the total number of K samples with the highest feature similarity;
[0044] For the unlabeled sample pool on client k Any sample x in i , the inconsistency score between models is calculated as:
[0045]
[0046] in Indicates the specific method used to calculate sample x i In local model and local distillation models A data-driven measure of the difference between the predicted probability distributions on , Indicates the specific method used to calculate sample x i In the server model f G and local distillation models A data-driven measure of the difference between the predicted probability distributions on ;
[0047] For the unlabeled sample pool on client k Any sample x in i , the inconsistency score of the model itself is calculated as:
[0048]
[0049] in Indicates the specific method used to calculate the sample x in the local model A measure of the difference between the probability distributions of data-driven predictions and learning-based predictions on ;
[0050] For the unlabeled sample pool on client k Any sample x in i , whose total inconsistency score is:
[0051] I(x i )=I intra (x i )×I inter (x i ).
[0052] Furthermore, in step S3,
[0053] A total of T communication rounds of training are carried out. In each communication round t, each client k uses the server model f of the previous round sent by the server G To initialize the local model Based on the local labeled sample pool Perform model training; after training is completed, each client uploads the local model parameters to the server, and the server obtains the new f according to the aggregation algorithm of federated learning G And send it to each client.
[0054] Furthermore, in step S3,
[0055] Utilize server model f G To initialize the local model When , directly replace the parameters;
[0056] Local Model About the Marker Sample Pool A sample label pair (x i ,y i )’s training loss is expressed as:
[0057]
[0058] in Represents a specific loss function used to measure the model prediction output With the given label y i The difference in predictions between
[0059] Server Model f G The update method using the aggregation algorithm is expressed as:
[0060]
[0061] in Represents a specific aggregation algorithm used to fuse the parameters of multiple client models to obtain the server model parameters.
[0062] Beneficial effects: The federated active data labeling method based on inter-model inconsistency and intra-model inconsistency provided by the present invention is suitable for distributed machine learning scenarios with limited data privacy and high labeling costs. On the one hand, the present invention effectively eliminates the prediction bias between local and server models by introducing a data-driven probabilistic paradigm, weakening the interference of data heterogeneity on the active data selection process in the federated learning scenario; on the other hand, combining the characteristics of federated learning multi-models, an efficient inconsistency-based fusion query strategy is designed, which can select the most informative samples for labeling under conditions of uneven data distribution, large noise interference, and limited labeling resources, thereby achieving better server model performance at a limited labeling cost. BRIEF DESCRIPTION OF THE DRAWINGS
[0063] Figure 1 is a flow chart of the method of the present invention;
[0064] Figure 2 It is a framework diagram of the method of the present invention;
[0065] Figure 3 This is a framework diagram of the method of the present invention for active data labeling on client k. DETAILED DESCRIPTION
[0066] like Figure 1 and Figure 2 The flowchart and framework structure diagram of the federated active data annotation process are shown.
[0067] A federated active data labeling method based on inter-model inconsistency and intra-model inconsistency includes the following steps:
[0068] Step S1, initialization of the federated learning active data labeling task;
[0069] The federated learning active data annotation scenario used in this method includes a server model f G , N local models and N local distillation models Each local client (taking client k as an example) maintains a labeled sample pool and an unlabeled sample pool and have one or more pools of unlabeled samples The annotator provides labels for samples in the dataset. The number of active data annotation rounds is set to Q rounds.
[0070] Marked sample pool on client k It satisfies: x i Represents the input feature corresponding to the i-th sample, y i ∈Y k ={1,…,C} represents the sample label corresponding to the i-th sample, Y k Representation dataset The corresponding label set, C is the label set Y k The number of categories contained in Indicates the labeled sample pool The number of samples included in .
[0071] Unlabeled sample pool on client k It satisfies: x i represents the input feature corresponding to the i-th sample, Indicates the unlabeled sample pool The number of samples included in .
[0072] 1 server model f G , N local models and N local distillation models All are C classification models, among which the server model f G and N local models The structure is identical, G refers to the server model f G The parameter θ k Refers to the local model on client k Parameters, Refers to the local distillation model on client k Parameters.
[0073] Step S2, active data labeling;
[0074] If it is the initial round, on each client (take client k as an example), randomly select the unlabeled sample pool Select a batch of samples and give them to local annotators for annotation, and add them to the local labeled sample pool accordingly The sample pool has never been marked Removed.
[0075] Otherwise, refer to Figure 3 As shown in the framework diagram, perform the following steps:
[0076] (a) First, on each client (taking client k as an example), use the server model f G and a local pool of labeled samples Local distillation model using knowledge distillation Conduct training;
[0077] (b) After training, the server model f is used on each client. G , local model and local distillation models Unlabeled sample pool The samples in are evaluated to obtain the learning-based prediction probability and data-driven prediction probability of each sample;
[0078] (c) Using the data-driven prediction probabilities obtained by the sample from the three models, the corresponding inter-model inconsistency scores are calculated; using the learning-based prediction probabilities and data-driven prediction probabilities obtained by the sample from the local model, the corresponding intra-model inconsistency scores are calculated;
[0079] (d) Finally, the two inconsistency scores of the samples are fused, and the samples with the highest total inconsistency scores are queried and handed over to the local annotator for annotation, and the corresponding samples are added to the local labeled sample pool. The sample pool has never been marked Removed.
[0080] Among them, the local distillation model About the Marker Sample Pool A sample label pair (x i ,y i )’s distillation training loss is expressed as:
[0081]
[0082] in represents the specific loss function, The cross entropy loss term is used to measure the model prediction output With the given label y i The difference in previous predictions, KL divergence loss is used to measure the model prediction output And the server model predicts the output f G (x i ), λ is the weight used to control the two loss terms, and T is the temperature coefficient used to control the smoothness of the input distribution in the KL divergence loss term.
[0083] For any sample x i , the learning-based prediction probability of model f (which refers to all models mentioned in the above steps) on label v is defined as:
[0084]
[0085] Where f(v|x) represents the model f for sample x i The corresponding output at label v.
[0086] For the unlabeled sample pool on client k Any sample x in i , the prediction probability of model f based on data driving is calculated as follows:
[0087] (1) Using model f G Unlabeled sample pool Predict each sample in the dataset, calculate the prediction probability based on learning, and regard the label with the highest prediction probability as a pseudo label to obtain the updated unlabeled sample pool in Indicates the pseudo label corresponding to the i-th sample;
[0088] (2) Use model f to extract the labeled sample pool and unlabeled sample pool The features of each sample in the pool are updated to obtain the labeled sample pool. and unlabeled sample pool where z i Indicates the feature corresponding to the i-th sample;
[0089] (3) Based on the extracted features, for each sample from the labeled sample pool and unlabeled sample pool The samples in the pool are calculated separately for them and the unlabeled samples The feature similarity between all samples in and the K samples with the highest feature similarity are obtained;
[0090] (4) For any sample x i , the data-driven prediction probability of model f (referring to all models mentioned in the above steps) on label v is defined as:
[0091]
[0092] in Represents all labeled samples from the labeled sample pool and unlabeled sample pool The sample in the sample x i is the total number of K samples with the highest feature similarity.
[0093] For the unlabeled sample pool on client k Any sample x in i , the inconsistency score between models is calculated as:
[0094]
[0095] in Indicates that the Earth Mover's distance is used to measure the sample x i In local model and local distillation models The difference between the data-driven prediction probability distributions on Indicates using bulldozer distance to measure sample x i In the server model f G and local distillation models The difference between the data-driven prediction probability distributions on .
[0096] For the unlabeled sample pool on client k Any sample x in i , the inconsistency score of the model itself is calculated as:
[0097]
[0098] in Indicates the use of bulldozer distance to measure the sample x in the local model The difference between data-driven prediction and learning-based prediction probability distribution.
[0099] For the unlabeled sample pool on client k Any sample x in i , whose total inconsistency score is:
[0100] I(x i )=I intra (x i )×I intra (x i ).
[0101] Step S3, federated learning training;
[0102] A total of T communication rounds of training are carried out. In each communication round t, each client (taking client k as an example) uses the server model f of the previous round sent by the server G To initialize the local model Based on the local labeled sample pool After the training is completed, each client uploads the local model parameters to the server, and the server obtains the new f according to the aggregation algorithm of federated learning. G And send it to each client.
[0103] Among them, using the server model f G To initialize the local model When , just replace the parameters directly.
[0104] Local Model About the Marker Sample Pool A sample label pair (x i ,y i ) is the cross entropy loss, which is expressed as:
[0105]
[0106] in Represents the cross entropy loss function, which is used to measure the model prediction output With the given label y i The difference in predictions.
[0107] Server Model f G The federated average algorithm FedAvg is used for aggregation, and the update method is expressed as:
[0108]
[0109] Step S4: If Q rounds of active data labeling have been performed, then the process ends; otherwise, the process proceeds to step S2;
[0110] The present invention introduces a data-driven probabilistic formula and effectively aligns the deviations between the local and server models based on the topological relationship of samples in the representation space, thereby more accurately evaluating the data distribution differences between the representation spaces of different models. In the active sample selection stage, samples with high prediction inconsistency may be samples with large amounts of information and difficult to learn, or they may be noise samples. The present invention introduces an intermediate model and calculates the prediction inconsistency between the model and the local model and the server model, and proposes a query strategy based on inter-model inconsistency to effectively filter out noise samples, while giving priority to samples that are most valuable for reducing inter-model prediction differences. In addition, the present invention also proposes a query strategy based on intra-model inconsistency by measuring the difference between the learning-based probability output and the data-driven probability output of the local model, and queries for samples that are most beneficial to improving the performance of the local model. Combined with the above-mentioned inconsistency strategy, the present invention aims to select an information-rich query sample set to improve the overall efficiency and performance of federated active learning while reducing annotation costs.
[0111] The above is only a preferred embodiment of the present invention. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principles of the present invention. These improvements and modifications should also be regarded as within the scope of protection of the present invention.
Claims
1. A federated active data labeling method based on inter-model inconsistency and intra-model inconsistency, characterized by: The following steps are involved: Step S1: Initialize the federated learning active data labeling task: Configure a federated learning active data annotation scenario that includes 1 server model, N local models, and N local distillation models. Each client maintains a pool of labeled and unlabeled samples and sets the total number of active annotation rounds Q. Step S2, active data labeling: Initial round: Each client randomly selects unlabeled samples to label and update the sample pool; Subsequent rounds: Use the server model and local labeled samples to distill and train the local distillation model; parallelize the learning-based prediction probabilities of the server / local / distillation models for unlabeled samples, and the data-driven prediction probabilities of the three models for the same samples; calculate the inter-model inconsistency score based on the data-driven probabilities; calculate the intra-model inconsistency score based on the two probabilities of the local model; fuse the scores and select the highest-scoring sample annotation to update the sample pool; Step S3, federated learning training: Perform T rounds of communication training. Each round includes: the client downloads the server model to initialize the local model; trains the local model based on the local labeled sample pool; the client uploads the local model parameters to the server, and updates the server model through the aggregation algorithm; Step S4: If Q rounds of active data labeling have been performed, then the process ends; otherwise, the process proceeds to step S2.
2. The method for federated active data labeling based on inter-model inconsistency and intra-model inconsistency according to claim 1, characterized in that: In step S1, The federated learning active data annotation setting scenario includes 1 server model f G , N local models and N local distillation models Each local client k maintains a labeled sample pool and an unlabeled sample pool and have one or more pools of unlabeled samples The annotator provides labels for samples in the dataset; the number of active data annotation rounds is set to Q rounds.
3. The method for federated active data labeling based on inter-model inconsistency and intra-model inconsistency according to claim 2, characterized in that: In step S1, Marked sample pool on client k It satisfies: x i Represents the input feature corresponding to the i-th sample, y i ∈Y k ={1,…,C} represents the sample label corresponding to the i-th sample, Y k Representation dataset The corresponding label set, C is the label set Y k The number of categories contained in Indicates the labeled sample pool The number of samples included in ; Unlabeled sample pool on client k It satisfies: x i represents the input feature corresponding to the i-th sample, Indicates the unlabeled sample pool The number of samples included in ; 1 server model f G , N local models and N local distillation models All are C classification models, among which the server model f G and N local models The structure is identical, G refers to the server model f G The parameter θ k Refers to the local model on client k Parameters, Refers to the local distillation model on client k Parameters.
4. The federated active data labeling method based on inter-model inconsistency and intra-model inconsistency according to claim 1 is characterized by: In step S2, If it is the initial round, then k random unlabeled samples are collected from the pool on each client. Select a batch of samples and give them to local annotators for annotation, and add them to the local labeled sample pool accordingly The sample pool has never been marked Removed; Otherwise, perform the following steps: (a) First, at each client k, use the server model f G and a local pool of labeled samples Local distillation model using knowledge distillation Conduct training; (b) After training, the server model f is used on each client. G , local model and local distillation models Unlabeled sample pool The samples in are evaluated to obtain the learning-based prediction probability and data-driven prediction probability of each sample; (c) Using the data-driven prediction probabilities obtained by the sample from the three models, the corresponding inter-model inconsistency scores are calculated; using the learning-based prediction probabilities and data-driven prediction probabilities obtained by the sample from the local model, the corresponding intra-model inconsistency scores are calculated; (d) Finally, the two inconsistency scores of the samples are fused, and the samples with the highest total inconsistency scores are queried and handed over to the local annotator for annotation, and the corresponding samples are added to the local labeled sample pool. The sample pool has never been marked Removed.
5. The method for federated active data labeling based on inter-model inconsistency and intra-model inconsistency according to claim 4, characterized in that: In step S2, Local distillation model About the Marker Pool A sample label pair (x i ,y i )’s distillation training loss is expressed as: in represents the specific loss function, Is the first loss term used to measure the model prediction output With the given label y i The difference in previous predictions, The second loss term is used to measure the model prediction output And the server model predicts the output f G (x i ) between the predicted differences; For any sample x i , the learning-based prediction probability of any model f on label v mentioned in the above steps is defined as: Where f(v|x) represents the model f for sample x i The corresponding output on label v; For the unlabeled sample pool on client k Any sample x in i , the prediction probability of model f based on data driving is calculated as follows: (1) Using model f G Unlabeled sample pool Predict each sample in the dataset, calculate the prediction probability based on learning, and regard the label with the highest prediction probability as a pseudo label to obtain the updated unlabeled sample pool in Indicates the pseudo label corresponding to the i-th sample; (2) Use model f to extract the labeled sample pool and unlabeled sample pool The features of each sample in the pool are updated to obtain the labeled sample pool and unlabeled sample pool where z i Indicates the feature corresponding to the i-th sample; (3) Based on the extracted features, for each sample from the labeled sample pool and unlabeled sample pool The samples in the pool are calculated separately for them and the unlabeled samples The feature similarity between all samples in and the K samples with the highest feature similarity are obtained; (4) For any sample x i , the data-driven prediction probability of any model f mentioned in the above steps on the label v is defined as: in Represents all labeled samples from the labeled sample pool and unlabeled sample pool The sample in the sample x i is the total number of K samples with the highest feature similarity; For the unlabeled sample pool on client k Any sample x in i , the inconsistency score between models is calculated as: in Indicates the specific method used to calculate sample x i In local model and local distillation models A data-driven measure of the difference between the predicted probability distributions on , Indicates the specific method used to calculate sample x i In the server model f G and local distillation models A data-driven measure of the difference between the predicted probability distributions on ; For the unlabeled sample pool on client k Any sample x in i , the inconsistency score of the model itself is calculated as: in Indicates the specific method used to calculate the sample x in the local model A measure of the difference between the probability distributions of data-driven predictions and learning-based predictions on For the unlabeled sample pool on client k Any sample x in i , whose total inconsistency score is: I(x i )=I intra (x i )×I inter (x i )。 6. The method for federated active data labeling based on inter-model inconsistency and intra-model inconsistency according to claim 1, characterized in that: In step S3, A total of T communication rounds of training are carried out. In each communication round t, each client k uses the server model f of the previous round sent by the server G To initialize the local model Based on the local labeled sample pool Perform model training; after training is completed, each client uploads the local model parameters to the server, and the server obtains the new f according to the aggregation algorithm of federated learning G And send it to each client.
7. The method for federated active data labeling based on inter-model inconsistency and intra-model inconsistency according to claim 6, characterized in that: In step S3, Utilize server model f G To initialize the local model When , directly replace the parameters; Local Model About the Marker Sample Pool A sample label pair (x i ,y i )’s training loss is expressed as: in Represents a specific loss function used to measure the model prediction output With the given label y i The difference in predictions between Server Model f G The update method using the aggregation algorithm is expressed as: in Represents a specific aggregation algorithm used to fuse the parameters of multiple client models to obtain the server model parameters.