Adaptive federated learning method based on differential privacy

Abstract

The present invention discloses an adaptive federated learning method based on differential privacy, which belongs to the field of security technology. The method comprises the following steps: initializing the model through the client, training the model using the cross-entropy loss function, and enhancing learning with unlabeled data; sending the local model update to the central server, and using the global model to guide the local model training of each client; the server intelligently selects the clients to participate in the next round of aggregation; the client and the central server collaborate to implement a dynamic model evaluation mechanism and adjust the strategy in a timely manner; the updated global model is sent to each client for the next round of local model training. This method can not only dynamically adjust the learning strategy according to the changes in data and the performance of the model, but also improve the generalization ability of the model through enhanced semantic distillation and multi-task learning strategies, effectively reducing communication costs, and is particularly effective in application scenarios where data is sensitive and unevenly distributed.

Description

Technical Field

[0001] The present invention relates to an adaptive federated learning method based on differential privacy, and belongs to the field of security technology. Background Art

[0002] In recent years, federated learning, a distributed machine learning method, has been widely used in various data-sensitive fields due to its ability to protect data privacy and accommodate the participation of multiple nodes. By allowing each node to complete model training locally and only transmitting model updates to a central server for aggregation, federated learning can reduce the risk of data leakage to a certain extent. However, existing federated learning methods still face some urgent challenges in their application.

[0003] First, the data distribution of participating nodes is often heterogeneous, meaning that the data features of each node are not independent and identically distributed. This data heterogeneity causes the model to adapt to new data while its memory for old data declines, leading to a phenomenon called "catastrophic forgetting" that affects the model's generalization ability. Second, traditional federated learning methods cannot fully protect data privacy during model updates. Especially when the data contains sensitive information, model updates may leak private information of nodes, posing a high security risk. These issues limit the promotion and application of federated learning in application scenarios with higher security and privacy requirements. Summary of the Invention

[0004] The purpose of this invention is to provide an adaptive federated learning method based on differential privacy, which aims to solve the technical problem that it is difficult to effectively solve the catastrophic forgetting problem caused by non-independent and identically distributed data while emphasizing privacy protection.

[0005] To achieve the above objectives, the present invention provides an adaptive federated learning method based on differential privacy. This technical solution adopts model distillation, intelligent client selection and weight update strategy, and applies differential privacy and secure multi-party computing technology. It not only ensures data privacy and security during the model update process, but also effectively alleviates the catastrophic forgetting problem in federated learning, providing a solution for the data sharing field that protects privacy and improves data utilization efficiency.

[0006] Specifically, this technical solution, after performing traditional federated learning data preprocessing and local model initialization pretraining, employs an enhanced semantic distillation method to update the global model after each round of aggregation. During this process, the global model not only guides each local model in learning global generalized knowledge, but also simultaneously learns related self-supervised learning tasks through multi-task learning, helping the local model to more comprehensively understand global knowledge. Furthermore, a meta-learning strategy is introduced to optimize the learning process of the local model, enabling it to quickly adapt to the knowledge learned from the global model.

[0007] This solution uses a client-selected weight update algorithm to comprehensively evaluate weights based on client data quality, diversity, and historical contributions, intelligently selecting clients for the next round of aggregation, thereby enhancing the model's generalization capabilities. Finally, the present invention applies differential privacy and secure multi-party computation technologies to ensure data privacy and security during the model update process. While ensuring privacy and security, this technical solution effectively mitigates the catastrophic forgetting problem encountered in federated learning by enhancing semantic distillation and optimizing client participation strategies, promoting the secure and efficient development of data-driven applications.

[0008] Specifically, the adaptive federated learning method based on differential privacy includes the following steps:

[0009] S10: Set N clients, each client collects labeled data sets and unlabeled data sets, and then the client performs data preprocessing operations and initializes the local model;

[0010] S20: The client uses the labeled dataset to train the model and uses the cross-entropy function to optimize the learning process. For the unlabeled dataset, pseudo labels are generated through self-supervised learning. The labeled dataset is then combined with the generated pseudo labels to further optimize the model parameters and perform local model updates.

[0011] S30: The client sends the local model update to the central server, which uses the global model to guide the local model training of each client through enhanced semantic distillation methods, multi-task learning and meta-learning strategies;

[0012] S40: The central server evaluates the model updates uploaded by each client, calculates the aggregation weight based on the client data quality, diversity, and historical contribution, performs a new round of global model updates, and dynamically adjusts the set of clients participating in the next round of aggregation to enhance the generalization ability of the global model.

[0013] S50: During global model update upload and global model download, a differential privacy mechanism is used to perturb sensitive parameters. Secure multi-party computing technology is also used to perform secure aggregation on the central server. This ensures data security during model updates and the security of the model during transmission between the client and server.

[0014] S60: The client and central server collaborate to evaluate model performance, monitor the model's learning progress in real time, and adjust the meta-learning strategy based on the evaluation results.

[0015] S70: Send the updated global model to each client for the next round of local model training until the predetermined performance index or learning rounds are reached and the local model training is completed.

[0016] The S10 specifically includes:

[0017] Set client set C = {C1, C2, ..., C N}, where N represents the total number of clients, C i Represents the i-th client;

[0018] For each client C i , prepare private dataset in represents a labeled dataset, represents an unlabeled dataset;

[0019] Initialize the local model

[0020] The S20 specifically includes:

[0021] For the labeled dataset, define the cross entropy loss function L(θ i ) for the client to learn from labeled data:

[0022]

[0023] Among them, P(y j |x;θ i ) represents the probability that the model of client i predicts the label category j for the input sample x, θ i is the model parameter, k is the total number of categories, y j is the label predicted as category j for the input sample x;

[0024] For unlabeled datasets, self-supervised learning tasks are designed to extract features and enhance the learning ability of the model. The loss function L aux (θ i ) is expressed as:

[0025]

[0026] Among them, f(x; θ i ) represents the loss function of self-supervised learning designed for input sample x;

[0027] Total loss L MTL (θ i ) is the weighted sum of the cross entropy loss and the self-supervised learning task loss:

[0028]

[0029] Among them, λ t represents the weight of the t-th self-supervised learning task, T represents the number of self-supervised learning tasks, Represents the loss function for the t-th self-supervised learning task.

[0030] The S30 specifically includes:

[0031] Perform model aggregation update and define the distillation loss function L distill (θ i ) is used by the client to learn from the global model:

[0032]

[0033] Among them, Q(y j |x;θ g ) represents the global model θ g The predicted probability of category j for the input sample x;

[0034] Use the local model to learn the output of the global model, and learn other tasks at the same time, using a multi-task learning strategy, assuming that there are other task losses L task (θ k ), then the total loss function L that integrates multi-task learning total (θ k )for:

[0035] L total (θ k )=L distill (θ k )+λL task (θ k )

[0036] Where λ is the task loss weight coefficient;

[0037] Use meta-learning strategies to dynamically adjust the learning rate and the weight coefficient λ in the loss function k As well as the priority between learning tasks, define a meta-goal L meta (Φ), verify the effectiveness of learning by the performance on the cross-validation set:

[0038]

[0039] Among them, Φ is the parameter of meta-learning, Φ * Is to make the meta-goal L meta (Φ) parameter to be minimized;

[0040] Based on the above total loss function L total (θ k ) and meta-learning strategy to update each client’s local model parameters θ k , the model parameters of each new round of local models Where η represents the learning rate, Represents the total loss function for parameter θ kgradient.

[0041] The S40 specifically includes:

[0042] Each client C i Based on data quality Q i 、Diversity V i and historical contribution P i The comprehensive score generates a weight calculation function:

[0043] W i =αQ i +βV i +γP i

[0044] Among them, W i is the weight of the i-th client, α, β and γ are preset coefficients used to adjust the data quality Q i 、Diversity V i and historical contribution P i Relative importance in weight calculations;

[0045] After obtaining the weight of each client, the model updates provided by all clients are aggregated to obtain the global model M g The update, the aggregation is weighted average, the new round of global model for:

[0046]

[0047] in, is the model parameter of the i-th client after the current iteration, and N is the total number of clients participating in the current round of updates.

[0048] The S50 specifically includes:

[0049] Adding random noise to the model parameters, under differential privacy, the model update of client i is expressed as:

[0050]

[0051] in, represents the model parameters after applying differential privacy, η i is the noise added according to differential privacy;

[0052] For secure multi-party computation, let f be the aggregation function and θg be the global model parameter, expressed as:

[0053]

[0054] in, is the set of model parameters after differential privacy is applied by all participating clients, Represents the model parameters after differential privacy is applied to the Nth client.

[0055] The S60 specifically includes:

[0056] Calculate C for each client i The accuracy of the model on the local validation set Acc i :

[0057]

[0058] Among them, |V i | is the number of samples in the local validation set, (x, y) represents a sample pair, P(y|x; θ i ) represents the model parameter θ i Next, the probability of predicting label y for input sample x;

[0059] The central server collects the performance indicators of each server and calculates the global accuracy Acc global :

[0060]

[0061] Where N is the total number of clients participating in the current round of updates;

[0062] Set lower as the lower limit of performance improvement and upper as the upper limit of performance improvement to obtain the learning rate η new The adjustment strategy is:

[0063]

[0064] The updated learning rate η new Sent to all clients for the next round of training.

[0065] The S70 specifically includes:

[0066] After completing a new round of global models After aggregation, the central server distributes the global model to all clients as the starting model for the next round of training. After receiving the global model, the client uses the global model as the initial parameter for the next round of training. The above process is repeated until the predetermined performance index or learning round is reached, and the local model training is completed.

[0067] Compared to existing technologies, the present invention offers the following advantages: It proposes an adaptive federated learning method based on differential privacy. This method cleverly combines differential privacy with secure multi-party computation (SMC) within a federated learning approach, ensuring data security and privacy during model training. By introducing differential privacy, clients can upload model updates while protecting their own data privacy. SMC ensures that individual client inputs are not leaked to other participants during the aggregation process, thereby improving data security. Furthermore, this technology dynamically adjusts learning strategies based on data changes and model performance. By monitoring the model's learning progress and effectiveness in real time, it intelligently adjusts the model's learning rate and parameters, thereby improving learning efficiency and adaptability. Furthermore, through enhanced semantic distillation and multi-task learning methods, this technology enables the model to effectively learn from the global model and extract a wider range of data features, helping to address uneven data distribution. This approach not only improves the model's generalization ability but also enhances the recognition efficiency of new categories and complex patterns. These features give this technology significant advantages in the field of federated learning, making it particularly suitable for distributed learning environments where data is sensitive and privacy is crucial. BRIEF DESCRIPTION OF THE DRAWINGS

[0068] Figure 1 It is a schematic diagram of the process of the present invention;

[0069] Figure 2 Schematic diagram of the hardware operating environment of the present invention. DETAILED DESCRIPTION

[0070] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of the present invention.

[0071] Example 1: Figure 1 As shown in Figure 2, the adaptive federated learning method based on differential privacy includes:

[0072] S10: Set N clients, each client collects labeled data sets and unlabeled data sets, and then the client performs data preprocessing operations and initializes the local model;

[0073] Specifically, the client's dataset is first divided into labeled and unlabeled data. Labeled data is used for supervised learning tasks, while unlabeled data is used for self-supervised learning or other self-supervised learning tasks to enhance the model's adaptability to unlabeled data. After data preprocessing is complete, the client's local model is initialized and uses the preprocessed dataset as input.

[0074] Furthermore, the data preprocessing process can be expressed as the following steps:

[0075] Divide the dataset into labeled dataset and unlabeled dataset;

[0076] Normalize each data set to keep the feature values ​​within a uniform range;

[0077] Format the data to fit the model input structure to ensure compatibility between the data and the model.

[0078] After preprocessing, each client initializes a local model and uses the preprocessed data as initial input for local training in subsequent steps.

[0079] S20: The client uses the labeled dataset to train the model and uses the cross-entropy function to optimize the learning process. For the unlabeled dataset, pseudo labels are generated through self-supervised learning. The labeled dataset is then combined with the generated pseudo labels to further optimize the model parameters and perform local model updates.

[0080] Specifically, each client trains the model locally, uses the cross-entropy function to optimize the learning process for labeled data, and uses unlabeled data for self-supervised learning tasks to enhance the model's learning ability.

[0081] In this embodiment, the local training method is determined based on whether the data is labeled;

[0082] Specifically, for the labeled dataset, the cross entropy loss function L(θ i ) for the client to learn from labeled data:

[0083]

[0084] Among them, P(y j |x;θ i ) represents the probability that the model of client i predicts the label category j for the input sample x, θ i is the model parameter, k is the total number of categories, y j is the label predicted as category j for the input sample x;

[0085] For unlabeled datasets, self-supervised learning tasks are designed to extract features and enhance the learning ability of the model. The loss function Laux (θ i ) is expressed as:

[0086]

[0087] Among them, f(x; θ i ) represents the loss function of self-supervised learning designed for input sample x;

[0088] Total loss L MTL (θ i 0 is the weighted sum of the cross entropy loss and the self-supervised learning task loss:

[0089]

[0090] Among them, λ t represents the weight of the t-th self-supervised learning task, T represents the number of self-supervised learning tasks, Represents the loss function for the t-th self-supervised learning task.

[0091] S30: The client sends the local model update to the central server, which uses the global model to guide the local model training of each client through enhanced semantic distillation methods, multi-task learning and meta-learning strategies;

[0092] Specifically, the client sends locally trained model updates to the central server, which aggregates and guides these updates to generate a global model.

[0093] In the technical solution provided in this embodiment, the client sends the model update after local training to the central server, and the update includes the feature information and model improvements learned by the client on the local data. The central server receives the model updates from each client and aggregates them to integrate the data features of different clients to generate a model representing the global data distribution. The aggregation process is based on weighted average or other algorithms, so that the updates of each client are reasonably integrated in the global model. The generated global model contains the data knowledge of all clients, so it has a wider generalization ability and higher robustness. Subsequently, the central server guides the client model with the global model, so that the client can more effectively improve its learning performance in the next round of training, and gradually achieve adaptation to complex data distribution and enhancement of generalization ability. The S30 includes:

[0094] Perform model aggregation update and define the distillation loss function L distill (θ i ) is used by the client to learn from the global model:

[0095]

[0096] Among them, Q(y j |x;θ g ) represents the global model θ g The predicted probability of category j for the input sample x;

[0097] Use the local model to learn the output of the global model, and learn other tasks at the same time, using a multi-task learning strategy, assuming that there are other task losses L task (θ k ), then the total loss function L that integrates multi-task learning total (θ k )for:

[0098] L total (θ k )=L distill (θ k )+λL task (θ k )

[0099] Where λ is the task loss weight coefficient;

[0100] Use meta-learning strategies to dynamically adjust the learning rate and the weight coefficient λ in the loss function k As well as the priority between learning tasks, define a meta-goal L meta (Φ), verify the effectiveness of learning by the performance on the cross-validation set:

[0101]

[0102] Among them, Φ is the parameter of meta-learning, Φ * Is to make the meta-goal L meta (Φ) parameter to be minimized;

[0103] Based on the above total loss function L total (θ k ) and meta-learning strategy to update each client’s local model parameters θ k , the model parameters of each new round of local models Where η represents the learning rate, Represents the total loss function for parameter θ k gradient.

[0104] S40: The central server evaluates the model updates uploaded by each client, calculates the aggregation weight based on the client data quality, diversity, and historical contribution, performs a new round of global model updates, and dynamically adjusts the set of clients participating in the next round of aggregation.

[0105] Specifically, each client C i Based on data quality Q i 、Diversity Vi and historical contribution P i The comprehensive score generates a weight calculation function:

[0106] W i =αQ i +βV i +γP i

[0107] Among them, W i is the weight of the i-th client, α, β and γ are preset coefficients used to adjust the data quality Q i 、Diversity V i and historical contribution P i Relative importance in weight calculations;

[0108] After obtaining the weight of each client, the model updates provided by all clients are aggregated to obtain the global model M g The update, the aggregation is weighted average, the new round of global model for:

[0109]

[0110] in, is the model parameter of the i-th client after the current iteration, and N is the total number of clients participating in the current round of updates.

[0111] S50: During the global model update upload and global model download process, a differential privacy mechanism is used to perturb sensitive parameters, and secure multi-party computing technology is combined to perform secure aggregation on the central server.

[0112] Specifically, during the entire federated learning training process, differential privacy technology and secure multi-party computing technology are introduced to not only protect data when model parameters are updated, but also ensure the security of the model during transmission between the client and the server, thereby effectively preventing data leakage. The detailed description of S50 is as follows:

[0113] Adding random noise to the model parameters, under differential privacy, the model update of client i is expressed as:

[0114]

[0115] in, represents the model parameters after applying differential privacy, η iThis is noise added based on differential privacy. The addition of noise masks the contribution of individual data points in the model parameters. Even if the updated model parameters are received by the server, it is difficult to infer the original data from them, thereby protecting user privacy. The degree of noise added can be flexibly adjusted based on the differential privacy budget to achieve a balance between privacy protection and model accuracy.

[0116] For secure multi-party computation, let f be the aggregation function and θg be the global model parameter, expressed as:

[0117]

[0118] in, is the set of model parameters after differential privacy is applied by all participating clients, represents the model parameters of the Nth client after differential privacy is applied. When using secure multi-party computation for aggregation, the server does not need to decrypt the parameters of each client, but instead performs the aggregate calculation directly on the encrypted parameters. Secure multi-party computation ensures that even if the central server or communication channel is attacked, the original model parameters of a single client cannot be obtained.

[0119] S60: The client and central server collaborate to evaluate model performance, monitor the model's learning progress in real time, and adjust the meta-learning strategy based on the evaluation results.

[0120] Specifically, the client and the central server collaborate to implement a dynamic model evaluation mechanism, monitor learning progress, and adjust strategies in a timely manner. The S60 is specifically as follows:

[0121] Calculate C for each client i The accuracy of the model on the local validation set Acc i :

[0122]

[0123] Among them, |V i | is the number of samples in the local validation set, (x, y) represents a sample pair, P(y|x; θ i ) represents the model parameter θ i Next, the probability of predicting label y for input sample x;

[0124] The central server collects the performance indicators of each server and calculates the global accuracy Acc global :

[0125]

[0126] Where N is the total number of clients participating in the current round of updates;

[0127] Set lower as the lower limit of performance improvement and upper as the upper limit of performance improvement to obtain the learning rate η new The adjustment strategy is:

[0128]

[0129] The updated learning rate η new Sent to all clients for the next round of training.

[0130] S70: Send the updated global model to each client for the next round of local model training until the predetermined performance index or learning rounds are reached and the local model training is completed;

[0131] Specifically, after completing a new round of global models After aggregation, the central server distributes the global model to all clients as the starting model for the next round of training. After receiving the global model, the client uses the global model as the initial parameter for the next round of training. The above process is repeated until the predetermined performance index or learning round is reached, and the local model training is completed.

[0132] Example 2: As another embodiment, Figure 2 This is a schematic diagram of the architecture of the hardware operating environment of the adaptive federated learning method based on differential privacy involved in Example 2 of the present invention.

[0133] like Figure 2 As shown, the federated learning system may include: a processor 1001, such as a CPU, a memory 1005, a user interface 1003, a network interface 1004, and a communication bus 1002. The communication bus 1002 is used to implement connection and communication between components. The user interface 1003 may include a display screen and an input unit, such as a keyboard. Optionally, the user interface 1003 may also include a standard wired interface and a wireless interface. The network interface 1004 may optionally include a standard wired interface and a wireless interface (such as a WI-FI interface). The memory 1005 may be a high-speed RAM memory or a stable memory, such as a disk memory. The memory 1005 may also be a storage device independent of the aforementioned processor 1001.

[0134] Those skilled in the art will understand that Figure 2 The federated learning system architecture shown in does not constitute a limitation on the machine learning system and may include more or fewer components than shown in the figure, or combine certain components, or arrange the components differently.

[0135] like Figure 2As shown, the memory 1005 as a storage medium may include an operating system, a network communication module, a user interface module, and an adaptive federated training program based on differential privacy. The operating system is a program that manages and controls the hardware and software resources of the machine learning system, and the federated training program and other software or programs run.

[0136] exist Figure 2 In the terminal shown, the user interface 1003 is mainly used to connect to the terminal and communicate data with the terminal; the network interface 1004 is mainly used for the background server and communicates data with the background server; the processor 1001 can be used to call the adaptive federated learning training program based on differential privacy stored in the memory 1005.

[0137] Based on the above hardware architecture, the adaptive federated learning method based on differential privacy in Example 2 includes the following steps:

[0138] S10: When the client receives the global model parameters sent by the server, the client preprocesses the local data and performs local model training.

[0139] The client first cleans and normalizes the local data to remove noise and outliers to ensure data quality. The data is then divided into labeled and unlabeled data to support different learning tasks. After preprocessing, the client trains the model on the local dataset and generates updated model parameters.

[0140] In this embodiment, communication between the client and the server can be achieved through different terminal devices, such as smartphones, personal computers, tablet computers, or servers. Global model parameters are aggregated and generated by the server and transmitted to each client. The client trains based on its local data to generate model updates that adapt to the local data distribution.

[0141] Step S20: After local training is completed, the client performs differential privacy processing on the model update parameters and transmits the updates to the server through secure multi-party computing technology.

[0142] First, the client adds differential privacy noise to the model update parameters to ensure data privacy is not leaked. Then, the client generates a random vector with the same dimension as the model update parameters to blur the model parameters, thereby further protecting privacy during transmission.

[0143] The client applies secure multi-party computation (SMC) technology to the generated differentially private parameters, encrypts them, and sends the obfuscated model update parameters to the server. Upon receiving the encrypted update parameters, the server aggregates them using a secure multi-party computation protocol without decrypting the client's data updates, ensuring data security during transmission.

[0144] Step S30: The server receives the encryption model update parameters from the client and performs an aggregation operation.

[0145] On the server side, a weighted approach is used to aggregate the updated parameters of all clients to generate a global model. The server also adjusts the learning strategy, including the learning rate and client selection strategy, in real time based on client feedback data (such as the loss value on the validation set) to ensure the generalization and stability of the global model.

[0146] During this process, the server determines whether the model to be trained has reached a convergence state based on the difference judgment method. Specifically:

[0147] Step S31: The server obtains the loss value of the previous global model and records the loss value after this aggregation as the current loss value.

[0148] Step S32: Calculate the difference between the previous loss value and the current loss value, and determine whether the difference is less than or equal to a preset convergence threshold.

[0149] Step S33: When the difference is less than or equal to the preset threshold, the server determines that the model has reached a converged state; otherwise, if the difference is greater than the threshold, it is determined that the model has not converged and continues with the next round of training and updating.

[0150] Through the above steps, Example 2 of the present invention uses differential privacy and secure multi-party computing technology to achieve data privacy protection and efficient aggregation updates in federated learning, ensuring the security and accuracy of model training.

[0151] While embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions, and alterations may be made to these embodiments without departing from the principles of the invention, the scope of which is defined by the appended claims and their equivalents.

Claims

1. Adaptive federated learning method based on differential privacy, characterized by: The following steps are involved: S10: Set N clients, each client collects labeled data sets and unlabeled data sets, and then the client performs data preprocessing operations and initializes the local model; S20: The client uses the labeled dataset to train the model and uses the cross entropy function to optimize the learning process; For unlabeled datasets, pseudo labels are generated through self-supervised learning methods. Then, the labeled dataset is combined with the generated pseudo labels to further optimize the model parameters and perform local model updates. S30: The client sends the local model update to the central server, which uses the global model to guide the local model training of each client through enhanced semantic distillation methods, multi-task learning and meta-learning strategies; S40: The central server evaluates the model updates uploaded by each client, calculates the aggregation weight based on the client data quality, diversity, and historical contribution, performs a new round of global model updates, and dynamically adjusts the set of clients participating in the next round of aggregation. S50: During the global model update upload and global model download process, a differential privacy mechanism is used to perturb sensitive parameters, and secure multi-party computing technology is combined to perform secure aggregation on the central server. S60: The client and central server collaborate to evaluate model performance, monitor the model's learning progress in real time, and adjust the meta-learning strategy based on the evaluation results. S70: Send the updated global model to each client for the next round of local model training until the predetermined performance index or learning rounds are reached and the local model training is completed; The S30 specifically includes: Perform model aggregation update and define the distillation loss function L distill (θ i ) is used by the client to learn from the global model: Among them, Q(y j |x;θ g ) represents the global model θ g The predicted probability of category j for the input sample x; Use the local model to learn the output of the global model, and learn other tasks at the same time, using a multi-task learning strategy, assuming that there are other task losses L task (θ k ), then the total loss function L that integrates multi-task learning total (θ k )for: L total (i k )=L distill (i k )+λL task (i k ) Where λ is the task loss weight coefficient; Use meta-learning strategies to dynamically adjust the learning rate and the weight coefficient λ in the loss function k As well as the priority between learning tasks, define a meta-goal L meta (Φ), verify the effectiveness of learning by the performance on the cross-validation set: Among them, Φ is the parameter of meta-learning, Φ * Is to make the meta-goal L meta (Φ) parameter to be minimized; Based on the above total loss function L total (θ k ) and meta-learning strategy to update each client’s local model parameters θ k , the model parameters of each new round of local models Where η represents the learning rate, Represents the total loss function for parameter θ k gradient.

2. The adaptive federated learning method based on differential privacy according to claim 1, characterized in that The S10 specifically includes: Set client set C = {C1, C2, ..., C N }, where N represents the total number of clients, C i Represents the i-th client; For each client C i , prepare private dataset in represents a labeled dataset, represents an unlabeled dataset; Initialize the local model 3. The adaptive federated learning method based on differential privacy according to claim 1, characterized in that The S20 specifically includes: For the labeled dataset, define the cross entropy loss function L(θ i ) for the client to learn from labeled data: Among them, P(y j |x;θ i ) represents the probability that the model of client i predicts the label category j for the input sample x, θ i is the model parameter, k is the total number of categories, y j is the label predicted as category j for the input sample x; For unlabeled datasets, self-supervised learning tasks are designed to extract features and enhance the learning ability of the model. The loss function L aux (θ i ) is expressed as: Among them, f(x; θ i ) represents the loss function of self-supervised learning designed for input sample x; Total loss L MTL (θ i ) is the weighted sum of the cross entropy loss and the self-supervised learning task loss: Among them, λ t represents the weight of the t-th self-supervised learning task, T represents the number of self-supervised learning tasks, Represents the loss function for the t-th self-supervised learning task.

4. The adaptive federated learning method based on differential privacy according to claim 1, characterized in that The S40 specifically includes: Each client C i Based on data quality Q i 、Diversity V i and historical contribution P i The comprehensive score generates a weight calculation function: W i =αQ i +βV i +γP i Among them, W i is the weight of the i-th client, α, β and γ are preset coefficients used to adjust the data quality Q i 、Diversity V i and historical contribution P i Relative importance in the weight calculation; After obtaining the weight of each client, the model updates provided by all clients are aggregated to obtain the global model M g The update, the aggregation is weighted average, the new round of global model for: in, is the model parameter of the i-th client after the current iteration, and N is the total number of clients participating in the current round of updates.

5. The adaptive federated learning method based on differential privacy according to claim 1, characterized in that The S50 specifically includes: Adding random noise to the model parameters, under differential privacy, the model update of client i is expressed as: in, represents the model parameters after differential privacy is applied, η i is the noise added according to differential privacy; For secure multi-party computation, let f be the aggregation function and θg be the global model parameter, expressed as: in, is the set of model parameters after differential privacy is applied by all participating clients, Represents the model parameters after differential privacy is applied to the Nth client.

6. The adaptive federated learning method based on differential privacy according to claim 1, characterized in that The S60 specifically includes: Calculate C for each client i The accuracy of the model on the local validation set Acc i : Among them, |V i | is the number of samples in the local validation set, (x, y) represents a sample pair, P(y|x; θ i ) represents the model parameter θ i Next, the probability of predicting label y for input sample x; The central server collects the performance indicators of each server and calculates the global accuracy Acc global : Where N is the total number of clients participating in the current round of updates; Set lower as the lower limit of performance improvement and upper as the upper limit of performance improvement to obtain the learning rate η new The adjustment strategy is: The updated learning rate η new Sent to all clients for the next round of training.

7. The adaptive federated learning method based on differential privacy according to claim 1, characterized in that The S70 specifically includes: After completing a new round of global models After aggregation, the central server distributes the global model to all clients as the starting model for the next round of training. After receiving the global model, the client uses the global model as the initial parameter for the next round of training. The above process is repeated until the predetermined performance index or learning round is reached, and the local model training is completed.

Images