Asynchronous federation model training method and device based on delay gradient prediction

Abstract

The invention discloses an asynchronous federal model training method and device based on delay gradient prediction, and the method comprises the steps: distributing a (t-1) th round of ciphertext global model to a plurality of clients, thereby enabling the clients to train the (t-1) th round of global model according to a local data set, and obtaining the model gradient information of a current round; receiving ciphertext model gradient information of the current round sent by the clients, determining an online client and a delay client in each client, and predicting delay model gradient information of the current round for the delay client; respectively determining an online compensation factor of the online client and a delay compensation factor of the delay client, and based on the online compensation factor and the delay compensation factor, performing aggregation processing on the online model gradient information of the online client and the delay model gradient information of the delay client to obtain aggregation gradient information; and carrying out gradient updating on the global model of the (t-1) th round based on the aggregation gradient information to obtain the global model of the current round.

Description

Technical Field

[0001] The present invention relates to the field of model training technology, and in particular to an asynchronous federated model training method and device based on delayed gradient prediction. Background Art

[0002] Federated Learning (FL), a distributed machine learning paradigm, allows multiple clients to collaboratively train a global model without uploading their local data, thereby protecting the privacy of each client's local data. FL has been widely used in healthcare, finance, the Internet of Things, and other fields.

[0003] Currently, traditional federated learning relies on client-synchronized gradient updates to train models. However, in real-world applications, client updates are often delayed due to network or computing resource limitations, which reduces model convergence speed and training accuracy. Furthermore, model training is typically based on plaintext information, which poses a risk of data leakage. Summary of the Invention

[0004] The present invention provides an asynchronous federated model training method and device based on delayed gradient prediction, which can improve the model training accuracy, increase the model convergence speed, and improve the data security during the model training process.

[0005] According to a first aspect of the present invention, there is provided an asynchronous federated model training method based on delayed gradient prediction, which is applied to an aggregation server and includes:

[0006] The ciphertext global model of the t-1th round is distributed to multiple clients, so that the clients train the decrypted global model corresponding to the ciphertext global model of the t-1th round based on the local data set to obtain the model gradient information of the current round, wherein the ciphertext global model of the t-1th round is encrypted using a preset homomorphic encryption algorithm based on the public key PK of the client. C1 Encrypting the global model of the t-1th round;

[0007] Receive the ciphertext model gradient information of the current round sent by the client, determine the online client and the delayed client in each of the clients based on the client identifier to which the ciphertext model gradient information of the current round belongs, and predict the delayed model gradient information of the current round for the delayed client based on the model gradient information of the global model of the t-1th round and the model gradient information of the global model of the t-2th round, wherein the ciphertext model gradient information is obtained by encrypting the model gradient information of the current round by the corresponding client using a preset homomorphic encryption algorithm;

[0008] An online compensation factor of the online client and a delay compensation factor of the delayed client are respectively determined; based on the online compensation factor and the delay compensation factor, decrypted online model gradient information corresponding to the online ciphertext model gradient information of the online client and the delayed model gradient information of the delayed client are aggregated to obtain aggregated gradient information; and a gradient update is performed on the global model of the t-1th round based on the aggregated gradient information to obtain the global model of the current round.

[0009] Optionally, predicting the delayed model gradient information of the current round for the delayed client based on the model gradient information of the global model of the t-1th round and the model gradient information of the global model of the t-2th round includes:

[0010] Based on the model gradient information w of the global model in the t-1th round (t-1) And the model gradient information w of the global model in the t-2th round (t-2) , calculate the global model change Δw (t) , where Δw (t) =w t-1 -w t-2 ;

[0011] Model gradient information w based on the global model in round t-1 (t-1) And the model gradient information w of the global model in the t-2th round (t-2) , calculate the model evolution rate factor k t ,in,

[0012] Based on the global model change Δw (t) and the model evolution rate factor k t , determine the delay model gradient information of the delay client in the current round in,

[0013] Optionally, respectively determining the online compensation factor of the online client and the delay compensation factor of the delayed client includes:

[0014] Determine the online delay round number τ of the online client zi , based on the number of online delay rounds τ zi , determine the online compensation factor γ zi ,in, γ0 is the initial compensation coefficient, α is the attenuation factor;

[0015] Determine the delay round number τ of the delayed client yi , based on the delay round number τ yi , determine the delay compensation factor γyi ,in,

[0016] Optionally, the aggregating, based on the online compensation factor and the delay compensation factor, the decrypted online model gradient information corresponding to the online ciphertext model gradient information of the online client and the delayed model gradient information of the delayed client to obtain the aggregated gradient information includes:

[0017] Determine the total number of clients |C| of the online clients and the delayed clients, the model gradient information w of the global model of the t-1th round (t-1) , the client gradient information change between the model gradient information of the t-1th round and the model gradient information of the current round obtained when each client i performs model training

[0018] Based on the online compensation factor, the delay compensation factor, the total number of clients |C|, and the model gradient information w (t-1) , the client gradient information change Determine the aggregate gradient information w t , Among them, ||·|| is the model gradient information encrypted using the preset homomorphic encryption algorithm, C is the client set, γ i is the delay compensation factor corresponding to the delayed client i or the delay compensation factor and online compensation factor corresponding to the online client i.

[0019] According to a second aspect of the present invention, there is provided an asynchronous federated model training method based on delayed gradient prediction, which is applied to a client and includes:

[0020] Receive the ciphertext global model of the t-1th round sent by the aggregation server, wherein the ciphertext global model of the t-1th round is the aggregation server using a preset homomorphic encryption algorithm based on the public key PK of the client C Obtained by encrypting the global model of round t-1;

[0021] According to the public key PK C1 The corresponding private key SK C1 Decrypting the ciphertext global model of the t-1th round to obtain the global model of the t-1th round;

[0022] Determine a local data set, and use the local data set to train the global model of the t-1th round to obtain model gradient information of the current round;

[0023] Based on the preset homomorphic encryption algorithm, using the public key PK C2The model gradient information of the current round is encrypted to obtain the ciphertext model gradient information of the current round.

[0024] Optionally, the using the local dataset to train the global model of the t-1th round to obtain model gradient information of the current round includes:

[0025] Quantizing the model gradient information of the global model of the t-1th round using a preset quantization precision parameter to obtain the global model after the t-1th round of processing;

[0026] A data set to be trained is randomly sampled from the local data set, and the global model after the processing of the t-1th round is iteratively trained using the data set to be trained to obtain the model gradient information of the current round.

[0027] According to a third aspect of the present invention, there is provided an asynchronous federated model training device based on delayed gradient prediction, which is applied to an aggregation server and includes:

[0028] The distribution unit is used to distribute the ciphertext global model of the t-1th round to multiple clients, so that the clients can train the decrypted global model corresponding to the ciphertext global model of the t-1th round based on the local data set to obtain the model gradient information of the current round, wherein the ciphertext global model of the t-1th round is encrypted using a preset homomorphic encryption algorithm based on the public key PK of the client. C1 Encrypting the global model of the t-1th round;

[0029] a gradient prediction unit, configured to receive the ciphertext model gradient information of the current round sent by the client, determine an online client and a delayed client in each of the clients based on a client identifier to which the ciphertext model gradient information of the current round belongs, and predict the delayed model gradient information of the current round for the delayed client based on the model gradient information of the global model of the t-1th round and the model gradient information of the global model of the t-2th round, wherein the ciphertext model gradient information is obtained by encrypting the model gradient information of the current round by the corresponding client using a preset homomorphic encryption algorithm;

[0030] The gradient updating unit is configured to respectively determine an online compensation factor of the online client and a delay compensation factor of the delayed client; based on the online compensation factor and the delay compensation factor, aggregate the decrypted online model gradient information corresponding to the online ciphertext model gradient information of the online client and the delayed model gradient information of the delayed client to obtain aggregated gradient information; and perform a gradient update on the global model of the t-1th round based on the aggregated gradient information to obtain the global model of the current round.

[0031] According to a fourth aspect of the present invention, there is provided an asynchronous federated model training device based on delayed gradient prediction, which is applied to a client and includes:

[0032] The receiving unit is configured to receive the encrypted global model of the t-1th round sent by the aggregation server, wherein the encrypted global model of the t-1th round is the encrypted global model of the client using the preset homomorphic encryption algorithm by the aggregation server according to the public key PK C Obtained by encrypting the global model of round t-1;

[0033] Decryption unit, used to decrypt the public key PK C1 The corresponding private key SK C1 Decrypting the ciphertext global model of the t-1th round to obtain the global model of the t-1th round;

[0034] A training unit, configured to determine a local data set and train the global model of the t-1th round using the local data set to obtain model gradient information of the current round;

[0035] Encryption unit, used to use public key PK based on preset homomorphic encryption algorithm C2 The model gradient information of the current round is encrypted to obtain the ciphertext model gradient information of the current round.

[0036] According to a fifth aspect of the present invention, a computer-readable storage medium is provided, on which a computer program is stored, which, when executed by a processor, implements the above asynchronous federated model training method based on delayed gradient prediction.

[0037] According to a sixth aspect of the present invention, a computer device is provided, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, the above-mentioned asynchronous federated model training method based on delayed gradient prediction is implemented.

[0038] According to the present invention, a model training method and device based on asynchronous federation with delayed gradient prediction is provided. Compared with the current method of gradient update training based on the gradient of the client synchronous update, the client of the present invention performs local training based on the global model obtained in the previous round, encrypts the trained gradient information, and uploads the encrypted ciphertext gradient information to the aggregation server. The aggregation server predicts a local gradient for the current round of model update for the delayed client that did not send the gradient information in time based on the gradient of the previous round and the gradient of the previous round. At the same time, a delay compensation factor is calculated for each client participating in the current round of model training (including delayed clients and online clients) to quantify their weight parameters in this round of aggregation. Finally, the gradients of the online client and the gradients of the delayed client are aggregated based on the weight parameters, and the previous round model in the aggregation server is updated according to the aggregation result. By predicting the gradient for the delayed client, the present invention can provide the delayed client with the necessary update compensation, thereby improving the convergence efficiency and final training accuracy of the overall model, and also avoid the time wasted waiting for the gradient information of the delayed client, thereby improving the training effect of the model. On the other hand, the present invention uses a lightweight homomorphic encryption algorithm to encrypt and protect the gradients of server and client private data to prevent malicious clients from intercepting the gradients, ensuring that the model accuracy and convergence rate can be effectively maintained in an asynchronous environment while providing enhanced privacy protection for client and server data, thereby achieving an efficient and secure model training process based on federated learning. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] The drawings described herein are used to provide a further understanding of the present invention and constitute a part of this application. The exemplary embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute an improper limitation of the present invention. In the drawings:

[0040] Figure 1 A flowchart of a model training method based on asynchronous federation and delayed gradient prediction according to an embodiment of the present invention is shown;

[0041] Figure 2 A flowchart of another asynchronous federated model training method based on delayed gradient prediction provided by an embodiment of the present invention is shown;

[0042] Figure 3 A schematic diagram of the structure of a model training device based on delayed gradient prediction and asynchronous federation applied to an aggregation server provided by an embodiment of the present invention is shown;

[0043] Figure 4 A schematic diagram of the structure of a model training device based on asynchronous federation and delayed gradient prediction applied on a client, provided by an embodiment of the present invention, is shown;

[0044] Figure 5 A schematic diagram of the physical structure of a computer device provided by an embodiment of the present invention is shown. DETAILED DESCRIPTION

[0045] The present invention will be described in detail below with reference to the accompanying drawings and in combination with embodiments. It should be noted that, unless there is a conflict, the embodiments and features in the embodiments of the present application can be combined with each other.

[0046] Currently, the method of gradient update training for the model relies on the gradient updated synchronously by the client. If the client is delayed in updating due to network or computing resource limitations, the model convergence speed and training accuracy will be reduced. At the same time, model training is usually based on plaintext information, which poses the risk of data leakage.

[0047] In order to solve the above problems, an embodiment of the present invention provides an asynchronous federated model training method based on delayed gradient prediction, such as Figure 1 As shown, the method includes:

[0048] 101. The aggregation server distributes the ciphertext global model of the t-1th round to multiple clients, wherein the ciphertext global model of the t-1th round is the aggregation server using the preset homomorphic encryption algorithm based on the client's public key PK C1 It is obtained by encrypting the global model of the t-1th round.

[0049] The preset homomorphic encryption algorithm can be a SHE (Somewhat Homomorphic Encryption, lightweight) homomorphic encryption algorithm. The multiple clients include online clients and delayed clients. Online clients are clients that can interact and respond with the aggregation server in a timely manner, that is, they can promptly transmit gradient information after model training to the aggregation server. Delayed clients are clients that cannot promptly transmit gradient information after model training to the aggregation server due to network reasons. t represents the current iterative update round, t is a positive integer, t≤T, and T represents the global number of iteration rounds.

[0050] Specifically, a lightweight SHE homomorphic encryption algorithm is used to encrypt the global model of the t-1th round, so that the encrypted ciphertext global model supports operations in an encrypted state without decrypting the ciphertext. This protects data privacy while achieving computing functions, and can reduce data transmission volume and communication overhead. In this embodiment of the present invention, if this is the first round of model training, the step of the aggregation server sending the ciphertext global model to the client can be omitted. The client can directly perform model training based on local data, or randomly initialize the global model already existing on the aggregation server. The aggregation server encrypts the global model and sends it to the client for local training.

[0051] In the embodiments of the present invention, the client is the data owner in asynchronous federated model training and can be a distributed terminal device or user. The terminal can be a smartphone, tablet, laptop, desktop computer, etc. Each client has its own local dataset and performs model training locally. The client is responsible for model training on the local data, generating gradients, and uploading them to the aggregation server. The aggregation server is a central server responsible for coordinating and integrating the gradients from various clients. After receiving gradients from multiple clients, the aggregation server calculates a delay compensation factor for all clients participating in the current round of training and predicts gradient information for delayed clients. Finally, based on the gradient information sent by the online clients and the gradient information predicted for delayed clients, the global model is updated. The aggregation server then distributes the updated global model to each client for model training, repeating the above steps until the required number of model training rounds is reached.

[0052] In the embodiments of the present invention, the delayed client is also the data owner in asynchronous federated model training and can be a distributed terminal device or user. The terminal can be a smartphone, tablet, laptop, desktop computer, etc. Each delayed client has its own local dataset and performs model training locally. The delayed client is responsible for training the model on the local data and generating gradient information. However, due to factors such as poor computing power and high network latency, the delayed client may not be able to perform model training and generate gradient information on time in each round, or may not be able to send gradient information to the aggregation server on time.

[0053] 102. The client receives the encrypted global model of round t-1 sent by the aggregation server, and the client uses the public key PK C1 The corresponding private key SK C1 Decrypt the ciphertext global model of the t-1th round to obtain the global model of the t-1th round.

[0054] Specifically, when the asynchronous federated model training task is started, the key management center generates a public-private key pair for the aggregation server and client to protect the data security during the model training process, while preventing the aggregation server from inferring the client's sensitive data based on the gradient information sent by the client. Under normal circumstances, the server has no right to know the private key.

[0055] 103. Determine the local data set. The client uses the local data set to train the global model of the t-1th round to obtain the model gradient information of the current round.

[0056] Among them, the number of iterations and iterative training learning rate when the client performs iterative model training locally can be set as needed, and the embodiments of this application do not specifically limit it. Gradient information is the gradient of the client's local model relative to the loss function, which is used to reflect the optimization direction and degree of the client's local data for the model parameters. Local data is the data collected locally by the client, which can be applied to the global model for model prediction, model training and other appropriate multi-party data joint processing. Local data varies with the application scenario and business. For example, the application scenario may include finance, insurance, government affairs, medical care, e-commerce, etc., and the local data can be commodity transaction data, financial transaction data, medical health data, user feature data, relationship feature data, etc. During the local model training process, the input data of the model is usually structured or unstructured data related to the business scenario, such as transaction flow in financial scenarios, electronic medical records in medical scenarios, user behavior logs in e-commerce scenarios, etc., which need to be cleaned, labeled and format converted to finally meet the format required by the model. The output data depends on the task type. For example, the classification task outputs category labels or probability distributions, the regression task outputs continuous values ​​(such as stock price predictions), and the generation task outputs text or image content. For example, if the input data is an electronic medical record in a medical scenario, the model output data can be the patient's medication information or treatment plan; if the input data is a user behavior log, the output data can be the user behavior category.

[0057] In this embodiment of the present invention, the global model obtained in the previous round (round t-1) is used as a basis to train the client's local model with local data. This allows the local model to learn some high-quality feature representations from the global model and to more quickly adjust its own weights using the model parameters of the global model, so that the local model can quickly converge to the optimal solution. It should be noted that the performance of different clients in the system may vary, for example, different clients may have different computing power and / or communication capabilities.

[0058] 104. The client uses the public key PK based on the preset homomorphic encryption algorithm C2 The model gradient information of the current round is encrypted to obtain the ciphertext model gradient information of the current round.

[0059] Among them, the preset homomorphic encryption algorithm can be a SHE homomorphic encryption algorithm.

[0060] 105. The client sends the current round of ciphertext model gradient information to the aggregation server.

[0061] In an embodiment of the present invention, a global model is trained for round t-1 based on a dataset on a local client to obtain model gradient information for the global model in round t (the current round). This model gradient information is encrypted to obtain ciphertext model gradient information, which is then transmitted to an aggregation server. The aggregation server then determines the update direction of the global model based on the ciphertext model gradient information and updates the parameters of the global model. By transmitting gradient information to reflect the local training status, sensitive data can be avoided from being transmitted to the aggregation server, protecting user data privacy while also conserving communication resources between the client and the aggregation server. Furthermore, by employing the SHE homomorphic encryption algorithm for encryption, privacy in the gradient information can be protected, enabling secure information sharing and privacy-preserving joint learning.

[0062] 106. The aggregation server receives the ciphertext model gradient information of the current round sent by the client, determines the online client and the delayed client in each client based on the client identifier to which the ciphertext model gradient information of the current round belongs, and predicts the delayed model gradient information of the current round for the delayed client based on the model gradient information of the global model of the t-1th round and the model gradient information of the global model of the t-2th round.

[0063] Among them, the local training process of the model of the delayed client is exactly the same as that of the online client. However, due to limited computing resources or network, the delayed client cannot upload its local gradient information to the aggregation server during this round of model aggregation.

[0064] In the embodiment of the present invention, the client identifier can be any identifier such as a character or icon. As long as the identifier can uniquely identify the client, the embodiment of the present invention does not specifically limit the form of the identifier. After distributing the ciphertext global model of the t-1th round to multiple clients, the aggregation server obtains the client identifier of each client. Then, after receiving the ciphertext model gradient information of the current round sent by the client, it determines the client identifier of the client that sent the gradient information, identifies the client that sent the gradient information as an online client, and identifies the delayed client in each client based on each client identifier and the online client identifier. Furthermore, since the delayed client did not upload the model gradient information to the aggregation server during the current round of model training, the aggregation server needs to predict the model gradient information of the current round for the delayed client based on the model gradient information of the global model of the t-1th round and the model gradient information of the global model of the t-2th round. This can prevent the problem of slow model convergence caused by abandoning the gradient contribution of the delayed client, thereby improving the convergence speed of the model. In an embodiment of the present invention, if the server waits for the delayed client network to recover before uploading gradient information for model update, time will be wasted. Therefore, the embodiment of the present invention can avoid waiting time by performing gradient prediction for the delayed client, thereby improving the training efficiency of the model.

[0065] 107. The aggregation server determines the online compensation factor of the online client and the delay compensation factor of the delayed client respectively, and aggregates the decrypted online model gradient information corresponding to the online ciphertext model gradient information of the online client and the delay model gradient information of the delayed client based on the online compensation factor and the delay compensation factor to obtain aggregated gradient information, and performs gradient update on the global model of the t-1th round based on the aggregated gradient information to obtain the global model of the current round.

[0066] For the embodiment of the present invention, the aggregation server can set the number of delay rounds of the online client in this round to 0, and automatically increase the number of delay rounds of the delayed client by 1. The embodiment of the present invention introduces an initial compensation coefficient and an attenuation factor to dynamically adjust the extrapolation strength of the delayed client to avoid model oscillation and improve convergence efficiency. Specifically, the aggregation server determines the online compensation factor of the online client and the delay compensation factor of the delayed client, and aggregates the model gradient information of the online client and the model gradient information of the delayed client according to the online compensation factor and the delay compensation factor. Finally, based on the aggregated gradient information obtained by aggregation, the global model of round t-1 in the aggregation server is gradient updated to obtain the global model of round t (current round). Furthermore, the global model of the current round is encrypted to obtain a ciphertext global model of the current round, and the ciphertext global model of the current round is used as the new ciphertext global model of the t-1th round. The new ciphertext global model of the t-1th round is distributed to multiple clients for a new round of model training to obtain new model gradient information of the current round. The new ciphertext model gradient information of the current round sent by the client is received, and based on the ciphertext model gradient information of the online client and the delayed model gradient information of the delayed client, the gradient of the new t-1th round global model is updated to obtain the new global model of the current round. The above process is repeated until the required number of training cycles is met. That is, if the current round t is greater than or equal to the preset global iteration number, the global model update is stopped. The global model obtained in the aggregation server after the last round of training is determined as the final model. The final trained model is then used to perform tasks such as predicting information in the corresponding application scenario.

[0067] An asynchronous federated model training method based on delayed gradient prediction, provided by embodiments of the present invention, performs local training on a local client based on the global model obtained in the previous round. The trained gradient information is encrypted using symmetric homomorphic encryption (SHE), and the quantized and compressed ciphertext gradients are uploaded to an aggregation server. The aggregation server predicts the gradients of the delayed client by calculating the global model change trend and a delay compensation factor. It then weightedly aggregates the gradient information of the online client and the predicted gradient information of the delayed client, using this weighted aggregated gradient information to update the global model gradient. A compensation factor driven by the number of delay rounds enables adaptive attenuation of the delayed gradient weight, avoiding model oscillation caused by overcorrection. The gradient information of the delayed client is extrapolated and predicted based on the global model change trend, accurately compensating for the contribution of the delayed client and improving model convergence efficiency and training accuracy. Furthermore, symmetric homomorphic encryption (SHE) is used to protect gradient information transmission, reducing computational overhead compared to traditional homomorphic encryption. Gradient parameters are compressed into integers using quantization and compression techniques, reducing the amount of ciphertext transmitted and balancing privacy protection with communication efficiency. This method is resistant to inference attacks from malicious servers and is compatible with secure aggregation protocols.

[0068] According to the present invention, a model training method based on asynchronous federation with delayed gradient prediction is provided. Compared with the current method of relying on the gradient of the client synchronous update to perform gradient update training on the model, the client of the present invention performs local training based on the global model obtained in the previous round, encrypts the trained gradient information, and uploads the encrypted ciphertext gradient information to the aggregation server. The aggregation server predicts a local gradient for the current round of model update for the delayed client that did not send the gradient information in time based on the gradient of the previous round and the gradient of the previous round, and calculates a delay compensation factor for each client participating in the current round of model training (including delayed clients and online clients) to quantify their weight parameters in this round of aggregation. Finally, the gradients of the online clients and the delayed clients are aggregated based on the weight parameters, and the previous round of models in the aggregation server are updated according to the aggregation results. By predicting gradients for delayed clients, the present invention can provide necessary update compensation for delayed clients, thereby improving the convergence efficiency of the overall model and the final training accuracy. On the other hand, the present invention uses a lightweight homomorphic encryption algorithm to encrypt and protect the gradients of server and client private data to prevent malicious clients from intercepting the gradients, ensuring that the model accuracy and convergence rate can be effectively maintained in an asynchronous environment while providing enhanced privacy protection for client and server data, thereby achieving an efficient and secure model training process based on federated learning.

[0069] Furthermore, in order to better illustrate the above process of classifying data, as a refinement and extension of the above embodiment, the embodiment of the present invention provides another asynchronous federated model training method based on delayed gradient prediction, such as Figure 2 As shown, the method includes:

[0070] 201. The aggregation server distributes the encrypted global model of round t-1 to multiple clients.

[0071] In this embodiment of the present invention, the model training system is first initialized. The model training system includes an aggregation server S and m clients C, where m is a positive integer. Specifically, the global iteration number T of the model training system, the local iteration number E of the client, and the learning rate η are set. The global model w1 of the aggregation server is randomly initialized. A public-private key pair is generated for encryption and decryption of model parameters between the aggregation server and the client, denoted as (PK, SK). If this is the first update, the aggregation server uses the client's public key PK to encrypt the initial global model w1, obtaining the ciphertext global model ||w1|| pk . The encrypted global model is then sent to each client.

[0072] 202. The client receives the encrypted global model of round t-1 sent by the aggregation server and uses the public key PKC1 The corresponding private key SK C1 Decrypt the ciphertext global model of the t-1th round to obtain the global model of the t-1th round.

[0073] Specifically, each client receives the encrypted global model ||w in round t-1 t-1 || and version number v, for the above ciphertext global model ||w t-1 || Decrypt as follows to obtain the global model of each client i

[0074]

[0075] 203. The client determines a local data set and quantizes the model gradient information of the global model of the t-1th round using a preset quantization accuracy parameter to obtain the processed global model of the t-1th round.

[0076] Among them, the type of local data set is determined according to the pre-application task of the model. Then, the model gradient information of the global model in the t-1th round is dimensionalized, i.e., quantized, according to the following formula to obtain the processed global model in the t-1th round:

[0077]

[0078] Among them, u is the preset quantization precision parameter, which is set according to actual needs. For example, the preset quantization precision parameter can be set to 10 7 .

[0079] 204. Randomly sample the training data set from the local data set, and use the training data set to iteratively train the global model after the processing of the t-1th round to obtain the model gradient information of the current round. Based on the preset homomorphic encryption algorithm, use the public key PK C2 The model gradient information of the current round is encrypted to obtain the ciphertext model gradient information of the current round and send it to the aggregation server.

[0080] In the embodiment of the present invention, since the asynchronous federation model training process requires multiple rounds of training, the embodiment of the present invention uses a single client C in the tth round. i The training process of is taken as an example to illustrate, i represents the client C i The index value of (Ci∈C). Client C i Get the encrypted global model ||w1|| distributed by the aggregation server pk And use the first private key SK to decrypt the ciphertext global model to obtain the plaintext global model w t Client C iUse local data to iterate E times to complete the local training of the global model and obtain the local model gradient information

[0081] Among them, client C i The process of using local data to train a local model includes: the client selects random samples from the training dataset to participate in this iterative training D i Represents client C i The local dataset, j represents the round of iteration, The training dataset is randomly selected in each iteration. The local model is then iteratively trained according to the following formula:

[0082]

[0083] Where η represents the learning rate of local training, represents the loss function, Represents client C i In the tth round (current round) of global training, the model parameter values ​​at the jth local model iteration are, Represents client C i In the tth round of global training, the model parameters updated after completing the jth local iteration are the client C i The corresponding model gradient information of the current round. The client can use multiple batches of data sets to train the model, and finally aggregate the gradient information after each training to obtain the aggregated gradient information

[0084] Furthermore, the aggregated gradient information of the current round of the client is dimensionally processed and encrypted as follows:

[0085]

[0086] in, is the model gradient information after dimension processing, u is the preset quantization accuracy parameter, is the ciphertext model gradient information of the current round. Thus, each client can obtain the ciphertext model gradient information according to the above method.

[0087] 205. Receive the ciphertext model gradient information of the current round sent by the client, and determine an online client and a delayed client in each client based on the client identifier to which the ciphertext model gradient information of the current round belongs.

[0088] Specifically, among the clients, the clients that have successfully sent the ciphertext model gradient information to the aggregation server are determined as online clients, and the clients that have not successfully sent the ciphertext model gradient information are determined as delayed clients.

[0089] 206. Model gradient information w based on the global model in round t-1 (t-1) And the model gradient information w of the global model in the t-2th round (t-2) , calculate the global model change Δw (t) .

[0090] Specifically, the global model change Δw is calculated according to the following formula: (t) :

[0091] Δw (t) =w t-1 -w t-2

[0092] 207. Model gradient information w based on the global model in round t-1 (t-1) And the model gradient information w of the global model in the t-2th round (t-2) , calculate the model evolution rate factor k t .

[0093] Specifically, according to the following formula, L based on two rounds of parameters 2 Norm calculation model evolution speed factor k t :

[0094]

[0095] 208. Based on the global model change Δw (t) and model evolution rate factor k t , determine the delay model gradient information of the delayed client in the current round

[0096] Specifically, the delay model gradient information of the delayed client in the current round is determined according to the following formula:

[0097]

[0098] 209. Determine an online compensation factor of the online client and a delay compensation factor of the delayed client respectively. Based on the online compensation factor and the delay compensation factor, aggregate the decrypted online model gradient information corresponding to the online ciphertext model gradient information of the online client and the delay model gradient information of the delayed client to obtain aggregated gradient information. Perform a gradient update on the global model of the t-1th round based on the aggregated gradient information to obtain the global model of the current round.

[0099] In order to perform gradient update on the global model in the aggregation server, it is necessary to first determine the online compensation factor of the online client and the delay compensation factor of the delayed client. Based on this, step 209 specifically includes: determining the online delay round number τ of the online client.zi , based on the number of online delay rounds τ zi , determine the online compensation factor γ zi ,in, γ0 is the initial compensation coefficient, α is the attenuation factor; Determine the delay round number τ of the delayed client yi , based on the delay round number τ yi , determine the delay compensation factor γ yi ,in,

[0100] The delay round number of the delayed client refers to the number of times the delayed client uploads the data. The online delay round number τ of the online client in the embodiment of the present invention is zi Can be set to 0.

[0101] Specifically, the aggregation server needs to calculate a compensation factor for all clients participating in this round of operations to guide their weights in this round of aggregation.

[0102] Furthermore, after determining the online compensation factor and the delay compensation factor, it is necessary to aggregate the model gradient information of each online client and the model gradient information of each delayed client. Based on this, the method includes: determining the total number of clients |C| of the online clients and the delayed clients, the model gradient information w of the global model of the t-1th round, and the total number of clients |C| of the online clients and the delayed clients. (t-1) , the client gradient information change between the model gradient information of the t-1th round and the model gradient information of the current round obtained when each client i performs model training Based on the online compensation factor, the delay compensation factor, the total number of clients |C|, and the model gradient information w (t-1) , the client gradient information change Determine the aggregate gradient information w t , Among them, ||·|| is the model gradient information encrypted using the preset homomorphic encryption algorithm, C is the client set, γ i is the delay compensation factor corresponding to the delayed client i or the delay compensation factor and online compensation factor corresponding to the online client i.

[0103] Specifically, the aggregation server calculates the compensation factors of the online client and the delayed client, and obtains the model gradient information of the online client and the delayed client. Then, the aggregation server obtains the aggregated gradient information ||w according to the aggregation formula t ||, and perform gradient updates on the t-1 round global model based on the aggregated gradient information to obtain the current round global model.

[0104] Furthermore, the aggregation server converts the current round of ciphertext global model Distribute to each client for the next round of training, and the update of the global model stops after reaching the set number of rounds T.

[0105] In this embodiment, local training is first performed on the local client based on the global model. The trained gradients are encrypted using the SHE homomorphic encryption algorithm and uploaded to the aggregation server. The aggregation server then calculates the client's weight value based on a delayed gradient prediction mechanism, and then completes the gradient weighted aggregation. By dynamically tracking the trend of global model changes, the client's gradient update is delayed, and an adaptive weight decay mechanism is combined to balance the compensation strength to prevent outdated updates from interfering with model convergence. Lightweight symmetric homomorphic encryption technology is used to protect the privacy of client gradients, and quantization compression is used to reduce communication and computational overhead.

[0106] According to another asynchronous federated model training method based on delayed gradient prediction provided by the present invention, compared with the current method of relying on the gradient of the client synchronous update to perform gradient update training on the model, the client of the present invention performs local training based on the global model obtained in the previous round, encrypts the trained gradient information, and uploads the encrypted ciphertext gradient information to the aggregation server. The aggregation server predicts a local gradient for the current round of model update for the delayed client that did not send the gradient information in time based on the gradient of the previous round and the gradient of the previous round, and calculates a delay compensation factor for each client participating in the current round of model training (including delayed clients and online clients) to quantify their weight parameters in this round of aggregation, and finally aggregates the gradients of the online client and the delayed client based on the weight parameters, and updates the previous round of model in the aggregation server according to the aggregation results. By predicting gradients for delayed clients, the present invention can provide necessary update compensation for delayed clients, thereby improving the convergence efficiency of the overall model and the final training accuracy. On the other hand, the present invention uses a lightweight homomorphic encryption algorithm to encrypt and protect the gradients of server and client private data to prevent malicious clients from intercepting the gradients, ensuring that the model accuracy and convergence rate can be effectively maintained in an asynchronous environment while providing enhanced privacy protection for client and server data, thereby achieving an efficient and secure model training process based on federated learning.

[0107] Further, as Figure 1 The specific implementation of the present invention provides an asynchronous federated model training device based on delayed gradient prediction, which is applied to an aggregation server, such as Figure 3 As shown, the device includes: a distribution unit 31, a gradient prediction unit 32, and a gradient update unit 33.

[0108] The distribution unit 31 can be used to distribute the ciphertext global model of the t-1th round to multiple clients, so that the clients can train the decrypted global model corresponding to the ciphertext global model of the t-1th round based on the local data set to obtain the model gradient information of the current round, wherein the ciphertext global model of the t-1th round is encrypted using a preset homomorphic encryption algorithm based on the public key PK of the client. C1 The global model of the t-1th round is encrypted.

[0109] The gradient prediction unit 32 can be used to receive the ciphertext model gradient information of the current round sent by the client, determine the online client and the delayed client in each of the clients based on the client identifier to which the ciphertext model gradient information of the current round belongs, and predict the delayed model gradient information of the current round for the delayed client based on the model gradient information of the global model of the t-1 round and the model gradient information of the global model of the t-2 round, wherein the ciphertext model gradient information is obtained by encrypting the model gradient information of the current round by the corresponding client using a preset homomorphic encryption algorithm.

[0110] The gradient updating unit 33 can be used to respectively determine the online compensation factor of the online client and the delay compensation factor of the delayed client, and based on the online compensation factor and the delay compensation factor, aggregate the decrypted online model gradient information corresponding to the online ciphertext model gradient information of the online client and the delayed model gradient information of the delayed client to obtain aggregated gradient information, and perform a gradient update on the global model of the t-1th round based on the aggregated gradient information to obtain the global model of the current round.

[0111] In a specific application scenario, in order to predict the delay model gradient information of the current round for the delay client, the gradient prediction unit 32 includes a calculation module 321 and a determination module 322 .

[0112] The calculation module 321 can be used to calculate the model gradient information w based on the global model of the t-1th round. (t-1) And the model gradient information w of the global model in the t-2th round (t-2) , calculate the global model change Δw (t) , where Δw (t) =w t-1 -w t-2 .

[0113] The calculation module 321 can also be used to calculate the model gradient information w of the global model based on the t-1 round. (t-1) And the model gradient information w of the global model in the t-2th round (t-2) , calculate the model evolution rate factor k t ,in,

[0114] The determination module 322 can also be used to determine the global model change Δw (t) and the model evolution rate factor k t , determine the delay model gradient information of the delay client in the current round in,

[0115] In a specific application scenario, in order to determine the online compensation factor of the online client and the delay compensation factor of the delayed client, the gradient updating unit 33 can be used to determine the online delay round number τ of the online client. zi , based on the number of online delay rounds τ zi , determine the online compensation factor γ zi ,in, γ0 is the initial compensation coefficient, α is the attenuation factor; Determine the delay round number τ of the delayed client yi , based on the delay round number τ yi , determine the delay compensation factor γ yi ,in,

[0116] In a specific application scenario, in order to determine the aggregated gradient information, the gradient updating unit 33 can be used to determine the total number of clients |C| of the online clients and the delayed clients, the model gradient information w of the global model of the t-1th round, and the model gradient information w of the global model of the t-1th round. (t-1) , the client gradient information change between the model gradient information of the t-1th round and the model gradient information of the current round obtained when each client i performs model training Based on the online compensation factor, the delay compensation factor, the total number of clients |C|, and the model gradient information w (t-1) , the client gradient information change Determine the aggregate gradient information w t , Among them, ||·|| is the model gradient information encrypted using the preset homomorphic encryption algorithm, C is the client set, γ i is the delay compensation factor corresponding to the delayed client i or the delay compensation factor and online compensation factor corresponding to the online client i.

[0117] Further, as Figure 1 The specific implementation of the present invention provides an asynchronous federated model training device based on delayed gradient prediction, which is applied to the client, such as Figure 4 As shown, the device includes: a receiving unit 41, a decryption unit 42, a training unit 43, and an encryption unit 44.

[0118] The receiving unit 41 can be used to receive the ciphertext global model of the t-1th round sent by the aggregation server, wherein the ciphertext global model of the t-1th round is the aggregation server using a preset homomorphic encryption algorithm based on the public key PK of the client C It is obtained by encrypting the global model of the t-1th round.

[0119] The decryption unit 42 can be used to decrypt the public key PK C1 The corresponding private key SK C1 The ciphertext global model of the t-1th round is decrypted to obtain the global model of the t-1th round.

[0120] The training unit 43 may be configured to determine a local data set, and use the local data set to train the global model of the t-1th round to obtain model gradient information of the current round.

[0121] The encryption unit 44 can be used to use the public key PK based on the preset homomorphic encryption algorithm. C2 The model gradient information of the current round is encrypted to obtain the ciphertext model gradient information of the current round.

[0122] In a specific application scenario, in order to use the local data set to train the global model of the t-1th round, the training unit 43 can be specifically used to use a preset quantization accuracy parameter to quantize the model gradient information of the global model of the t-1th round to obtain the global model after processing in the t-1th round; randomly sample a data set to be trained from the local data set, and use the data set to be trained to iteratively train the global model after processing in the t-1th round to obtain the model gradient information of the current round.

[0123] It should be noted that for other corresponding descriptions of the functional modules involved in the asynchronous federated model training device based on delayed gradient prediction provided by the embodiment of the present invention, please refer to Figure 1 The corresponding description of the method shown will not be repeated here.

[0124] Based on the above Figure 1 The method shown, accordingly, an embodiment of the present invention further provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the following steps: distributing the ciphertext global model of the t-1th round to multiple clients, so that the clients train the decrypted global model corresponding to the ciphertext global model of the t-1th round based on the local data set to obtain the model gradient information of the current round, wherein the ciphertext global model of the t-1th round is encrypted using a preset homomorphic encryption algorithm based on the public key PK of the client C1The method comprises the steps of: encrypting the global model of the t-1 round; receiving the ciphertext model gradient information of the current round sent by the client, determining the online client and the delayed client in each client based on the client identifier to which the ciphertext model gradient information of the current round belongs, and predicting the delayed model gradient information of the current round for the delayed client based on the model gradient information of the global model of the t-1 round and the model gradient information of the global model of the t-2 round, wherein the ciphertext model gradient information is obtained by encrypting the model gradient information of the current round by the corresponding client using a preset homomorphic encryption algorithm; respectively determining the online compensation factor of the online client and the delay compensation factor of the delayed client, and aggregating the decrypted online model gradient information corresponding to the online ciphertext model gradient information of the online client and the delay model gradient information of the delayed client based on the online compensation factor and the delay compensation factor to obtain aggregated gradient information, and performing gradient update on the global model of the t-1 round based on the aggregated gradient information to obtain the global model of the current round.

[0125] Based on the above Figure 1 The method shown and Figure 3 、 Figure 4 The embodiment of the device shown in the figure, the embodiment of the present invention also provides a physical structure diagram of a computer device, such as Figure 5 As shown, the computer device includes: a processor 51, a memory 52, and a computer program stored in the memory 52 and executable on the processor, wherein the memory 52 and the processor 51 are both arranged on a bus 53, and when the processor 51 executes the program, the following steps are implemented: distributing the ciphertext global model of the t-1th round to multiple clients, so that the clients train the decrypted global model corresponding to the ciphertext global model of the t-1th round based on the local data set to obtain the model gradient information of the current round, wherein the ciphertext global model of the t-1th round is encrypted using a preset homomorphic encryption algorithm based on the public key PK of the client. C1The method comprises the steps of: encrypting the global model of the t-1 round; receiving the ciphertext model gradient information of the current round sent by the client, determining the online client and the delayed client in each client based on the client identifier to which the ciphertext model gradient information of the current round belongs, and predicting the delayed model gradient information of the current round for the delayed client based on the model gradient information of the global model of the t-1 round and the model gradient information of the global model of the t-2 round, wherein the ciphertext model gradient information is obtained by encrypting the model gradient information of the current round by the corresponding client using a preset homomorphic encryption algorithm; respectively determining the online compensation factor of the online client and the delay compensation factor of the delayed client, and aggregating the decrypted online model gradient information corresponding to the online ciphertext model gradient information of the online client and the delay model gradient information of the delayed client based on the online compensation factor and the delay compensation factor to obtain aggregated gradient information, and performing gradient update on the global model of the t-1 round based on the aggregated gradient information to obtain the global model of the current round.

[0126] Through the technical solution of the present invention, the present invention performs local training based on the global model obtained in the previous round by the client, encrypts the trained gradient information, and uploads the encrypted ciphertext gradient information to the aggregation server. The aggregation server predicts a local gradient for the current round model update for the delayed client that did not send the gradient information in time based on the gradient of the previous round and the gradient of the previous round. At the same time, a delay compensation factor is calculated for each client participating in the current round of model training (including delayed clients and online clients) to quantify their weight parameters aggregated in this round. Finally, the gradients of the online client and the gradients of the delayed client are aggregated based on the weight parameters, and the previous round model in the aggregation server is updated according to the aggregation results. By predicting the gradient for the delayed client, the present invention can provide the necessary update compensation for the delayed client, thereby improving the convergence efficiency of the overall model and the final training accuracy. On the other hand, the present invention uses a lightweight homomorphic encryption algorithm to encrypt and protect the gradients of the server and client private data to prevent malicious clients from intercepting the gradients, ensuring that the model accuracy and convergence rate can be effectively maintained in an asynchronous environment while providing enhanced privacy protection for client data and server data, thereby achieving an efficient and secure model training process based on federated learning.

[0127] Obviously, those skilled in the art will appreciate that the various modules or steps of the present invention described above can be implemented using a general-purpose computing device, centralized on a single computing device, or distributed across a network of multiple computing devices. Alternatively, they can be implemented using program code executable by a computing device, which can then be stored in a storage device and executed by the computing device. In some cases, the steps shown or described can be performed in a different order than that shown, or can be fabricated as separate integrated circuit modules, or multiple modules or steps can be fabricated as a single integrated circuit module. Thus, the present invention is not limited to any particular combination of hardware and software.

[0128] The foregoing description is merely a preferred embodiment of the present invention and is not intended to limit the present invention. Those skilled in the art will readily appreciate that various modifications and variations of the present invention are possible. Any modifications, equivalent substitutions, or improvements made within the spirit and principles of the present invention shall be included within the scope of protection of the present invention.

Claims

1. A model training method based on asynchronous federation with delayed gradient prediction, applied to an aggregation server, characterized in that: include: The ciphertext global model of the t-1th round is distributed to multiple clients, so that the clients train the decrypted global model corresponding to the ciphertext global model of the t-1th round based on the local data set to obtain the model gradient information of the current round, wherein the ciphertext global model of the t-1th round is encrypted using a preset homomorphic encryption algorithm based on the public key PK of the client. C1 Encrypting the global model of the t-1th round; Receive the ciphertext model gradient information of the current round sent by the client, determine the online client and the delayed client in each of the clients based on the client identifier to which the ciphertext model gradient information of the current round belongs, and predict the delayed model gradient information of the current round for the delayed client based on the model gradient information of the global model of the t-1th round and the model gradient information of the global model of the t-2th round, wherein the ciphertext model gradient information is obtained by encrypting the model gradient information of the current round by the corresponding client using a preset homomorphic encryption algorithm; An online compensation factor of the online client and a delay compensation factor of the delayed client are respectively determined; based on the online compensation factor and the delay compensation factor, decrypted online model gradient information corresponding to the online ciphertext model gradient information of the online client and the delayed model gradient information of the delayed client are aggregated to obtain aggregated gradient information; and a gradient update is performed on the global model of the t-1th round based on the aggregated gradient information to obtain the global model of the current round.

2. The method according to claim 1, characterized in that The predicting, for the delayed client, the delayed model gradient information of the current round based on the model gradient information of the global model of the t-1th round and the model gradient information of the global model of the t-2th round includes: Based on the model gradient information w of the global model in the t-1th round (t-1) And the model gradient information w of the global model in the t-2th round (t-2) , calculate the global model change Δw (t) , where Δw (t) =w t-1 -w t-2 ; Model gradient information w based on the global model in round t-1 (t-1) And the model gradient information w of the global model in the t-2th round (t-2) , calculate the model evolution rate factor k t ,in, Based on the global model change Δw (t) and the model evolution rate factor k t , determine the delay model gradient information of the delay client in the current round in, 3. The method according to claim 1, characterized in that The separately determining the online compensation factor of the online client and the delay compensation factor of the delayed client includes: Determine the online delay round number τ of the online client zi , based on the number of online delay rounds τ zi , determine the online compensation factor γ zi ,in, γ0 is the initial compensation coefficient, α is the attenuation factor; Determine the delay round number τ of the delayed client yi , based on the delay round number τ yi , determine the delay compensation factor γ yi ,in, 4. The method according to claim 1, wherein The step of aggregating the decrypted online model gradient information corresponding to the online ciphertext model gradient information of the online client and the delayed model gradient information of the delayed client based on the online compensation factor and the delay compensation factor to obtain aggregated gradient information includes: Determine the total number of clients |C| of the online clients and the delayed clients, the model gradient information w of the global model of the t-1th round (t-1) , the client gradient information change between the model gradient information of the t-1th round and the model gradient information of the current round obtained when each client i performs model training Based on the online compensation factor, the delay compensation factor, the total number of clients |C|, and the model gradient information w (t-1) , the client gradient information change Determine the aggregate gradient information w t , Among them, ||·|| is the model gradient information encrypted using the preset homomorphic encryption algorithm, C is the client set, γ i is the delay compensation factor corresponding to the delayed client i or the delay compensation factor and online compensation factor corresponding to the online client i.

5. An asynchronous federated model training method based on delayed gradient prediction, applied to the client, characterized in that: include: Receive the ciphertext global model of the t-1th round sent by the aggregation server, wherein the ciphertext global model of the t-1th round is the aggregation server using a preset homomorphic encryption algorithm based on the public key PK of the client C Obtained by encrypting the global model of round t-1; According to the public key PK C1 The corresponding private key SK C1 Decrypting the ciphertext global model of the t-1th round to obtain the global model of the t-1th round; Determine a local data set, and use the local data set to train the global model of the t-1th round to obtain model gradient information of the current round; Based on the preset homomorphic encryption algorithm, using the public key PK C2 The model gradient information of the current round is encrypted to obtain the ciphertext model gradient information of the current round.

6. The method according to claim 5, characterized in that The using the local data set to train the global model of the t-1th round to obtain the model gradient information of the current round includes: Quantizing the model gradient information of the global model of the t-1th round using a preset quantization precision parameter to obtain the global model after the t-1th round of processing; A data set to be trained is randomly sampled from the local data set, and the global model after the processing of the t-1th round is iteratively trained using the data set to be trained to obtain the model gradient information of the current round.

7. An asynchronous federated model training device based on delayed gradient prediction, applied to an aggregation server, characterized in that: include: The distribution unit is used to distribute the ciphertext global model of the t-1th round to multiple clients, so that the clients can train the decrypted global model corresponding to the ciphertext global model of the t-1th round based on the local data set to obtain the model gradient information of the current round, wherein the ciphertext global model of the t-1th round is encrypted using a preset homomorphic encryption algorithm based on the public key PK of the client. C1 Encrypting the global model of the t-1th round; a gradient prediction unit, configured to receive the ciphertext model gradient information of the current round sent by the client, determine an online client and a delayed client in each of the clients based on a client identifier to which the ciphertext model gradient information of the current round belongs, and predict the delayed model gradient information of the current round for the delayed client based on the model gradient information of the global model of the t-1th round and the model gradient information of the global model of the t-2th round, wherein the ciphertext model gradient information is obtained by encrypting the model gradient information of the current round by the corresponding client using a preset homomorphic encryption algorithm; The gradient updating unit is configured to respectively determine an online compensation factor of the online client and a delay compensation factor of the delayed client; based on the online compensation factor and the delay compensation factor, aggregate the decrypted online model gradient information corresponding to the online ciphertext model gradient information of the online client and the delayed model gradient information of the delayed client to obtain aggregated gradient information; and perform a gradient update on the global model of the t-1th round based on the aggregated gradient information to obtain the global model of the current round.

8. An asynchronous federated model training device based on delayed gradient prediction, applied to a client, characterized in that: include: The receiving unit is configured to receive the encrypted global model of the t-1th round sent by the aggregation server, wherein the encrypted global model of the t-1th round is the encrypted global model of the client using the preset homomorphic encryption algorithm by the aggregation server according to the public key PK C Obtained by encrypting the global model of round t-1; Decryption unit, used to decrypt the public key PK C1 The corresponding private key SK C1 Decrypting the ciphertext global model of the t-1th round to obtain the global model of the t-1th round; A training unit, configured to determine a local data set and train the global model of the t-1th round using the local data set to obtain model gradient information of the current round; Encryption unit, used to use public key PK based on preset homomorphic encryption algorithm C2 The model gradient information of the current round is encrypted to obtain the ciphertext model gradient information of the current round.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 6 are implemented.

10. A computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 6 are implemented.

Images