Privacy protection decentralized federated learning method and device with high precision guarantee

Abstract

The application discloses a high-precision guaranteed privacy protection decentralized federated learning method and device, which comprises the following steps: initializing model weight and gradient tracking variable of a client participating in federated learning; sending pure random noise to a neighbor client by each client at the first communication; subtracting the total received noise from the total sent noise after receiving the neighbor noise; adding the difference between the sending noise and the receiving noise to the update of the gradient tracking variable to disturb the subsequent gradient information, preventing an eavesdropper from obtaining accurate gradient values to infer the original data; sending model weight and gradient tracking variable to a neighbor client by each client at the subsequent communication; and correspondingly, aggregating the received parameters, since the difference between the sending noise and the receiving noise in the gradient tracking variable can be zeroed in a global perspective, the stability of the global gradient and the global tracking of the tracking variable can be ensured; finally, each client continuously exchanges information and updates the model until the expected model precision is reached.

Description

Technical Field

[0001] This invention relates to the field of federated learning technology, and in particular to a high-precision, privacy-preserving decentralized federated learning method and apparatus. Background Technology

[0002] In the fields of modern artificial intelligence and machine learning, the effectiveness of a system depends on the continuous improvement of machine learning models, computer systems and their architectures, as well as the accumulation of large amounts of data resources. Typically, the machine learning process begins with the collection of key data, which is aggregated and stored in a centralized data processing center or core server. Subsequently, a computing cluster is configured with one or more computing devices (whether virtual or physical) to train a machine learning model closely related to a specific task, optimizing the model parameters. After model training is complete, the model is used to perform inference processing using new data, i.e., executing the prediction task. However, during this process, the transmission of large amounts of end-user data may be subject to eavesdropping risks, and the storage of data in centralized data centers or core servers may also be vulnerable to security attacks, increasing the risk of user privacy breaches.

[0003] Federated learning offers a solution to the problem of direct data collection. It aggregates model gradient values ​​trained on local data by various end users through a central server, enabling collaborative model training. Federated learning avoids the need for direct data transmission, becoming a novel privacy-preserving computing paradigm. However, if the central server or cloud service provider fails, the entire system may suddenly stop operating without any warning. To overcome the limitations of central servers, a decentralized federated learning approach is proposed. In this approach, multiple data owners interact directly with neighbors based on communication topology to aggregate model gradient values, reducing the risk of system paralysis due to single points of failure.

[0004] Nevertheless, in decentralized federated learning, because model gradients are generated based on the original data, the transmitted gradient values ​​may expose the characteristics of the original data, making them vulnerable to security threats such as data reconstruction attacks and model inversion attacks, leading to data leakage. Existing protection measures, such as differential privacy techniques that introduce random noise into the transmitted gradient values, increase the difficulty of data reconstruction but do not provide sufficient protection, and the noise interference may affect the model's predictive performance.

[0005] Therefore, there is an urgent need to develop a high-precision, privacy-preserving decentralized federated learning method that can effectively protect participant data while ensuring that the model has highly accurate predictive performance. Summary of the Invention

[0006] This invention aims to address the issues of privacy leaks and original data reconstruction that may result from gradient tracking variables passed between clients in decentralized federated learning, and provides a high-precision, privacy-preserving decentralized federated learning method and apparatus.

[0007] According to a first aspect of the embodiments of this application, a high-precision, privacy-preserving decentralized federated learning method is provided, comprising:

[0008] Each client participating in the decentralized federated learning process predetermines the machine learning model structure and parameter framework that need to be trained collaboratively, initializes the model weights and corresponding gradient tracking variables based on its own local dataset, and then determines the aggregation weights of neighboring clients according to its own importance to neighboring clients on the communication topology graph.

[0009] After the parameters required for the training process are determined locally, each client begins to communicate with its neighboring clients. During the first communication, each client generates noise with a mean of zero completely randomly and sends pure noise to its neighboring clients, while also receiving noise sent by its neighboring clients. Then, each client performs the subtraction of the transmitted and received noise and appends the difference to its local gradient tracking variable to update the gradient tracking variable.

[0010] In subsequent communications, each client sends model weights and gradient tracking variables to its neighboring clients, while aggregating the corresponding weights and gradient tracking variables received from neighboring clients, and updating its own weights and gradient tracking variables by combining the model gradient values ​​trained on the local dataset.

[0011] In subsequent rounds, model weights and gradient tracking variables are continuously exchanged and aggregated until the model jointly trained by the clients reaches the expected prediction accuracy, so that each client obtains a trained machine learning model and makes corresponding predictions.

[0012] According to a second aspect of the embodiments of this application, a high-precision, privacy-preserving decentralized federated learning device is provided, comprising:

[0013] The negotiation module is used by each client participating in the decentralized federated learning process to predetermine the machine learning model structure and parameter framework that need to be trained collaboratively, initialize the model weights and corresponding gradient tracking variables based on its own local dataset, and then determine the aggregation weights of neighboring clients according to its own importance to neighboring clients on the communication topology graph.

[0014] The noise-adding module is used to determine the parameters required for the training process locally. After each client starts communicating with its neighboring clients, during the first communication, each client generates noise with a mean of zero completely randomly and sends pure noise to its neighboring clients. At the same time, it needs to receive the noise sent by its neighboring clients. Then, each client performs the subtraction of the transmitted and received noise and appends the difference to its local gradient tracking variable to update the gradient tracking variable.

[0015] The aggregation module is used in subsequent communications, where each client sends model weights and gradient tracking variables to neighboring clients, aggregates the corresponding weights and gradient tracking variables received from neighboring clients, and updates its own weights and gradient tracking variables by combining the model gradient values ​​trained on the local dataset.

[0016] The testing module is used to continuously exchange and aggregate model weights and gradient tracking variables in subsequent rounds until the model jointly trained by the clients reaches the expected prediction accuracy, so that each client obtains a trained machine learning model and makes corresponding predictions.

[0017] According to a third aspect of the embodiments of this application, an electronic device is provided, characterized in that it includes:

[0018] One or more processors;

[0019] Memory, used to store one or more programs;

[0020] When the one or more programs are executed by the one or more processors, the one or more processors perform the method as described in the first aspect.

[0021] According to a fourth aspect of the embodiments of this application, a computer-readable storage medium is provided, on which computer instructions are stored, which, when executed by a processor, implement the steps of the method as described in the first aspect.

[0022] Beneficial effects:

[0023] This application differs from traditional decentralized federated learning methods and privacy-preserving approaches by proposing a privacy-preserving decentralized federated learning method based on high accuracy assurance. Unlike previous methods that directly add noise to gradients, this method introduces the difference between transmitted and received noise into the gradient tracking variables of each client through noise exchange. This noise difference perturbs the local gradient information, thus confusing eavesdroppers. Furthermore, globally, the noise received by all clients is equal to the noise sent, achieving global noise elimination and stable global gradient updates, thereby ensuring an accurate decentralized federated learning process and lossless model prediction accuracy. This method does not introduce additional computational or communication overhead, but effectively prevents data reconstruction attacks and guarantees high-precision prediction performance. This method can be widely applied to various decentralized federated learning systems, achieving dual guarantees of accuracy and privacy. Attached Figure Description

[0024] Figure 1 A flowchart illustrating a high-precision, privacy-preserving, decentralized federated learning method provided in this embodiment of the invention.

[0025] Figure 2 A schematic diagram of a system model for applying the decentralized federated learning method provided in an embodiment of the present invention.

[0026] Figure 3 This is a model prediction accuracy diagram in the decentralized federated learning method provided in the embodiments of the present invention.

[0027] Figure 4 A diagram illustrating the privacy protection capabilities of the decentralized federated learning method provided in this embodiment of the invention.

[0028] Figure 5 This diagram illustrates the original data leakage when the decentralized federated learning method provided in this embodiment of the invention is subjected to a data reconstruction attack.

[0029] Figure 6 This invention provides a block diagram of a high-precision, privacy-preserving, decentralized federated learning device. Detailed Implementation

[0030] The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the specific content of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the protection scope of the present invention. Contents not described in detail in the embodiments of the present invention belong to the prior art known to those skilled in the art.

[0031] This invention provides a high-precision, privacy-preserving decentralized federated learning method that can be widely applied to decentralized federated learning scenarios with high precision and privacy computation requirements. In this scenario, each client first sends and receives random noise, using the difference between the transmitted and received noise as "noise" to add to the gradient, thereby protecting gradient information and preventing eavesdroppers from obtaining precise gradient information and inferring the original data. Simultaneously, in a decentralized system, the globally transmitted and received noise is balanced, meaning the sum of the noise differences across all clients is always equal to zero, ensuring stable updates to the global gradient, promoting efficient aggregation in decentralized federated learning, and achieving lossless prediction accuracy in the final model. The effectiveness of this method's high precision and privacy protection is demonstrated by both theoretical guarantees and experimental results.

[0032] The method of the present invention will be described in detail below with reference to this scenario.

[0033] Figure 1 This is a flowchart illustrating a high-precision, privacy-preserving, decentralized federated learning method according to an exemplary embodiment, such as... Figure 1 As shown, the method may include the following steps:

[0034] S1: Each client participating in the decentralized federated learning process predetermines the machine learning model structure and parameter framework that needs to be trained collaboratively, initializes the model weights and corresponding gradient tracking variables based on its own local dataset, and then determines the aggregation weights of neighboring clients based on its own importance to neighboring clients on the communication topology graph.

[0035] Specifically, during the parameter initialization process described above, each client must negotiate with its neighbors and initialize its own parameters based on its own situation, including:

[0036] Let n represent the total number of clients participating in decentralized federated learning, and C... i This represents the i-th participating client; w i The model weights for this client are represented by ▽w i This represents the model gradient value for the client, expressed in y. i The gradient tracking variable corresponding to this client is represented by λ, which represents the update step size of this client; Indicates client C i The "outer neighbor", that is, the one that receives the client C i The neighbor of the sent information; using Indicates client C i The "inner neighbor", that is, client C i Receives the message from the neighbor that sent it. For each internal neighbor C j Client C i Set the aggregate weight a for it ijAnd guarantee For each external neighbor C l Client C i Set the aggregate weight a for it li And guarantee By a ij The matrix A formed is [a ij ] n×n The matrix is ​​a double random matrix, which ensures the consistency of parameters in the final trained model of federated learning.

[0037] Each client C i The model weights w need to be initialized randomly. i 0 Initialize gradient tracking variables based on model gradients. Gradient tracking variables are used to continuously track global gradients in decentralized federated learning, such as... It is crucial to protect gradient tracking variables to prevent eavesdroppers from inferring the original data from the received gradient tracking variables, which could lead to the leakage of the personal privacy of data owners participating in federated learning.

[0038] S2: After the parameters required for the training process are determined locally, each client starts to communicate with its neighboring clients. During the first communication, each client generates noise with a mean of zero completely randomly and sends pure noise to its neighboring clients. At the same time, it needs to receive the noise sent by its neighboring clients. Then each client performs the subtraction of the transmitted and received noise and appends the difference to its local gradient tracking variable to update the gradient tracking variable.

[0039] Specifically, the client generates random noise and exchanges noise information with neighboring clients, then adds the difference between the transmitted and received noise to its local gradient tracking variables, including:

[0040] Using δ li Indicates each client C i To the external neighbor client C l Send random noise, using δ ij Indicates client C i Received from internal neighbor client C j The random noise. Each client subtracts the total noise received from its internal neighbors from the total noise sent to its external neighbors, i.e., in For client C i Total noise transmitted, For client C i Total noise received.

[0041] Then, each client adds this difference in transmit and receive noise to its local gradient tracking variable, that is... This introduces a perturbation into the accurate gradient tracking variable. Since the gradient tracking variable is updated iteratively, the next update requires the result of the previous update. Therefore, the initially introduced difference between the transmitted and received gradients can persist in the gradient tracking variable, thus ensuring that eavesdroppers cannot obtain accurate gradient information to infer the original data throughout the entire process.

[0042] Specifically, through iterative updates, the gradient tracking variable is updated in the subsequent t-th round. In, there exists A t (δ S -δ R The disturbance value of ), where A t Let δ represent the power of t of a birandom matrix A. S The client sends a vector composed of noise, that is... δ R The client receives a vector composed of noise, that is... Due to the birandom matrix A and its power A t The noise reduction method indirectly achieves adaptive noise addition, as it controls the aggregation process and the convergence process of the client's consistent state in decentralized federated learning. The noise proposed in this method can be added in a corresponding amount based on the client's aggregation state in decentralized federated learning, thus preventing excessive noise from disrupting consistency and the normal training process.

[0043] S3: In subsequent communications, each client sends model weights and gradient tracking variables to neighboring clients, while aggregating the corresponding weights and gradient tracking variables received from neighboring clients, and updating its own weights and gradient tracking variables by combining the model gradient values ​​trained on the local dataset.

[0044] Specifically, the client sends its own model weights and gradient tracking variables, and updates and iterates the model parameters based on its own model gradients and the corresponding parameters received from its internal neighbors, including:

[0045] use In the t-th round, client C i Model weights, Let C represent the gradient tracking variable in round t, where t ≥ 1. In round t, each client C... i To the external neighbor client C l Send model weights and gradient tracking variables Simultaneously received from the internal neighbor client C j Model weights and gradient tracking variables

[0046] First, client C iThe model weights need to be aggregated and updated, that is... Where a ij The initial aggregation weights are λ, and the update step size is λ. Then, based on the updated model weights, the model is trained on the local private dataset to obtain a new set of model gradient values.

[0047] Then, client C i Aggregate and update gradient tracking variables, that is... As described in S2, the update of the gradient tracking variable needs to be based on the state of the previous step. Therefore, the difference between the transmitted and received noise can continuously perturb the variable to prevent eavesdroppers from obtaining accurate gradient information to infer the original sensitive data.

[0048] Meanwhile, since the difference between transmit and receive noise in the gradient tracking variable can be eliminated globally, the global gradient tracking capability and stable update of the global gradient can be guaranteed. Specifically, from a global perspective, all noise differences are aggregated, i.e.

[0049]

[0050] By utilizing the property that the global sum of gradient differences equals zero, the global tracking property of the gradient tracking variable can be guaranteed:

[0051]

[0052] From the above equation, we can derive... That is to say After introducing noise in the initial step, since the model weights are not updated, the noise can be eliminated globally, maintaining the global tracking of gradient tracking variables and ensuring an accurate aggregation process.

[0053]

[0054] The aforementioned gradient tracking variable global tracking characteristics and stable global gradient updates ensure accurate aggregation and stable training process in decentralized federated learning, enabling the machine learning model trained by this method to achieve prediction accuracy comparable to the model trained by the original noisy algorithm.

[0055] S4: In subsequent rounds, model weights and gradient tracking variables are continuously exchanged and aggregated until the model jointly trained by the clients reaches the expected prediction accuracy, so that each client obtains a trained machine learning model and makes corresponding predictions.

[0056] Specifically, each client participating in decentralized federated learning can pre-determine a total number of communication rounds T, after which the federated learning training process automatically ends; alternatively, it can choose not to specify a total number of communication rounds, and after each communication to aggregate and update the model weights, test the model's prediction accuracy. If each client achieves the expected accuracy, the training process can end. This method ensures the consistency of the final aggregated model weights across all participating clients, guaranteeing the fairness of decentralized federated learning, incentivizing more high-value data owners to participate in training, and promoting the flow of data elements and the construction of high-precision models.

[0057] The transaction processing method of this invention is implemented on an Ubuntu 18.04 system on a server with 192GB of memory and an Intel core 2.80GHz processor. The total number of communication rounds is pre-set to T = 1500. The noise introduced by this method is Gaussian noise with a mean of 0 and a variance of 4.5 added in the initial communication rounds. In the embodiment, a differential privacy method is also introduced for comparison. Differential privacy adds Gaussian noise with a variance of 0.003 in each of the 1500 rounds, making the total amount of noise added by the two methods consistent, thus achieving a fair comparison.

[0058] Figure 2 This is a schematic diagram of a system model for the decentralized federated learning method provided in this embodiment of the invention. In the initial round, the five clients of the decentralized federated learning send random noise and introduce the difference between the transmitted and received noise into the update of the local gradient to perturb the subsequent gradient. In subsequent rounds, each client aggregates the model weights and gradients. The difference between the transmitted and received noise attached to the gradient can be eliminated to zero from a global perspective, ensuring that the global gradient is not affected by noise, thereby maintaining the accurate decentralized federated learning aggregation process and prediction accuracy.

[0059] The performance results of the high-precision, privacy-preserving decentralized federated learning method (LPPA mechanism) proposed in this invention were tested and analyzed through simulation experiments, as follows: Figure 3 , Figure 4 As shown, different noise variances (i.e., different σ) are introduced. 2 When the LPPA mechanism is used, its model prediction accuracy is comparable to that of the original noise-free algorithm (Ori.), while significantly improving the accuracy compared to existing differential privacy methods (DP mechanism) (see [link]). Figure 3 Furthermore, compared to differential privacy and the original noisy algorithm (without protection mechanisms), the LPPA mechanism achieves the highest privacy protection performance (see [link to relevant documentation]). Figure 4The higher the error margin (MSE) of a data reconstruction attack, the stronger the privacy protection capability. Visualizing the results, it can be seen that the original, unprotected algorithm completely exposes the original data under a data reconstruction attack. Differential privacy methods provide some privacy protection but still allow for some data recovery. Under the LPPA mechanism proposed in this invention, the eavesdropper cannot recover the original data at all, effectively protecting the personal privacy of the data owner (see...). Figure 5 ).

[0060] Corresponding to the aforementioned embodiments of the high-precision, privacy-preserving decentralized federated learning method, this application also provides embodiments of the high-precision, privacy-preserving decentralized federated learning method. (Refer to...) Figure 6 The device includes:

[0061] Negotiation Module 1 is used by each client participating in the decentralized federated learning process to predetermine the machine learning model structure and parameter framework that need to be trained collaboratively, initialize the model weights and corresponding gradient tracking variables based on its own local dataset, and then determine the aggregation weights of neighboring clients according to its own importance to neighboring clients on the communication topology graph.

[0062] Noise module 2 is used to determine the parameters required for the training process locally. After each client starts communicating with its neighboring clients, during the first communication, each client generates noise with a mean of zero completely randomly and sends pure noise to its neighboring clients. At the same time, it needs to receive the noise sent by its neighboring clients. Then each client performs the subtraction of the transmitted and received noise and appends the difference to its local gradient tracking variable to update the gradient tracking variable.

[0063] The aggregation module 3 is used to send model weights and gradient tracking variables to neighboring clients in subsequent communications, while aggregating the corresponding weights and gradient tracking variables received from neighboring clients, and updating its own weights and gradient tracking variables by combining the model gradient values ​​trained on the local dataset.

[0064] Test module 4 is used to continuously exchange and aggregate model weights and gradient tracking variables in subsequent rounds until the model jointly trained by the clients reaches the expected prediction accuracy, so that each client obtains a trained machine learning model and makes corresponding predictions. Regarding the apparatus in the above embodiments, the specific manner in which each module performs its operations has been described in detail in the embodiments related to the method, and will not be elaborated upon here.

[0065] For the device embodiments, since they basically correspond to the method embodiments, the relevant parts can be referred to in the description of the method embodiments. The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this application according to actual needs. Those skilled in the art can understand and implement this without creative effort.

[0066] Accordingly, this application also provides an electronic device, comprising: one or more processors; a memory for storing one or more programs; wherein when the one or more programs are executed by the one or more processors, the one or more processors implement the method as described in the first aspect; this application also provides a computer-readable storage medium storing computer instructions thereon, characterized in that, when the instructions are executed by a processor, they implement the steps of the high-precision privacy-preserving decentralized federated learning method described above.

[0067] It should be understood that this application is not limited to the precise structure described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from its scope. The scope of this application is limited only by the appended claims.

Claims

1. A high-precision, privacy-preserving, decentralized federated learning method, characterized in that, include: Each client participating in the decentralized federated learning process predetermines the machine learning model structure and parameter framework that need to be trained collaboratively, initializes the model weights and corresponding gradient tracking variables based on its own local dataset, and then determines the aggregation weights of neighboring clients according to its own importance to neighboring clients on the communication topology graph. After the parameters required for the training process are determined locally, each client begins to communicate with its neighboring clients. During the first communication, each client generates noise with a mean of zero completely randomly and sends pure noise to its neighboring clients, while also receiving noise sent by its neighboring clients. Then, each client performs the subtraction of the transmitted and received noise and appends the difference to its local gradient tracking variable to update the gradient tracking variable. In subsequent communications, each client sends model weights and gradient tracking variables to its neighboring clients, while aggregating the corresponding weights and gradient tracking variables received from neighboring clients, and updating its own weights and gradient tracking variables by combining the model gradient values ​​trained on the local dataset. In subsequent rounds, model weights and gradient tracking variables are continuously exchanged and aggregated until the model jointly trained by the clients reaches the expected prediction accuracy, so that each client obtains a trained machine learning model and makes corresponding predictions.

2. The method according to claim 1, characterized in that, Each client C participating in federated training i Protecting the gradient information with the transceiver noise difference in the first communication: In the formula: For client C i Total noise transmitted, For client C i Total noise received; Indicates client C i The "outer neighbor", that is, the one that receives the client C i The neighbor to whom the message was sent; Indicates client C i The "inner neighbor", that is, client C i The neighbor who sent the message received it; δ li Indicates each client C i To the external neighbor client C l Transmitted random noise, δ ij Indicates client C i Received internal neighbor client C j Random noise; Attached to the gradient tracking variable on the local client for protection.

3. The method according to claim 1, characterized in that, The gradient tracking variables exchanged by clients participating in federated learning can be protected by the following formula: A t (d S -d R ) In the formula: matrix A = [a ij ] n×n Let a be a double random matrix, where a ij It is client C i For each inner neighbor C j Set the aggregation weight, and ensure that... Similarly, C i For each external neighbor C l Set the aggregate weight a for it li And guarantee Finally, a double random matrix A is constructed; A t Let δ be the power of t of a birandom matrix A. S The client sends a vector composed of noise, that is... δ R The client receives a vector composed of noise, that is...

4. The method according to claim 1, characterized in that, In subsequent rounds, clients participating in federated learning exchange and aggregate model weights: In the formula: In the t-th round of communication, client C i Model weights, Let a be the gradient tracking variable in round t, where t≥1, a ij For aggregate weights, λ is the client C. i The update step size; in round t, each client C i To the external neighbor client C l Send model weights and gradient tracking variables Simultaneously received from the internal neighbor client C j Model weights and gradient tracking variables Then, the weight information of all adjacent clients is aggregated to obtain a new round of model weights.

5. The method according to claim 1, characterized in that, In subsequent rounds, clients participating in federated learning exchange and aggregate gradient tracking variables: In the formula In the t-th round of communication, client C i gradient tracking variables, For client C in the t-th round of communication i Local model gradient values; Since, from a global perspective, aggregating all noise differences can eliminate the globally introduced noise to zero, that is... By utilizing the property that the global sum of gradient differences cancels out to zero, we can ensure that the global tracking property of the gradient tracking variable remains unchanged, that is: From the above formula, we can derive...

6. The method according to claim 1, characterized in that, Also includes: Each client participating in federated learning pre-specifies the total number of communication rounds T, and the training process of federated learning automatically ends after T rounds of communication. Alternatively, the total number of communication rounds can be left unspecified. After each communication round aggregates and updates the model weights, the model's prediction accuracy is tested. If each client achieves the expected accuracy, the federated learning training process can be terminated.

7. A high-precision, privacy-preserving decentralized federated learning device, characterized in that, include: The negotiation module is used by each client participating in the decentralized federated learning process to predetermine the machine learning model structure and parameter framework that need to be trained collaboratively, initialize the model weights and corresponding gradient tracking variables based on its own local dataset, and then determine the aggregation weights of neighboring clients according to its own importance to neighboring clients on the communication topology graph. The noise-adding module is used to determine the parameters required for the training process locally. After each client starts communicating with its neighboring clients, during the first communication, each client generates noise with a mean of zero completely randomly and sends pure noise to its neighboring clients. At the same time, it needs to receive the noise sent by its neighboring clients. Then, each client performs the subtraction of the transmitted and received noise and appends the difference to its local gradient tracking variable to update the gradient tracking variable. The aggregation module is used in subsequent communications, where each client sends model weights and gradient tracking variables to neighboring clients, aggregates the corresponding weights and gradient tracking variables received from neighboring clients, and updates its own weights and gradient tracking variables by combining the model gradient values ​​trained on the local dataset. The testing module is used to continuously exchange and aggregate model weights and gradient tracking variables in subsequent rounds until the model jointly trained by the clients reaches the expected prediction accuracy, so that each client obtains a trained machine learning model and makes corresponding predictions.

8. An electronic device, characterized in that, include: One or more processors; Memory, used to store one or more programs; When the one or more programs are executed by the one or more processors, the one or more processors implement the method as described in any one of claims 1-6.

9. A computer-readable storage medium storing computer instructions thereon, characterized in that, When executed by the processor, this instruction implements the steps of the method as described in any one of claims 1-6.

Images