High-model-precision anti-poisoning-attack decentration federated learning method and device

Abstract

The invention discloses a high-model-precision anti-poisoning attack decentration federal learning method and a high-model-precision anti-poisoning attack decentration federal learning device. The method comprises the following steps that: each client participating in federated learning initializes a model weight, a gradient tracking variable and an aggregation weight for a neighbor client, and a benign client initializes a record variable for recording the gradient contribution of the neighbor client; the benign client aggregates the received model weight and gradient tracking variable, trains the model and updates the model parameter, updates the record variable and adjusts the aggregation weight of the neighbor client based on consistency check; the malicious client side aggregates the received model weight and the gradient tracking variable, trains the model and injects the malicious gradient into model parameters; a benign client gradually adjusts the aggregation weight of a malicious neighbor client to zero, instantaneously clears the historical gradient contribution of the malicious neighbor client based on a record variable, and optimizes the polluted model weight based on benign gradient aggregation, so that the beneficial information from the malicious client is effectively reserved, and the model precision is improved.

Description

Technical Field

[0001] The present invention relates to the technical field of federated learning, and in particular to a high-model-precision, poison-attack-resistant decentralized federated learning method and device. Background Art

[0002] In the context of the rapid development of artificial intelligence, training machine learning models from massive data has become a key link. The machine learning process usually begins with the data collection stage, which is often stored in a data processing center. Then, the center's computing cluster uses algorithms such as stochastic gradient descent to train the model and optimize its model weights. After training, the model will be deployed to the server for prediction tasks. However, this process has certain hidden dangers: the collection of a large amount of end-user data not only consumes valuable communication resources, but also makes the centrally stored data vulnerable to attacks, posing the risk of user privacy leakage.

[0003] In order to solve the problems caused by the traditional data collection model, Federated Learning, as an emerging technology, provides a solution. Federated Learning aggregates the model weights and gradients obtained by each end user on local data through a central server, thereby collaboratively training the model. This method effectively avoids the direct transmission of user data and becomes a training paradigm that protects privacy. However, traditional federated learning relies on a central server. Once the server fails, the entire system will be paralyzed, resulting in training interruption.

[0004] To overcome this centralized limitation, decentralized federated learning came into being. In decentralized federated learning, multiple clients (i.e., data owners) achieve collaborative model training by aggregating model parameters and gradients of neighboring clients based on communication topology. This approach significantly reduces the risk of system paralysis due to single point failure.

[0005] The advantages of decentralized federated learning have been verified in many practical applications, especially in the fields of finance, healthcare, etc. For example, in the financial industry, there is a lack of a unified central platform between multiple banks and other financial institutions. Through decentralized federated learning, they can use their local data to jointly train machine learning models without directly exchanging data, thereby improving the accuracy of credit assessment, anti-fraud detection and other businesses, while effectively protecting users' personal privacy.

[0006] Nevertheless, decentralized federated learning relies on information exchange between clients to jointly train models. Once a malicious client sends harmful model gradients to interfere with the aggregation process, decentralized federated learning will fail to converge, resulting in a decrease in the accuracy of the final model (i.e., poisoning attack). For example, when multiple banks jointly train a credit scoring model, if one of the banks deliberately sends tampered model parameters to disrupt the training process for malicious purposes, the prediction results of the credit scoring model will be completely distorted. Such malicious behavior not only affects the performance of the model, but also undermines the trust basis for cooperation among all parties.

[0007] Current anti-poisoning robust aggregation minimizes the impact of malicious parameters by aggregating the median or average of received parameters in each iteration, but this method cannot completely exclude all malicious parameters in each iteration. In addition, the "detect-restart" method identifies all malicious clients in the pre-aggregation process and then restarts the aggregation process of decentralized federated learning between benign clients, which not only wastes a lot of communication and computing resources, but also ignores the valuable information contained in malicious parameters, making it difficult to achieve a high-precision decentralized federated learning model.

[0008] In response to the above problems, it is urgent to develop a decentralized federated learning method with high model accuracy and the ability to resist poisoning attacks. This method should not only be able to defend against a variety of poisoning attacks, but also effectively utilize training data to ensure that the model has highly accurate prediction capabilities. Summary of the invention

[0009] The present invention aims to solve the problem that malicious clients send malicious parameters in decentralized federated learning, resulting in the inability of the aggregation process to converge and the reduction of model accuracy, and to provide a decentralized federated learning method and device with high model accuracy and resistance to poisoning attacks.

[0010] According to a first aspect of an embodiment of the present application, a decentralized federated learning method with high model accuracy and resistance to poisoning attacks is provided, comprising:

[0011] In the initialization phase, each client participating in the decentralized federated learning process determines the structure of the collaboratively trained machine learning model in advance, and randomly initializes the model weights, the corresponding gradient tracking variables, and evenly distributes the aggregate weights to each neighbor client. In order to prevent malicious clients from sending malicious gradients for poisoning attacks, each benign client also needs to initialize a record variable for recording the gradient contribution of each neighbor client.

[0012] During the aggregation process, each benign client exchanges model parameters with its neighbor clients, aggregates the received model weights and gradient tracking variables, and performs model training in combination with the local data set, thereby updating its own model weights and gradient tracking variables. At the same time, the benign client updates the recorded variables of the neighbor clients according to the received gradient tracking variables, and adjusts the aggregation weights of each neighbor client based on the consistency check;

[0013] During the aggregation process, each malicious client also exchanges model parameters with its neighboring clients, aggregates the received model weights and gradient tracking variables, and trains the model in combination with the local malicious dataset, thereby injecting malicious gradients into its own model weights and gradient tracking variables, interfering with the aggregation process of benign clients and reducing the final model accuracy;

[0014] When the aggregation weight of malicious neighbor clients is adjusted to zero based on consistency check, each benign client immediately clears all malicious gradient contributions of the corresponding malicious neighbor clients according to the recorded variables. Then, the benign clients optimize the contaminated model weights according to the normal gradient aggregation process. The beneficial parts from the malicious clients are retained during the optimization process, thereby effectively improving the model accuracy.

[0015] According to a second aspect of an embodiment of the present application, a high-model-precision anti-poisoning attack decentralized federated learning device is provided, comprising:

[0016] Configuration module, used for the initialization phase, each client participating in the decentralized federated learning process to determine the structure of the collaboratively trained machine learning model in advance, and to randomly initialize the model weights, the corresponding gradient tracking variables, and evenly distribute the aggregate weights to each neighbor client. In order to prevent malicious clients from sending malicious gradients for poisoning attacks, each benign client also needs to initialize the recording variables used to record the gradient contribution of each neighbor client;

[0017] Benign module, used for each benign client to exchange model parameters with its neighbor clients during the aggregation process, aggregate the received model weights and gradient tracking variables, and perform model training in combination with the local data set, thereby updating its own model weights and gradient tracking variables. At the same time, the benign client updates the record variables of the neighbor clients according to the received gradient tracking variables, and adjusts the aggregation weight of each neighbor client based on the consistency check;

[0018] The malicious module is used to exchange model parameters with its neighbor clients during the aggregation process, aggregate the received model weights and gradient tracking variables, and train the model in combination with the local malicious dataset, thereby injecting malicious gradients into its own model weights and gradient tracking variables, interfering with the aggregation process of benign clients and reducing the final model accuracy;

[0019] The optimization module is used to adjust the aggregation weight of malicious neighbor clients to zero based on consistency check. Each benign client immediately clears all malicious gradient contributions of the corresponding malicious neighbor clients according to the recorded variables. Then, the benign client optimizes the contaminated model weights according to the normal gradient aggregation process. The beneficial parts from the malicious clients are retained during the optimization process, thereby effectively improving the model accuracy.

[0020] According to a third aspect of an embodiment of the present application, there is provided an electronic device, characterized in that it includes:

[0021] one or more processors;

[0022] A memory for storing one or more programs;

[0023] When the one or more programs are executed by the one or more processors, the one or more processors implement the method as described in the first aspect.

[0024] According to a fourth aspect of an embodiment of the present application, a computer-readable storage medium is provided, on which computer instructions are stored, and when the instructions are executed by a processor, the steps of the method described in the first aspect are implemented.

[0025] Beneficial effects:

[0026] This application is different from the robust aggregation method and the "detection-restart" method of the existing decentralized federated learning method to resist poisoning attacks, but is a decentralized federated learning method based on high model accuracy to resist poisoning attacks. This method uses record variables to enable benign clients to record the gradient contribution of each neighbor client from the beginning of aggregation, and uses the consistency of the gradient to gradually adjust the aggregation weight of the neighbor client. Once the aggregation weight is adjusted to zero, all historical gradient contributions of the malicious neighbor client are immediately cleared. Because the aggregation weight is adjusted step by step, it is possible to more robustly identify malicious neighbor clients and retain benign neighbor clients. The existence of the record variable enables the benign client to accurately and completely eliminate malicious gradients. In addition, after eliminating the malicious gradient, this application uses the gradient aggregation between benign clients to optimize the contaminated model weights. This optimization process retains the beneficial parts previously submitted by the malicious client, and effectively uses the data of the malicious client to improve the model accuracy. This method does not introduce communication overhead, and only needs to maintain an additional record variable for each neighbor client locally, which can effectively prevent a variety of poisoning attacks and effectively improve the prediction accuracy of the model. This method can be widely used in various decentralized federated learning systems to achieve dual guarantees of accuracy and security, and is helpful for joint model training in fields such as finance and medicine. BRIEF DESCRIPTION OF THE DRAWINGS

[0027] Figure 1A flowchart of a decentralized federated learning method with high model accuracy and resistance to poisoning attacks provided in an embodiment of the present invention.

[0028] Figure 2 A schematic diagram of a system model for applying the decentralized federated learning method provided in an embodiment of the present invention.

[0029] Figure 3 A graph showing the prediction accuracy of a convolutional neural network model for the MNIST dataset under resistance to targeted poisoning attacks in the decentralized federated learning method provided in an embodiment of the present invention.

[0030] Figure 4 A graph showing the prediction accuracy of a convolutional neural network model for the MNIST dataset against untargeted poisoning attacks in the decentralized federated learning method provided in an embodiment of the present invention.

[0031] Figure 5 A block diagram of a decentralized federated learning device with high model accuracy and resistance to poisoning attacks provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0032] The following is a clear and complete description of the technical solutions in the embodiments of the present invention in combination with the specific content of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, rather than all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work belong to the protection scope of the present invention. The contents not described in detail in the embodiments of the present invention belong to the prior art known to professional and technical personnel in this field.

[0033] The embodiment of the present invention provides a high-model-precision decentralized federated learning method that is resistant to poisoning attacks, which can be widely used in decentralized federated learning scenarios with high precision and resistance to poisoning attacks. In this scenario, a benign client maintains a record variable for each of its neighbor clients to record all historical gradient contributions of each neighbor client. At the same time, the benign client gradually adjusts the aggregation weight of the neighbor client through the consistency of the gradient tracking variable. Once the aggregation weight is adjusted to zero, the benign client immediately clears all historical gradient contributions of the corresponding neighbor client. Since the present invention only removes malicious influences in the model gradient and uses benign gradient aggregation to optimize the contaminated model weights, it can effectively retain the useful information in the parameters previously submitted by the malicious client, thereby greatly improving the model accuracy. The high precision and security of this method are both theoretically guaranteed and experimentally explained to illustrate its effectiveness.

[0034] The method of the present invention is described in detail below in conjunction with this scenario.

[0035] Figure 1is a flowchart of a decentralized federated learning method for resisting poisoning attacks with high model accuracy according to an exemplary embodiment. Figure 1 As shown, the method comprises the following steps:

[0036] S1: In the initialization phase, each client participating in the decentralized federated learning process determines the structure of the collaboratively trained machine learning model in advance, and randomly initializes the model weights, the corresponding gradient tracking variables, and evenly distributes the aggregate weights to each neighbor client. In order to prevent malicious clients from sending malicious gradients for poisoning attacks, each benign client also needs to initialize a record variable for recording the gradient contribution of each neighbor client;

[0037] Specifically, during the parameter initialization process of the above method, each client must negotiate with its neighbors and initialize its own parameters according to its own conditions, including:

[0038] use Indicates the total number of clients participating in decentralized federated learning, Participation Clients; use Indicates the model weight of the client, using Represents the model gradient value of the client, using Indicates the gradient tracking variable corresponding to the client, using Indicates the update step of the client; Represents the client The set of neighbors of . For each neighbor , Client Set the initial aggregation weight for it ,in Represents the client The total number of neighbor clients, where .Depend on The composed matrix is ​​a row random matrix, which can ensure the parameter consistency of the final training model of federated learning.

[0039] Each client Need to randomly initialize model weights , initialize the gradient tracking variable according to the local model gradient , the gradient tracking variable is used to continuously track the global gradient in decentralized federated learning, that is, .

[0040] Each benign client also needs to initialize the record variable used to record the gradient contribution of each neighbor client, that is, initialize .

[0041] S2: During the aggregation process, each malicious client also exchanges model parameters with its neighbor clients, aggregates the received model weights and gradient tracking variables, and trains the model in combination with the local malicious dataset, thereby injecting malicious gradients into its own model weights and gradient tracking variables, interfering with the aggregation process of benign clients and reducing the final model accuracy;

[0042] Specifically, each benign client needs to update the record variables and aggregation weights of each neighbor client in each iteration, and also needs to update its own model weights and gradient tracking variables, including:

[0043] Update the record variables for each neighbor client:

[0044]

[0045] Where: For benign clients To its neighbor clients In the The record variable when the aggregation is iterated, Is a benign client To its neighbor clients In the The aggregation weight during the iteration aggregation, Is the client In the Gradient tracking variable during iteration aggregation. Represents a benign client Receiving neighbor clients The gradient aggregation variable is aggregated to the local gradient tracking variable according to the corresponding aggregation weight The value of the client For Clients In the The gradient contribution at iteration . In fact, it represents the period from the beginning of aggregation to the In the iteration, the client For Clients All historical gradient contributions of .

[0046] Update the aggregate weight for each neighbor client:

[0047]

[0048] Where: Is a benign client To its neighbor clients In the The aggregation weight assigned during the iteration aggregation, Is the client In the Gradient tracking variable when aggregating over iterations, Is the client In the Gradient tracking variable when aggregating over iterations, is the total number of clients participating in decentralized federated learning. From the adjustment of the aggregation weight in the above formula, it can be seen that each benign client adopts two normalization processes to comprehensively consider all neighboring clients in the first The gradient of each iteration tracks the variable information and the historical aggregate weight information.

[0049] Update its own model weights:

[0050]

[0051] Where: Indicates Client during round iteration Local model weights, Indicates The client Local gradient tracking variables. Is the client To its neighbor clients In the The aggregation weight assigned at the iteration, For Clients The update step size is . In each iteration, each client To neighbor clients Send model weights and gradient tracking variables , while receiving neighbor clients The model weight and gradient tracking variables ; Then, the weights of all adjacent clients are aggregated according to the formula to obtain the new round of model weights .

[0052] Update its own gradient tracking variables:

[0053]

[0054] Where: Indicates The client The gradient tracking variable of For the The client Local model gradient value; is a signal function that outputs 1 when the aggregate weight is greater than 0 and outputs 0 when the aggregate weight is equal to 0. Malicious Neighbor Client Aggregation weight When adjusted to 0, The output is 0, then Will abandon the client All historical gradient contributions , that is, instantly clear malicious clients All historical malicious influences.

[0055] The beneficial effects of the above updates are reflected in the formulas and convergence analysis as follows, including the consistency of gradient aggregation and the global tracking of gradient tracking variables, including:

[0056] Consistency of gradient aggregation: When all malicious clients are detected and eliminated, the gradient aggregation of benign clients will be transformed into the following form:

[0057]

[0058] in is the total number of benign clients. From the above formula, we can see that:

[0059]

[0060] Subtracting the two equations yields:

[0061]

[0062] The above formula is the standard form of gradient tracking variable aggregation in the normal decentralized federated learning process, so the present invention can ensure that the convergence between benign clients after removing malicious clients is the same as the convergence without malicious client aggregation. And because the present invention uses the gradient aggregation of benign clients to iteratively optimize the contaminated model weights, it will retain the useful information submitted by malicious clients in the model weights, thereby effectively improving the final model accuracy.

[0063] Global tracking of gradient tracking variables:

[0064]

[0065] From the above formula, we can deduce , that is, the global tracking property of the gradient tracking variable is not affected under the present invention, which further ensures the convergence of the aggregation process.

[0066] S3: During the aggregation process, each malicious client also exchanges model parameters with its neighbor clients, aggregates the received model weights and gradient tracking variables, and trains the model in combination with the local malicious dataset, thereby injecting malicious gradients into its own model weights and gradient tracking variables, interfering with the aggregation process of benign clients and reducing the final model accuracy;

[0067] Specifically, each benign client needs to generate malicious gradients in each iterative update and inject the malicious gradients into its own model weights and gradient tracking variables, including:

[0068] Update its own model weights:

[0069]

[0070] Where: is the aggregation weight of the malicious client to its neighbor clients. Since the malicious client does not update the aggregation weight of its neighbor clients, the aggregation weight is always .

[0071] Based on the updated model weights, the model is trained on the local malicious dataset and back-propagated to obtain the malicious gradient , and then inject the malicious gradient into the gradient tracking variable:

[0072]

[0073] in For malicious clients No. The local malicious gradient at the iteration, For malicious clients No. The local normal gradient at iteration , The malicious degree of the injected malicious gradient. The malicious client only injects a part of the malicious gradient value into the gradient tracking variable, and then uses this value to indirectly poison the model weight. Because there is a corresponding relationship between the model weight and the gradient tracking variable, if any poisoning interference is performed on both at the same time, it is easy to be detected. The process of poisoning the gradient tracking variable and indirectly poisoning the model weight can increase the difficulty of benign client detection and increase the success rate of poisoning attacks.

[0074] S4: When the aggregation weight of malicious neighbor clients is adjusted to zero based on consistency check, each benign client immediately clears all malicious gradient contributions of the corresponding malicious neighbor clients according to the recorded variables. Then, the benign clients optimize the contaminated model weights according to the normal gradient aggregation process. The beneficial parts from the malicious clients are retained during the optimization process, thereby effectively improving the model accuracy.

[0075] Specifically, each client participating in decentralized federated learning can specify a total number of iterations in advance. ,exist The training process of federated learning ends automatically after rounds of iteration (all malicious clients have been detected and excluded in the middle of round T); the total number of iterations can also be unspecified. After detecting and excluding all malicious clients, the prediction accuracy of the model is tested each time the model weights are aggregated and updated. If each client achieves the expected accuracy, the training process can be ended. This method can ensure that the model finally trained by each participating client not only effectively utilizes its own local data, but also can utilize the valuable data in malicious clients. At the same time, it also ensures the security of the training process, can encourage more high-value data owners to participate in training, and promote the circulation of data elements and the construction of high-precision models.

[0076] The decentralized federated learning method of the present invention is implemented on an Ubuntu 18.04 system running on an Intel core 2.80 GHz server with 192GB of memory, and the total number of iterations is preset to be .

[0077] Figure 2 A schematic diagram of the system model for the application of the decentralized federated learning method provided in the embodiment of the present invention. Each benign client of the decentralized federated learning needs to do two things at the same time: "detection" and "recording". The benign client needs to detect the consistency of the received gradient tracking variables and adjust the aggregation weight of the neighboring clients according to the consistency. The benign client also needs to maintain a record variable to record all the historical gradient contributions of each neighboring client from the beginning of aggregation to the present. Once the aggregation weight is reduced to 0, all the historical malicious influences of all malicious neighboring clients are immediately eliminated by eliminating the record variable.

[0078] The performance results of the high-model-precision anti-poisoning attack decentralized federated learning method (i.e., GPD mechanism) proposed in this invention are analyzed through simulation experiments. Figure 3 , Figure 4 The simulation experiment compared and analyzed the most advanced "targeted poisoning attack" technology ( Figure 3 ) and "untargeted poisoning attack" technology ( Figure 4), and the most advanced defense methods (Lead, WBC, LDP methods). At the same time, Upper represents the model accuracy when only benign clients are aggregated, and Lower represents the model accuracy when there is no defense algorithm aggregation. Under both targeted poisoning attacks and non-targeted poisoning attacks, the method provided by the present invention (GPD) shows the best model accuracy. It is worth noting that the method provided by the present invention shows a model accuracy that is not inferior to Upper in the following, which proves that the method provided by the present invention can effectively use the useful information provided by malicious clients to improve the model accuracy, thereby making full use of the training data.

[0079] Corresponding to the above-mentioned embodiment of the high model accuracy anti-poisoning attack decentralized federated learning method, the present application also provides an embodiment of the high model accuracy anti-poisoning attack decentralized federated learning method. Figure 5 , the device comprises:

[0080] Configuration module 1 is used to determine the structure of the collaboratively trained machine learning model in advance for each client participating in the decentralized federated learning process during the initialization phase, and to randomly initialize the model weights, the corresponding gradient tracking variables, and evenly distribute the aggregate weights to each neighbor client. In order to prevent malicious clients from sending malicious gradients for poisoning attacks, each benign client also needs to initialize a record variable for recording the gradient contribution of each neighbor client;

[0081] Benign module 2, used for each benign client to exchange model parameters with its neighbor clients during the aggregation process, aggregate the received model weights and gradient tracking variables, and perform model training in combination with the local data set, so as to update its own model weights and gradient tracking variables. At the same time, the benign client updates the record variables of the neighbor clients according to the received gradient tracking variables, and adjusts the aggregation weight of each neighbor client based on the consistency check;

[0082] Malicious module 3, used in the aggregation process, each malicious client exchanges model parameters with its neighbor clients, aggregates the received model weights and gradient tracking variables, and performs model training in combination with the local malicious data set, thereby injecting malicious gradients into its own model weights and gradient tracking variables, interfering with the aggregation process of benign clients and reducing the final model accuracy;

[0083] Optimization module 4 is used to adjust the aggregation weight of malicious neighbor clients to zero based on consistency check. Each benign client immediately clears all malicious gradient contributions of the corresponding malicious neighbor clients according to the recorded variables. Then, the benign client optimizes the contaminated model weights according to the normal gradient aggregation process. The beneficial parts from the malicious clients are retained during the optimization process, thereby effectively improving the model accuracy.

[0084] For the device embodiment, since it basically corresponds to the method embodiment, the relevant parts can refer to the partial description of the method embodiment. The device embodiment described above is only schematic, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the present application scheme. A person of ordinary skill in the art can understand and implement it without paying any creative work.

[0085] Correspondingly, the present application also provides an electronic device, comprising: one or more processors; a memory for storing one or more programs; when the one or more programs are executed by the one or more processors, the one or more processors implement the method as described in the first aspect; the present application also provides a computer-readable storage medium, on which computer instructions are stored, characterized in that when the instructions are executed by the processor, the steps of the high-model-precision anti-poisoning attack decentralized federated learning method as described above are implemented.

[0086] It should be understood that the present application is not limited to the precise structures that have been described above and shown in the drawings, and that various modifications and changes may be made without departing from the scope thereof. The scope of the present application is limited only by the appended claims.

Claims

1. A decentralized federated learning method with high model accuracy and resistance to poisoning attacks, characterized in that: include: In the initialization phase, each client participating in the decentralized federated learning process determines the structure of the collaboratively trained machine learning model in advance, and randomly initializes the model weights, the corresponding gradient tracking variables, and evenly distributes the aggregate weights to each neighbor client. In order to prevent malicious clients from sending malicious gradients for poisoning attacks, each benign client also needs to initialize a record variable for recording the gradient contribution of each neighbor client. During the aggregation process, each benign client exchanges model parameters with its neighbor clients, aggregates the received model weights and gradient tracking variables, and performs model training in combination with the local data set, thereby updating its own model weights and gradient tracking variables. At the same time, the benign client updates the recorded variables of the neighbor clients according to the received gradient tracking variables, and adjusts the aggregation weights of each neighbor client based on the consistency check; During the aggregation process, each malicious client also exchanges model parameters with its neighboring clients, aggregates the received model weights and gradient tracking variables, and trains the model in combination with the local malicious dataset, thereby injecting malicious gradients into its own model weights and gradient tracking variables, interfering with the aggregation process of benign clients and reducing the final model accuracy; When the aggregation weight of malicious neighbor clients is adjusted to zero based on consistency check, each benign client immediately clears all malicious gradient contributions of the corresponding malicious neighbor clients according to the recorded variables. Then, the benign clients optimize the contaminated model weights according to the normal gradient aggregation process. The beneficial parts from the malicious clients are retained during the optimization process, thereby effectively improving the model accuracy.

2. The method according to claim 1, characterized in that Each benign client also needs to initialize the record variables used to record the gradient contribution of each neighbor client, including: Where: For Clients For each neighbor client The initialized record variable, Is the client The set of all neighbor clients.

3. The method according to claim 1, characterized in that The benign client updates the recorded variables of the neighbor client according to the received gradient tracking variables, including: Where: For benign clients To its neighbor clients In the The record variable when the aggregation is iterated, Is a benign client To its neighbor clients In the The aggregation weight during the iteration aggregation, Is the client In the Gradient tracking variable when aggregating over iterations, For benign clients Receiving neighbor clients The gradient tracking variable is then aggregated to the local gradient tracking variable according to the corresponding aggregation weight The value of .

4. The method according to claim 1, characterized in that: Benign clients adjust the aggregation weight of each neighbor client based on consistency checks, including: Where: Is a benign client To its neighbor clients In the The aggregation weight assigned during the iteration aggregation, Is the client In the Gradient tracking variable when aggregating over iterations, Is the client In the Gradient tracking variable when aggregating over iterations, The total number of clients participating in decentralized federated learning.

5. According to the method of claim 1, each benign client exchanges model parameters with its neighbor clients, aggregates the received model weights and gradient tracking variables, and performs model training in combination with the local data set, thereby updating its own model weights and gradient tracking variables, specifically including: Where: Indicates Client during round iteration Local model weights, Indicates The client Local gradient tracking variables, Is the client To its neighbor clients In the The aggregation weight assigned at the iteration, For Clients The update step size is For the The client The local model gradient value, It is a signal function that outputs 1 when the aggregation weight is greater than 0 and outputs 0 when the aggregation weight is equal to 0.

6. The method according to claim 1, characterized in that Each malicious client also exchanges model parameters with its neighbor clients, aggregates the received model weights and gradient tracking variables, and trains the model with the local malicious dataset, thereby injecting malicious gradients into its own model weights and gradient tracking variables, including: in For malicious clients No. The local malicious gradient at the iteration, For malicious clients No. The local normal gradient at iteration , Indicates the maliciousness of the injected malicious gradient.

7. The method according to claim 1, characterized in that Also includes: Each client participating in federated learning specifies the total number of iterations in advance ,exist The training process of federated learning ends automatically after one round of iteration; Or do not specify the total number of iterations. After detecting and eliminating all malicious clients, test the prediction accuracy of the model each time the model weights are updated in an aggregated manner. If each benign client achieves the expected accuracy, the federated learning training process can be terminated.

8. A decentralized federated learning device with high model accuracy and resistance to poisoning attacks, characterized in that: include: Configuration module, used for the initialization phase, each client participating in the decentralized federated learning process to determine the structure of the collaboratively trained machine learning model in advance, and to randomly initialize the model weights, the corresponding gradient tracking variables, and evenly distribute the aggregate weights to each neighbor client. In order to prevent malicious clients from sending malicious gradients for poisoning attacks, each benign client also needs to initialize the recording variables used to record the gradient contribution of each neighbor client; Benign module, used for each benign client to exchange model parameters with its neighbor clients during the aggregation process, aggregate the received model weights and gradient tracking variables, and perform model training in combination with the local data set, thereby updating its own model weights and gradient tracking variables. At the same time, the benign client updates the record variables of the neighbor clients according to the received gradient tracking variables, and adjusts the aggregation weight of each neighbor client based on the consistency check; The malicious module is used to exchange model parameters with its neighbor clients during the aggregation process, aggregate the received model weights and gradient tracking variables, and train the model in combination with the local malicious dataset, thereby injecting malicious gradients into its own model weights and gradient tracking variables, interfering with the aggregation process of benign clients and reducing the final model accuracy; The optimization module is used to adjust the aggregation weight of malicious neighbor clients to zero based on consistency check. Each benign client immediately clears all malicious gradient contributions of the corresponding malicious neighbor clients according to the recorded variables. Then, the benign client optimizes the contaminated model weights according to the normal gradient aggregation process. The beneficial parts from the malicious clients are retained during the optimization process, thereby effectively improving the model accuracy.

9. An electronic device, characterized in that: include: one or more processors; A memory for storing one or more programs; When the one or more programs are executed by the one or more processors, the one or more processors implement the method according to any one of claims 1 to 8.

10. A computer-readable storage medium having computer instructions stored thereon, characterized in that: When the instruction is executed by a processor, the steps of the method according to any one of claims 1 to 8 are implemented.

Images