High-model-accuracy decentralized federated learning method and device with resistance to poisoning attacks

Abstract

The present invention discloses a high-model-precision, poison-attack-resistant decentralized federated learning method and device. The method comprises: each client participating in federated learning initializes model weights, gradient tracking variables, and aggregate weights for neighboring clients; a benign client initializes a record variable that records the gradient contributions of its neighboring clients; the benign client aggregates the received model weights and gradient tracking variables, trains the model and updates the model parameters, updates the record variable, and adjusts the aggregate weights for neighboring clients based on a consistency check; a malicious client aggregates the received model weights and gradient tracking variables, trains the model, and injects malicious gradients into the model parameters; the benign client gradually adjusts the aggregate weights of its malicious neighboring clients to zero, instantaneously clears the malicious client's historical gradient contributions based on the record variable, and then optimizes the contaminated model weights based on the benign gradient aggregation, effectively retaining useful information from the malicious client and improving model accuracy.

Description

Technical Field

[0001] The present invention relates to the field of federated learning technology, and in particular to a decentralized federated learning method and device with high model accuracy and resistance to poisoning attacks. Background Art

[0002] Amid the rapid development of artificial intelligence, training machine learning models from massive amounts of data has become a critical step. The machine learning process typically begins with data collection, which is often centrally stored in a data processing center. The central computing cluster then trains the model using algorithms such as stochastic gradient descent to optimize its weights. Once trained, the model is deployed to servers for prediction tasks. However, this process presents certain risks: collecting large amounts of end-user data not only consumes valuable communication resources but also makes the centrally stored data vulnerable to attacks, posing the risk of user privacy breaches.

[0003] Federated learning, an emerging technology, offers a solution to the problems associated with traditional data collection models. Federated learning uses a centralized server to aggregate model weights and gradients trained on local data by each end user, enabling collaborative model training. This approach effectively avoids the direct transmission of user data, making it a privacy-preserving training paradigm. However, traditional federated learning relies on a centralized server. A server failure can paralyze the entire system, disrupting training.

[0004] To overcome this centralization limitation, decentralized federated learning (DFL) emerged. In DFL, multiple clients (data owners) collaborate on model training by aggregating model parameters and gradients from neighboring clients based on a communication topology. This approach significantly reduces the risk of system failure due to a single point of failure.

[0005] The advantages of decentralized federated learning have been demonstrated in numerous real-world applications, particularly in finance, healthcare, and other fields. For example, in the financial industry, multiple banks and other financial institutions lack a unified central platform. Through decentralized federated learning, these institutions can leverage their respective local data to jointly train machine learning models without directly exchanging data. This improves the accuracy of credit assessments, anti-fraud detection, and other services, while effectively protecting user privacy.

[0006] However, decentralized federated learning relies on information exchange between clients to jointly train models. If a malicious client sends harmful model gradients to disrupt the aggregation process, decentralized federated learning will fail to converge, resulting in a decrease in the accuracy of the final model (a poisoning attack). For example, when multiple banks are jointly training a credit scoring model, if one bank intentionally sends tampered model parameters to disrupt the training process for malicious purposes, the credit scoring model's predictions will be completely distorted. Such malicious behavior not only affects model performance but also undermines the trust that underpins the collaboration among all parties.

[0007] Current poisoning-resistant robust aggregation minimizes the impact of malicious parameters by aggregating the median or mean of received parameters in each iteration. However, this approach cannot completely eliminate all malicious parameters in each iteration. Furthermore, the "detect-and-restart" approach identifies all malicious clients during the pre-aggregation process and then restarts the decentralized federated learning aggregation process between benign clients. This not only wastes significant communication and computing resources but also ignores the valuable information contained in the malicious parameters, making it difficult to achieve a high-precision decentralized federated learning model.

[0008] To address these issues, there is an urgent need to develop a decentralized federated learning approach that delivers high model accuracy and is resistant to poisoning attacks. This approach should not only be able to defend against various poisoning attacks but also effectively utilize training data to ensure the model's highly accurate prediction capabilities. Summary of the Invention

[0009] The present invention aims to solve the problem in decentralized federated learning that malicious clients send malicious parameters, resulting in the inability of the aggregation process to converge and the reduction of model accuracy, and provide a decentralized federated learning method and device with high model accuracy and resistance to poisoning attacks.

[0010] According to a first aspect of an embodiment of the present application, a decentralized federated learning method with high model accuracy and resistance to poisoning attacks is provided, comprising:

[0011] During the initialization phase, each client participating in the decentralized federated learning process determines the structure of the collaboratively trained machine learning model in advance, randomly initializes the model weights and corresponding gradient tracking variables, and evenly distributes the aggregate weights to each neighboring client. To prevent malicious clients from sending malicious gradients for poisoning attacks, each benign client also needs to initialize a recording variable used to record the gradient contribution of each neighboring client.

[0012] During the aggregation process, each benign client exchanges model parameters with its neighboring clients, aggregates the received model weights and gradient tracking variables, and performs model training based on the local dataset to update its own model weights and gradient tracking variables. At the same time, the benign client updates the recorded variables of the neighboring clients based on the received gradient tracking variables and adjusts the aggregation weights of each neighboring client based on consistency checks.

[0013] During the aggregation process, each malicious client also exchanges model parameters with its neighboring clients, aggregates the received model weights and gradient tracking variables, and trains the model with the local malicious dataset. This injects malicious gradients into its own model weights and gradient tracking variables, interfering with the aggregation process of benign clients and reducing the final model accuracy.

[0014] When the aggregation weight of a malicious neighbor client is adjusted to zero based on a consistency check, each benign client immediately clears all malicious gradient contributions of the corresponding malicious neighbor client according to the recorded variables. Then, the benign client optimizes the contaminated model weights according to the normal gradient aggregation process. The beneficial parts from the malicious client are retained during the optimization process, thereby effectively improving the model accuracy.

[0015] According to a second aspect of an embodiment of the present application, a decentralized federated learning device with high model accuracy and resistance to poisoning attacks is provided, comprising:

[0016] The configuration module is used to determine the collaborative machine learning model structure in advance during the initialization phase for each client participating in the decentralized federated learning process. It also randomly initializes the model weights and corresponding gradient tracking variables, and evenly distributes the aggregate weights to each neighboring client. To prevent malicious clients from sending malicious gradients for poisoning attacks, each benign client also needs to initialize a recording variable for recording the gradient contribution of each neighboring client.

[0017] The benign module is used to exchange model parameters with its neighboring clients during the aggregation process, aggregate the received model weights and gradient tracking variables, and perform model training on the local dataset to update its own model weights and gradient tracking variables. At the same time, the benign client updates the recorded variables of the neighboring clients based on the received gradient tracking variables and adjusts the aggregation weight of each neighboring client based on consistency checks.

[0018] The malicious module is used to exchange model parameters with its neighboring clients during the aggregation process, aggregate the received model weights and gradient tracking variables, and train the model with the local malicious dataset. This injects malicious gradients into its own model weights and gradient tracking variables, interfering with the aggregation process of benign clients and reducing the final model accuracy.

[0019] The optimization module is used to adjust the aggregation weight of malicious neighbor clients to zero based on consistency checks. Each benign client immediately clears all malicious gradient contributions of the corresponding malicious neighbor clients according to the recorded variables. Then, the benign client optimizes the contaminated model weights according to the normal gradient aggregation process. The optimization process retains the beneficial parts from the malicious clients, thereby effectively improving the model accuracy.

[0020] According to a third aspect of an embodiment of the present application, there is provided an electronic device, characterized by comprising:

[0021] one or more processors;

[0022] a memory for storing one or more programs;

[0023] When the one or more programs are executed by the one or more processors, the one or more processors implement the method as described in the first aspect.

[0024] According to a fourth aspect of an embodiment of the present application, a computer-readable storage medium is provided, on which computer instructions are stored. When the instructions are executed by a processor, the steps of the method described in the first aspect are implemented.

[0025] Beneficial effects:

[0026] This application is different from the robust aggregation method and "detection-restart" method of the existing decentralized federated learning method to resist poisoning attacks. Instead, it is a decentralized federated learning method that resists poisoning attacks based on high model accuracy. This method uses record variables to enable benign clients to record the gradient contribution of each neighbor client from the beginning of aggregation, and uses the consistency of gradients to gradually adjust the aggregation weight of the neighbor client. Once the aggregation weight is adjusted to zero, all historical gradient contributions of the malicious neighbor client are immediately cleared. Because the aggregation weight is adjusted gradually, it is possible to more robustly identify malicious neighbor clients and retain benign neighbor clients. The existence of the record variable enables benign clients to accurately and completely eliminate malicious gradients. In addition, after eliminating the malicious gradients, this application uses gradient aggregation between benign clients to optimize the contaminated model weights. This optimization process retains the beneficial parts previously submitted by the malicious client, and effectively utilizes the data of the malicious client to improve the model accuracy. This method does not introduce communication overhead. It only requires the local maintenance of an additional record variable for each neighbor client, which can effectively prevent multiple poisoning attack methods and effectively improve the prediction accuracy of the model. This method can be widely used in various decentralized federated learning systems to achieve dual guarantees of accuracy and security, and is helpful for joint model training in fields such as finance and healthcare. BRIEF DESCRIPTION OF THE DRAWINGS

[0027] Figure 1A flowchart of a decentralized federated learning method with high model accuracy and resistance to poisoning attacks provided by an embodiment of the present invention.

[0028] Figure 2 Schematic diagram of a system model for applying the decentralized federated learning method provided in an embodiment of the present invention.

[0029] Figure 3 A graph showing the prediction accuracy of a convolutional neural network model for the MNIST dataset, while resisting targeted poisoning attacks, in the decentralized federated learning method provided in an embodiment of the present invention.

[0030] Figure 4 A graph showing the prediction accuracy of a convolutional neural network model for the MNIST dataset, while resisting untargeted poisoning attacks, in the decentralized federated learning method provided in an embodiment of the present invention.

[0031] Figure 5 A block diagram of a decentralized federated learning device with high model accuracy and resistance to poisoning attacks provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0032] The following is a clear and complete description of the technical solutions in the embodiments of the present invention in conjunction with the specific content of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present invention. The contents not described in detail in the embodiments of the present invention belong to the prior art known to professional and technical personnel in this field.

[0033] The embodiment of the present invention provides a decentralized federated learning method with high model accuracy and resistance to poisoning attacks, which can be widely used in decentralized federated learning scenarios with high accuracy and resistance to poisoning attacks. In this scenario, a benign client maintains a record variable for each of its neighbor clients to record all historical gradient contributions of each neighbor client. At the same time, the benign client gradually adjusts the aggregation weight of the neighbor client through the consistency of the gradient tracking variable. Once the aggregation weight is adjusted to zero, the benign client immediately clears all historical gradient contributions of the corresponding neighbor client. Since the present invention only clears the malicious influence in the model gradient and uses benign gradient aggregation to optimize the contaminated model weights, it can effectively retain the useful information in the parameters previously submitted by the malicious client, thereby greatly improving the model accuracy. The high accuracy and security of this method are both theoretically guaranteed and experimentally explained to illustrate its effectiveness.

[0034] The method of the present invention is described in detail below in conjunction with this scenario.

[0035] Figure 1FIG. 1 is a flowchart of a decentralized federated learning method for resisting poisoning attacks with high model accuracy according to an exemplary embodiment. Figure 1 As shown, the method includes the following steps:

[0036] S1: During the initialization phase, each client participating in the decentralized federated learning process determines the structure of the collaboratively trained machine learning model in advance, randomly initializes the model weights, the corresponding gradient tracking variables, and evenly distributes the aggregate weights to each neighboring client. To prevent malicious clients from sending malicious gradients for poisoning attacks, each benign client also initializes a recording variable for recording the gradient contribution of each neighboring client.

[0037] Specifically, during the parameter initialization process described above, each client must negotiate with its neighbors and initialize its own parameters based on its own conditions, including:

[0038] use Indicates the total number of clients participating in decentralized federated learning, using Indicates participation Clients; use Indicates the model weight of the client, using Indicates the model gradient value of the client, using Indicates the gradient tracking variable corresponding to the client, using Indicates the update step of the client; Represents the client The set of neighbors of . For each neighbor , client Set the initial aggregation weight for it ,in Represents the client The total number of neighbor clients, where .Depend on The matrix is ​​a row random matrix, which can ensure the parameter consistency of the final training model of federated learning.

[0039] Each client Need to randomly initialize model weights , initialize the gradient tracking variable according to the local model gradient , the gradient tracking variable is used to continuously track the global gradient in decentralized federated learning, that is, .

[0040] Each benign client also needs to initialize the record variable used to record the gradient contribution of each neighbor client, that is, initialize .

[0041] S2: During the aggregation process, each malicious client also exchanges model parameters with its neighboring clients, aggregates the received model weights and gradient tracking variables, and trains the model with the local malicious dataset. This injects malicious gradients into its own model weights and gradient tracking variables, interfering with the aggregation process of benign clients and reducing the final model accuracy.

[0042] Specifically, each benign client needs to update the record variables and aggregation weights of each neighbor client in each iteration, and also needs to update its own model weights and gradient tracking variables, including:

[0043] Update the record variables for each neighbor client:

[0044]

[0045] Where: For benign clients To its neighbor clients In the Record variables when iterating aggregation, Is a benign client To its neighbor clients In the The aggregation weight during the iterative aggregation, Is the client In the Gradient tracking variable during iteration aggregation. Represents a benign client Receiving neighbor clients The gradient aggregation variable is aggregated to the local gradient tracking variable according to the corresponding aggregation weight The value of the client For the client In the The gradient contribution at iteration . Therefore In fact, it represents the process from the beginning of aggregation to the In the iteration, the client For the client All historical gradient contributions of .

[0046] Update the aggregate weight for each neighbor client:

[0047]

[0048] Where: Is a benign client To its neighbor clients In the The aggregation weight assigned during the iterative aggregation, Is the client In the Gradient tracking variable during iteration aggregation, Is the client In the Gradient tracking variable during iteration aggregation, is the total number of clients participating in decentralized federated learning. From the adjustment of the aggregation weight in the above formula, it can be seen that each benign client takes two normalization processes to comprehensively consider all neighboring clients in the first The gradient tracking variable information of each iteration and the historical aggregate weight information.

[0049] Update its own model weights:

[0050]

[0051] Where: Indicates the Client during round iteration local model weights, Indicates the The client Local gradient tracking variables. Is the client To its neighbor clients In the The aggregation weight assigned at the iteration, For the client The update step size is . In each iteration, each client To neighbor clients Send model weights and gradient tracking variables , while receiving neighbor clients The model weight and gradient tracking variables ; Then, the weights of all adjacent clients are aggregated according to the formula to obtain the new round of model weights .

[0052] Update its own gradient tracking variables:

[0053]

[0054] Where: Indicates the The client Gradient tracking variables, For the The client Local model gradient value; is a signal function that outputs 1 when the aggregate weight is greater than 0 and outputs 0 when the aggregate weight is equal to 0. Malicious Neighbor Client Aggregation weight When adjusted to 0, The output is 0, then Will abandon the client All historical gradient contributions , that is, instantly clear malicious clients All historical malicious influences.

[0055] The beneficial effects of the above updates are reflected in the formulas and convergence analysis as follows, including the consistency of gradient aggregation and the global tracking of gradient tracking variables, including:

[0056] Consistency of gradient aggregation: When all malicious clients are detected and eliminated, the gradient aggregation of benign clients will be transformed into the following form:

[0057]

[0058] in is the total number of benign clients. From the above formula, we can see that:

[0059]

[0060] Subtracting the two equations yields:

[0061]

[0062] The above formula represents the standard form of gradient tracking variable aggregation in normal decentralized federated learning. Therefore, the present invention ensures that the convergence between benign clients after removing malicious clients is the same as when there is no aggregation of malicious clients. Furthermore, because the present invention utilizes the gradient aggregation of benign clients to iteratively optimize the contaminated model weights, it retains the useful information submitted by malicious clients in the model weights, effectively improving the final model accuracy.

[0063] Global tracking of gradient tracking variables:

[0064]

[0065] From the above formula, we can deduce , that is, the global tracking property of the gradient tracking variable is not affected under the present invention, which further ensures the convergence of the aggregation process.

[0066] S3: During the aggregation process, each malicious client also exchanges model parameters with its neighboring clients, aggregates the received model weights and gradient tracking variables, and trains the model with the local malicious dataset. This injects malicious gradients into its own model weights and gradient tracking variables, interfering with the aggregation process of benign clients and reducing the final model accuracy.

[0067] Specifically, each benign client needs to generate malicious gradients in each iterative update and inject the malicious gradients into its own model weights and gradient tracking variables, including:

[0068] Update its own model weights:

[0069]

[0070] Where: is the aggregation weight of the malicious client to its neighbor clients. Since the malicious client does not update the aggregation weight of its neighbor clients, the aggregation weight is always .

[0071] Based on the updated model weights, the model is trained on the local malicious dataset and back-propagated to obtain the malicious gradient. , and then inject malicious gradients into the gradient tracking variable:

[0072]

[0073] in For malicious clients No. The local malicious gradient at the iteration, For malicious clients No. The local normal gradient at iteration , The malicious level of malicious gradient injection. A malicious client injects only a portion of a malicious gradient into the gradient tracking variable, then uses this value to indirectly poison the model weights. Because there is a correspondence between model weights and gradient tracking variables, simultaneous poisoning of both is easily detected. Poisoning the gradient tracking variable and indirectly poisoning the model weights can make it more difficult for benign clients to detect, increasing the success rate of poisoning attacks.

[0074] S4: When the aggregation weight of a malicious neighbor client is adjusted to zero based on the consistency check, each benign client immediately clears all malicious gradient contributions of the corresponding malicious neighbor client according to the recorded variables. Then, the benign client optimizes the contaminated model weights according to the normal gradient aggregation process. The beneficial parts from the malicious client are retained during the optimization process, thereby effectively improving the model accuracy.

[0075] Specifically, each client participating in decentralized federated learning can specify a total number of iterations in advance. ,exist Federated learning training automatically ends after rounds of iterations (all malicious clients have been detected and eliminated midway through round T). Alternatively, a total number of iterations can be omitted. After all malicious clients have been detected and eliminated, the model's prediction accuracy is tested during each aggregate update of the model weights. Training ends if each client achieves the expected accuracy. This method ensures that the model ultimately trained by each participating client not only effectively utilizes its own local data but also leverages valuable data from malicious clients. It also ensures security during the training process, encourages more high-value data owners to participate in training, and promotes the circulation of data elements and the construction of high-precision models.

[0076] The decentralized federated learning method of the present invention is implemented on an Ubuntu 18.04 system running on an Intel core 2.80 GHz server with 192GB of memory, and the total number of iterations is preset to be .

[0077] Figure 2 This is a schematic diagram of the system model for the decentralized federated learning method provided in an embodiment of the present invention. Each benign client in decentralized federated learning needs to perform both "detection" and "recording." Benign clients need to detect the consistency of received gradient tracking variables and adjust the aggregation weights for neighboring clients based on the consistency. Benign clients also need to maintain a record variable that records the historical gradient contributions of each neighboring client from the start of aggregation to the present. Once the aggregate weight is reduced to 0, the record variable is eliminated to immediately eliminate all historical malicious influence of all malicious neighboring clients.

[0078] The performance results of the high-model-precision anti-poisoning attack decentralized federated learning method (i.e., GPD mechanism) proposed in this paper are analyzed through simulation experiments. Figure 3 、 Figure 4 The simulation experiment compared and analyzed the most advanced "targeted poisoning attack" technology ( Figure 3 ) and "untargeted poisoning attack" technology ( Figure 4), as well as state-of-the-art defenses (Lead, WBC, and LDP methods). Upper represents the model accuracy when only benign clients are aggregated, while Lower represents the model accuracy when no defense algorithm is aggregated. The method provided by this invention (GPD) demonstrates optimal model accuracy under both targeted and untargeted poisoning attacks. Notably, the method provided by this invention demonstrates comparable model accuracy to Upper in the following scenario, demonstrating that the method provided by this invention can effectively leverage the beneficial information provided by malicious clients to improve model accuracy, thereby fully utilizing the training data.

[0079] Corresponding to the above-mentioned embodiment of the high model accuracy anti-poisoning attack decentralized federated learning method, the present application also provides an embodiment of the high model accuracy anti-poisoning attack decentralized federated learning method. Figure 5 , the device comprises:

[0080] Configuration Module 1 is used to determine the collaborative machine learning model structure in advance during the initialization phase for each client participating in the decentralized federated learning process. The client then randomly initializes the model weights and corresponding gradient tracking variables, and evenly distributes the aggregate weights to each neighboring client. To prevent malicious clients from sending malicious gradients for poisoning attacks, each benign client also initializes a recording variable for recording the gradient contribution of each neighboring client.

[0081] Benign Module 2: During the aggregation process, each benign client exchanges model parameters with its neighboring clients, aggregates the received model weights and gradient tracking variables, and performs model training based on the local dataset to update its own model weights and gradient tracking variables. At the same time, the benign client updates the recorded variables for the neighboring clients based on the received gradient tracking variables and adjusts the aggregated weight for each neighboring client based on consistency checks.

[0082] Malicious module 3 is used to exchange model parameters with its neighboring clients during the aggregation process, aggregate the received model weights and gradient tracking variables, and train the model with the local malicious dataset. This injects malicious gradients into its own model weights and gradient tracking variables, interfering with the aggregation process of benign clients and reducing the final model accuracy.

[0083] Optimization module 4 is used to adjust the aggregation weight of malicious neighbor clients to zero based on consistency check. Each benign client immediately clears all malicious gradient contributions of the corresponding malicious neighbor clients according to the recorded variables. Then, the benign client optimizes the contaminated model weights according to the normal gradient aggregation process. The beneficial parts from the malicious clients are retained during the optimization process, thereby effectively improving the model accuracy.

[0084] For the device embodiments, since they basically correspond to the method embodiments, the relevant parts can be referred to the partial description of the method embodiments. The device embodiments described above are merely schematic, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the present application scheme. A person of ordinary skill in the art can understand and implement it without paying any creative work.

[0085] Accordingly, the present application also provides an electronic device, comprising: one or more processors; a memory for storing one or more programs; when the one or more programs are executed by the one or more processors, the one or more processors implement the method as described in the first aspect; the present application also provides a computer-readable storage medium on which computer instructions are stored, characterized in that when the instructions are executed by the processor, the steps of the decentralized federated learning method with high model accuracy and resistance to poisoning attacks as described above are implemented.

[0086] It should be understood that the present application is not limited to the exact structures described above and shown in the drawings, and that various modifications and changes may be made without departing from the scope thereof. The scope of the present application is limited only by the appended claims.

Claims

1. A decentralized federated learning method with high model accuracy and resistance to poisoning attacks, characterized by: include: During the initialization phase, each client participating in the decentralized federated learning process determines the structure of the collaboratively trained machine learning model in advance, randomly initializes the model weights and corresponding gradient tracking variables, and evenly distributes the aggregate weights to each neighboring client. To prevent malicious clients from sending malicious gradients for poisoning attacks, each benign client also needs to initialize a recording variable used to record the gradient contribution of each neighboring client. During the aggregation process, each benign client exchanges model parameters with its neighboring clients, aggregates the received model weights and gradient tracking variables, and performs model training based on the local dataset to update its own model weights and gradient tracking variables. At the same time, the benign client updates the recorded variables of the neighboring clients based on the received gradient tracking variables and adjusts the aggregation weights of each neighboring client based on consistency checks. During the aggregation process, each malicious client also exchanges model parameters with its neighboring clients, aggregates the received model weights and gradient tracking variables, and trains the model with the local malicious dataset. This injects malicious gradients into its own model weights and gradient tracking variables, interfering with the aggregation process of benign clients and reducing the final model accuracy. When the aggregation weight of a malicious neighbor client is adjusted to zero based on a consistency check, each benign client immediately clears all malicious gradient contributions of the corresponding malicious neighbor client according to the recorded variables. Then, the benign client optimizes the contaminated model weights according to the normal gradient aggregation process. The beneficial parts from the malicious client are retained during the optimization process, thereby effectively improving the model accuracy.

2. The method according to claim 1, characterized in that Each benign client also needs to initialize the record variables used to record the gradient contribution of each neighbor client, including: ； Where: For the client For each neighbor client The initialized record variable, Is the client The set of all neighbor clients.

3. The method according to claim 2, characterized in that The benign client updates the recorded variables of the neighbor client based on the received gradient tracking variables, including: ； Where: For benign clients To its neighbor clients In the Record variables when iterating aggregation, Is a benign client To its neighbor clients In the The aggregation weight during the iterative aggregation, Is the client In the Gradient tracking variable during iteration aggregation, For benign clients Receiving neighbor clients The gradient tracking variable is then aggregated to the local gradient tracking variable according to the corresponding aggregation weight value.

4. The method according to claim 3, characterized in that Benign clients adjust the aggregation weight for each neighbor client based on consistency checks, including: ； Where: Is a benign client To its neighbor clients In the The aggregation weight assigned during the iterative aggregation, Is the client In the Gradient tracking variable during iteration aggregation, Is the client In the Gradient tracking variable during iteration aggregation, The total number of clients participating in decentralized federated learning.

5. The method according to claim 4, wherein each benign client exchanges model parameters with its neighboring clients, aggregates the received model weights and gradient tracking variables, and performs model training on the local dataset to update its own model weights and gradient tracking variables, specifically including: ； ； Where: Indicates the Client during round iteration local model weights, Indicates the The client local gradient tracking variables, Is the client To its neighbor clients In the The aggregation weight assigned at the iteration, For the client The update step size, For the The client Local model gradient value, It is a signal function that outputs 1 when the aggregation weight is greater than 0 and outputs 0 when the aggregation weight is equal to 0.

6. The method according to claim 5, characterized in that Each malicious client also exchanges model parameters with its neighboring clients, aggregates the received model weights and gradient tracking variables, and trains the model with the local malicious dataset, thereby injecting malicious gradients into its own model weights and gradient tracking variables. Specifically, ； ； in For malicious clients No. The local malicious gradient at the iteration, For malicious clients No. The local normal gradient at iteration , Indicates the maliciousness of the injected malicious gradient.

7. The method according to claim 1, characterized in that Also includes: Each client participating in federated learning specifies the total number of iterations in advance ,exist The training process of federated learning automatically ends after one round of iteration; Alternatively, you can choose not to specify the total number of iterations. After detecting and eliminating all malicious clients, test the model's prediction accuracy each time the model weights are updated. If every benign client achieves the expected accuracy, the federated learning training process ends.

8. A decentralized federated learning device with high model accuracy and resistance to poisoning attacks, characterized by: include: The configuration module is used to determine the collaborative machine learning model structure in advance during the initialization phase for each client participating in the decentralized federated learning process. It also randomly initializes the model weights and corresponding gradient tracking variables, and evenly distributes the aggregate weights to each neighboring client. To prevent malicious clients from sending malicious gradients for poisoning attacks, each benign client also needs to initialize a recording variable for recording the gradient contribution of each neighboring client. The benign module is used to exchange model parameters with its neighboring clients during the aggregation process, aggregate the received model weights and gradient tracking variables, and perform model training on the local dataset to update its own model weights and gradient tracking variables. At the same time, the benign client updates the recorded variables of the neighboring clients based on the received gradient tracking variables and adjusts the aggregation weight of each neighboring client based on consistency checks. The malicious module is used to exchange model parameters with its neighboring clients during the aggregation process, aggregate the received model weights and gradient tracking variables, and train the model with the local malicious dataset. This injects malicious gradients into its own model weights and gradient tracking variables, interfering with the aggregation process of benign clients and reducing the final model accuracy. The optimization module is used to adjust the aggregation weight of malicious neighbor clients to zero based on consistency checks. Each benign client immediately clears all malicious gradient contributions of the corresponding malicious neighbor clients according to the recorded variables. Then, the benign client optimizes the contaminated model weights according to the normal gradient aggregation process. The optimization process retains the beneficial parts from the malicious clients, thereby effectively improving the model accuracy.

9. An electronic device, characterized in that: include: one or more processors; a memory for storing one or more programs; When the one or more programs are executed by the one or more processors, the one or more processors implement the method according to any one of claims 1 to 8.

10. A computer-readable storage medium having computer instructions stored thereon, characterized in that: When the instruction is executed by a processor, the steps of the method according to any one of claims 1 to 8 are implemented.

Images