Decentralized federated learning method and system based on personalized local differential privacy
By combining blockchain and personalized local differential privacy in a decentralized federated learning approach, the shortcomings of existing federated learning in terms of privacy protection, accuracy, and robustness are addressed, achieving more efficient data privacy protection and improved model performance.
Patent Information
- Application Number
- CN202410983672.2
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-07-22
- Publication Date
- 2024-11-12
AI Technical Summary
Existing federated learning methods have shortcomings in privacy protection, accuracy and aggregation robustness, especially in heterogeneous data distribution where it is difficult to effectively balance these aspects, and centralized architectures are susceptible to single points of failure and privacy breaches.
By combining blockchain technology and employing a decentralized federated learning method with personalized local differential privacy, a good balance between privacy, accuracy, and robustness is ensured through personalized privacy budget allocation and parameter-based weighted aggregation algorithms.
It improves the transparency and security of data privacy protection, enhances the stability and resistance to attacks of the system, reduces the impact of noise on model accuracy, and improves model performance and the accuracy of data analysis.
Smart Images

Figure CN118940857A_ABST
Abstract
Description
Technical Field
[0001] The present invention belongs to the technical field of federated learning, and in particular to a decentralized federated learning method and system based on personalized local differential privacy. Background Art
[0002] The statements in this section merely provide background information related to the present invention and do not necessarily constitute prior art.
[0003] With the rapid development of cloud computing, the Internet of Things, big data and other technologies, various production processes in life will continue to generate a large amount of data. These data are difficult to process with traditional methods, which requires the use of machine learning methods to process massive data in order to perform various tasks; however, as privacy and confidentiality protection are gradually valued, data island problems have emerged. When processing data, traditional centralized machine learning methods require a central server to collect raw data from all devices through wireless transmission before processing and analyzing; however, this not only brings a large transmission cost, but also the data of some clients will involve sensitive information, and the transmission of raw data may lead to their privacy leakage, which in turn leads to these clients being unwilling to participate in the training process in order to protect data privacy, thus forming data islands.
[0004] To solve this problem, federated learning came into being; as a privacy-protected distributed machine learning, federated learning can protect data security and ensure the consistency of data rights and responsibilities, and can better solve problems such as data silos and data privacy; currently, federated learning has been widely used in various fields.
[0005] However, existing federated learning mainly relies on parameter servers to generate or update global model parameters, which is a typical centralized architecture. This architecture has problems such as single point failure, privacy leakage and performance bottleneck, because all model parameters rely on a central server; and the credibility of the global model depends on the parameter server and is constrained by the centralized credit model; the entire training process is easily affected by server failures; malicious parameter servers can even poison the model, generate inaccurate global updates, and then distort all local updates, causing the entire collaborative training process to go wrong; therefore, during the model training process, it is particularly important to use a suitable encryption scheme for local model updates and maintain the global model on distributed nodes; blockchain, as a distributed shared general ledger maintained by multiple parties, can be used to replace parameter servers in federated learning and store relevant information during model training through distributed ledger technology, cryptographic algorithms, peer-to-peer communication, consensus mechanisms, smart contracts, etc.
[0006] In addition, some studies have shown that unencrypted intermediate parameters can be used to infer important information in training data, exposing the private data of participants. Researchers mostly use technologies such as homomorphic encryption, Shamir secret sharing, and differential privacy to resist such inference attacks; however, the encryption and decryption operations in the encryption scheme may bring additional computational overhead to model training and significantly reduce the accuracy of the model; and differential privacy confuses the true information of weights by adding noise to the model, although at the expense of accuracy; therefore, balancing privacy and accuracy is a focus in the design of differential privacy methods.
[0007] In the context of heterogeneous data distribution, the global model aggregation process becomes particularly critical; the local data owned by different participants vary significantly in quality and quantity, so aggregating the noisy parameters of these local nodes to form a comprehensive model is crucial for the successful application of differential privacy methods; however, the introduction of noise and the possible existence of malicious attackers may hinder the correct execution of the aggregation algorithm; currently, research on privacy-enhancing methods based on differential privacy is relatively insufficient, and some methods are only applicable to the case where the datasets have independent and identical distribution (I ID), which may not be applicable enough in actual federated learning scenarios; although these methods have been proven to be effective under specific conditions, they have limitations when dealing with non-I ID datasets, and their robustness in the context of federated learning may not be sufficient to cope with various challenges in the real world.
[0008] Therefore, existing federated learning methods are insufficient in balancing privacy, accuracy, and aggregation robustness. Summary of the invention
[0009] To overcome the shortcomings of the above-mentioned prior art, the present invention provides a decentralized federated learning method and system based on personalized local differential privacy, combines federated learning with blockchain, and uses personalized privacy budget allocation and parameter-based weighted aggregation algorithm to ensure an effective trade-off between privacy, accuracy and robustness of federated learning.
[0010] To achieve the above objectives, one or more embodiments of the present invention provide the following technical solutions:
[0011] The first aspect of the present invention provides a decentralized federated learning method based on personalized local differential privacy.
[0012] A decentralized federated learning method based on personalized local differential privacy, including:
[0013] Initialize the blockchain network for federated learning, including registering the task publisher and several clients participating in the training as blockchain nodes, the task publisher publishing the training task to the blockchain network, and allocating the initial privacy budget of the client;
[0014] Each client receives training tasks from the blockchain, uses the local data set to perform local iterative training on the global model downloaded from the blockchain, and adaptively allocates the privacy budget and performs local differential privacy processing on the local model parameters until the conditions for stopping the iteration are met. The trained local model parameters are obtained and uploaded to the blockchain;
[0015] The blockchain performs weighted aggregation on the local model parameters uploaded by the client to obtain the aggregated global model and sends it back to the task publisher;
[0016] The privacy budget adopts a personalized privacy budget allocation strategy, including initial privacy budget allocation based on data set weighting and adaptive privacy budget allocation based on model accuracy.
[0017] Furthermore, the initial privacy budget allocated to the client is allocated based on the weight of the data set, specifically:
[0018] The weight of each client is calculated based on the amount of local data;
[0019] The initial privacy budget of each client is allocated to the total budget set by the task publisher according to the weight.
[0020] Furthermore, the adaptive allocation of the privacy budget is to adopt the adaptive allocation of the privacy budget based on the model accuracy, specifically:
[0021] If the local model accuracy of the current round is lower than that of the previous round, the privacy budget for the next round is increased by a preset value.
[0022] Furthermore, the local differential privacy processing of the local model parameters is specifically as follows:
[0023] Based on the updated model parameters, calculate the sensitivity of the training data;
[0024] Calculate the Gaussian noise to add to the gradient based on the privacy budget and sensitivity;
[0025] Add the calculated noise to the gradient;
[0026] Based on the gradient, the model parameters are updated using the stochastic gradient descent algorithm.
[0027] Furthermore, the local iterative training comprises the following specific steps:
[0028] (1) Download the latest global model from the blockchain;
[0029] (2) Input the training data into the model and calculate the model's predicted value through forward propagation;
[0030] (3) Compare the model’s predicted value with the true label and construct a loss function;
[0031] (4) Calculate the gradient information of the loss function relative to the model parameters through the back propagation algorithm;
[0032] (5) Based on the latest gradient information, local differential privacy processing is performed on the local model parameters to obtain a model with updated parameters;
[0033] (6) Repeat steps (2) to (5) until the stopping condition is reached.
[0034] Furthermore, the blockchain performs weighted aggregation on the local model parameters uploaded by the client, specifically:
[0035] When the local model parameters in the blockchain reach the preset number, the model aggregation smart contract is triggered, and the local model parameters are aggregated using a parameter-based weighted aggregation algorithm, and the aggregated parameters are used to update the global model.
[0036] Furthermore, the parameter-based weighted aggregation algorithm is specifically:
[0037] Calculate the cosine similarity between the local model uploaded by the client and the global model to obtain the credibility of the client;
[0038] Calculate aggregation parameters based on the client’s trustworthiness, data volume, and privacy budget;
[0039] Based on the aggregation parameters, the global model and the local model parameters uploaded by the client are weightedly aggregated.
[0040] A second aspect of the present invention provides a decentralized federated learning system based on personalized local differential privacy.
[0041] A decentralized federated learning system based on personalized local differential privacy, including:
[0042] The initialization module is configured to: initialize the blockchain network for federated learning, including registering the task publisher and several clients participating in the training as blockchain nodes, the task publisher publishing the training task to the blockchain network, and allocating the initial privacy budget of the client;
[0043] The local training module is configured as follows: each client receives the training task from the blockchain, uses the local data set to perform local iterative training on the global model downloaded from the blockchain, and adaptively allocates the privacy budget and performs local differential privacy processing on the local model parameters until the iteration stop condition is met, obtains the trained local model parameters, and uploads them to the blockchain;
[0044] The weighted aggregation module is configured as follows: the blockchain performs weighted aggregation on the local model parameters uploaded by the client, obtains the aggregated global model, and transmits it back to the task publisher;
[0045] The privacy budget adopts a personalized privacy budget allocation strategy, including initial privacy budget allocation based on data set weighting and adaptive privacy budget allocation based on model accuracy.
[0046] The third aspect of the present invention provides a computer-readable storage medium having a program stored thereon, which, when executed by a processor, implements the steps in the decentralized federated learning method based on personalized local differential privacy as described in the first aspect of the present invention.
[0047] The fourth aspect of the present invention provides an electronic device, comprising a memory, a processor, and a program stored in the memory and executable on the processor, wherein when the processor executes the program, the steps in the decentralized federated learning method based on personalized local differential privacy as described in the first aspect of the present invention are implemented.
[0048] One or more of the above technical solutions have the following beneficial effects:
[0049] 1. The present invention combines blockchain with federated learning and replaces the central server with blockchain, which not only strengthens data privacy protection and model credibility, but also improves the security and decentralization of the system; blockchain technology ensures the transparency and immutability of data transmission and processing, and provides a trusted environment for participants; at the same time, the decentralized model aggregation mechanism makes the system more stable and resistant to attacks, while traceable records help resolve potential disputes and conflicts, providing a more reliable and secure framework for federated learning.
[0050] 2. This invention uses local differential privacy based on personalized privacy budget allocation, which can better reduce the impact of added noise on model accuracy compared with other local differential privacy methods. The smart contract makes personalized initial allocation of privacy budget for each client based on the size of each client's data set. In subsequent rounds, adaptive budget allocation is made based on the model accuracy of the previous round, so as to achieve a better balance between privacy protection and data utilization, improve the availability and privacy protection level of the model, reduce unnecessary noise addition, and improve model performance and data analysis accuracy.
[0051] 3. The present invention uses a parameter-based weighted aggregation method to generate a new global model, taking the size of the data set owned by the client, the similarity between the local model and the global model of the previous round, and the privacy budget consumed by the client in this round of training as weighted parameters; compared with other methods, it has significant advantages. First, larger data sets usually contain more information, and the trained models are more robust. Weighting ensures that these models have a greater impact on the global model and improves overall performance. Secondly, considering the similarity between the local model and the global model of the previous round, weighted aggregation can effectively utilize the previous model state, accelerate the convergence process of the model and improve the stability and consistency of the model; in addition, by considering the privacy budget consumed by the client in this round of training as a weighted parameter, this method can effectively manage and optimize the resource allocation in the federated learning process while protecting individual privacy; in general, this method significantly improves the performance, robustness and adaptability of the global model by reasonably allocating weights.
[0052] Advantages of additional aspects of the present invention will be given in part in the following description, and in part will become obvious from the following description, or will be learned through practice of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS
[0053] The accompanying drawings in the specification, which constitute a part of the present invention, are used to provide a further understanding of the present invention. The exemplary embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute improper limitations on the present invention.
[0054] Figure 1 This is a functional diagram of the first embodiment.
[0055] Figure 2 This is a flow chart of the method of the first embodiment. DETAILED DESCRIPTION
[0056] It should be noted that the following detailed descriptions are illustrative and are intended to provide further explanation of the present application. Unless otherwise specified, all technical and scientific terms used in the present invention have the same meanings as those commonly understood by those skilled in the art to which the present application belongs.
[0057] It should be noted that the terms used herein are only for describing specific embodiments and are not intended to limit the exemplary embodiments according to the present application. As used herein, unless the context clearly indicates otherwise, the singular form is also intended to include the plural form. In addition, it should be understood that when the terms "comprise" and / or "include" are used in this specification, it indicates the presence of features, steps, operations, devices, components and / or combinations thereof.
[0058] Explanation of terms
[0059] Federated learning is essentially a distributed machine learning technology in which clients (such as tablets, mobile phones, and IoT devices) jointly train models under the coordination of a central server (such as a service provider). The client is responsible for training local data to obtain a local model, and the central server is responsible for weighted aggregation of local models to obtain a global model. After multiple rounds of iterations, a model that is close to the results of centralized machine learning is finally obtained, which effectively reduces many privacy risks brought about by the aggregation of traditional machine learning source data.
[0060] Blockchain was first used as the underlying technology of the Bitcoin protocol. It is a distributed public ledger that uses Merkle trees and hash functions for encryption. Its main advantages include decentralization, openness, transparency, anonymity, and traceability.
[0061] Differential privacy is a privacy protection technology. The core idea is to prevent re-identification attacks on individual data by introducing a certain degree of noise or disturbance into the data.
[0062] Embodiment 1
[0063] In one embodiment of the present disclosure, a decentralized federated learning method based on personalized local differential privacy is provided, comprising the following steps:
[0064] Step S1: Initialize the blockchain network for federated learning, including registering the task publisher and several clients participating in the training as blockchain nodes, the task publisher publishing the training task to the blockchain network, and allocating the initial privacy budget of the client;
[0065] The privacy budget adopts a personalized privacy budget allocation strategy, including initial privacy budget allocation based on data set weighting and adaptive privacy budget allocation based on model accuracy.
[0066] Furthermore, the initial privacy budget allocated to the client is allocated based on the weight of the data set, specifically:
[0067] The weight of each client is calculated based on the amount of local data;
[0068] The initial privacy budget of each client is allocated to the total budget set by the task publisher according to the weight.
[0069] Step S2: Each client receives the training task from the blockchain, uses the local data set to perform local iterative training on the global model downloaded from the blockchain, and adaptively allocates the privacy budget and performs local differential privacy processing on the local model parameters until the conditions for stopping the iteration are met. The trained local model parameters are obtained and uploaded to the blockchain;
[0070] Furthermore, the adaptive allocation of the privacy budget is to adopt the adaptive allocation of the privacy budget based on the model accuracy, specifically:
[0071] If the local model accuracy of the current round is lower than that of the previous round, the privacy budget for the next round is increased by a preset value.
[0072] Furthermore, the local differential privacy processing of the local model parameters is specifically as follows:
[0073] Based on the updated model parameters, calculate the sensitivity of the training data;
[0074] Calculate the Gaussian noise to add to the gradient based on the privacy budget and sensitivity;
[0075] Add the calculated noise to the gradient;
[0076] Based on the gradient, the model parameters are updated using the stochastic gradient descent algorithm.
[0077] Furthermore, the local iterative training comprises the following specific steps:
[0078] (1) Download the latest global model from the blockchain;
[0079] (2) Input the training data into the model and calculate the model's predicted value through forward propagation;
[0080] (3) Compare the model’s predicted value with the true label and construct a loss function;
[0081] (4) Calculate the gradient information of the loss function relative to the model parameters through the back propagation algorithm;
[0082] (5) Based on the latest gradient information, local differential privacy processing is performed on the local model parameters to obtain a model with updated parameters;
[0083] (6) Repeat steps (2) to (5) until the stopping condition is reached.
[0084] Step S3: The blockchain performs weighted aggregation on the local model parameters uploaded by the client to obtain the aggregated global model and sends it back to the task publisher;
[0085] Furthermore, the blockchain performs weighted aggregation on the local model parameters uploaded by the client, specifically:
[0086] When the local model parameters in the blockchain reach the preset number, the model aggregation smart contract is triggered, and the local model parameters are aggregated using a parameter-based weighted aggregation algorithm, and the aggregated parameters are used to update the global model.
[0087] Furthermore, the parameter-based weighted aggregation algorithm is specifically:
[0088] Calculate the cosine similarity between the local model uploaded by the client and the global model to obtain the credibility of the client;
[0089] Calculate aggregation parameters based on the client’s trustworthiness, data volume, and privacy budget;
[0090] Based on the aggregation parameters, the global model and the local model parameters uploaded by the client are weightedly aggregated.
[0091] The following is a detailed description of the implementation process of the decentralized federated learning method based on personalized local differential privacy in this embodiment.
[0092] In this method, if Figure 1 As shown in the figure, three roles are involved: task publisher, client and blockchain. The functions of the task publisher include establishing a blockchain network, registering nodes, publishing training tasks, verifying and managing new models, and receiving global models. The functions of the client include receiving training tasks, downloading global models, local model training, and uploading local model parameters. The blockchain has blockchain initialization, storage of registration information, storage of initial models, deployment of smart contracts, privacy budget allocation, storage of local models, and update and storage of global models.
[0093] Based on the above three roles, a specific implementation example is provided, such as Figure 2 As shown:
[0094] 1. Blockchain initialization:
[0095] Establish a federated learning blockchain network through a distributed ledger platform.
[0096] 2. Register the node:
[0097] The task publisher and the clients participating in the training are registered on the blockchain network respectively and registered as blockchain nodes.
[0098] 3. Release training tasks:
[0099] The task publisher publishes the training task to the blockchain network in the form of a smart contract, creates a new block in the blockchain network, and writes the initial model into the new block.
[0100] 4. Personalized privacy budget allocation:
[0101] The privacy budget of each client node is individually allocated to achieve a balance between model accuracy and privacy.
[0102] 4.1 Initial allocation strategy of privacy budget based on dataset weighting:
[0103] During the first two rounds of training, the initial privacy budget is allocated by weighting according to the size of the dataset (i.e., the amount of data) uploaded by each client when registering.
[0104] Specifically, suppose there are N clients participating in federated learning, and the size of the dataset of each client i is n i .
[0105] The weight of each client fpw i , calculated according to the size of its data set, the formula is:
[0106]
[0107] in, is the sum of the dataset sizes of all clients.
[0108] Each client’s initial privacy budget ∈ i , according to the weight fpw i The distribution is expressed as follows:
[0109] ∈ i =∈·fpw i
[0110] Where ∈ is the total budget set by the task publisher.
[0111] 4.2 Privacy budget adaptive allocation strategy based on model accuracy:
[0112] Starting from the third round of training, if the local model accuracy of round t is lower than that of round t-1, the privacy budget of round t+1 is increased by c (c is a pre-set value) to add less noise.
[0113] 5. Local model training:
[0114] The client node uses local data to train the latest global model. The specific steps are:
[0115] 5.1 Global model download:
[0116] Client nodes download the latest global model parameters from the blockchain.
[0117] 5.2 Forward Propagation:
[0118] The client node inputs the training data into the model and calculates the model's predicted value through forward propagation.
[0119] 5.3 Calculate the loss:
[0120] The client node compares the model's predicted value with the true label, selects mean-square error (MSE) as the loss function, and calculates the value of the loss function to measure the accuracy of the model's prediction.
[0121] 5.4 Back Propagation:
[0122] In the tth round, client node k calculates the gradient of the loss function relative to the model parameters through the back propagation algorithm
[0123] 5.5 Calculation sensitivity:
[0124] Based on the updated model parameters, the client node calculates the sensitivity of the participant's training data, that is, the maximum degree to which changing an entry (or a group of entries) in the database will cause the gradient to change. In Stochastic Gradient Descent (SGD), the norm is usually used to measure the sensitivity of the gradient. The calculation formula of sensitivity S is as follows:
[0125]
[0126] Where D and D′ are two datasets that differ by only one entry, g(w; D) and g(w; D′) represent the gradients of the loss function calculated based on datasets D and D′ with respect to the model parameter w, respectively, and ||·||2 represents the L2 norm (also known as the Euclidean norm, which is a commonly used norm used to measure the “length” or “size” of a vector).
[0127] 5.6 Calculation of noise:
[0128] The client node calculates the Gaussian noise N(0,σ 2 ) is calculated as follows:
[0129]
[0130] Among them, N(0,σ 2 ) means that the mean is 0 and the variance is σ 2 is a normal distribution, δ is the probability of privacy failure (usually a very small value, such as 10 -5 or smaller).
[0131] 5.7 Adding noise:
[0132] Client node k adjusts the calculated noise according to the number of training rounds and adds it to the gradient to protect the privacy of individual data; the gradient after adding noise is expressed as:
[0133]
[0134] 5.8 Update weights:
[0135] The client node updates the model parameters using the stochastic gradient descent (SGD) algorithm based on the calculated gradient information, which can be expressed as:
[0136]
[0137] Among them, w t,k is the model parameter of client node k in round t, is the gradient of client node k after adding noise in round t, and η is the learning rate.
[0138] 5.9 Repeat iteration:
[0139] Repeat the above steps until the stopping condition is reached (such as reaching the maximum number of iterations, the loss function converges, etc.).
[0140] 6. Upload local model parameters:
[0141] The client node writes the locally trained model parameters into a new block of the blockchain, which triggers the model verification smart contract. The model verification smart contract determines whether the model parameters are qualified by calculating the cosine similarity between the model parameters uploaded by the client node this round and the local model parameters uploaded in the previous round. The model parameters that pass the verification are successfully written into the new block. The calculation formula of cosine similarity is:
[0142]
[0143] In the formula, The model parameters obtained after the tth round of training uploaded by client node k The local model parameters uploaded in the previous round cosine similarity, θ(0.8<θ<0.95) is the preset cosine similarity threshold.
[0144] Based on the above cosine similarity, the judgment formula is as follows:
[0145]
[0146] 7. Global model update:
[0147] When the number of local model parameters in the blockchain reaches a certain level, the model aggregation smart contract will be triggered, and a weighted aggregation algorithm based on the parameter Poma will be used to aggregate the local model parameters, and the aggregated parameters will be used to update the global model. The size of the parameter Poma depends on the size of the dataset owned by the client, the credibility of the local model uploaded by the client, and the privacy budget consumed by the client to complete the training of the local model.
[0148] 7.1 Computing model credibility: The model aggregation smart contract calculates the cosine similarity between the local model parameters obtained by client k after the t-th round of training and the global model parameters w of the previous round to obtain the credibility C of client k and the global model parameters w of the previous round t-1 to obtain the credibility C of client k k :
[0149]
[0150] 7.2 Calculating parameter Poma: The model aggregation smart contract calculates parameter Poma by computing the credibility, data volume, and privacy budget of client k k :
[0151]
[0152] where n k is the data volume of client k, ∈ k is the privacy budget of client k, C k is the credibility of client k, and a, β, γ are parameters, where 0 < a, β, γ < 1 and a + β + γ = 1
[0153] 7.3 Global model aggregation: Aggregate the global model of the previous round with the weighted local model parameters uploaded by m clients, which is expressed by the formula:
[0154]
[0155] where α is a balancing parameter (usually between 0 and 1) used to balance the global model of the previous round and the local model parameters of the current round
[0156] 8. New model verification:
[0157] If the aggregated global model does not meet the accuracy requirements of the task publisher, write the new global model into the blockchain as the initial model for the next round of training and go to 4 for a new round of model training; if the aggregated global model meets the requirements of the task publisher, proceed to the next step
[0158] 9. Model acquisition:
[0159] Send the model to the task publisher and write the relevant information of the completed training into the blockchain
[0160] Embodiment 2
[0161] In an embodiment of the present disclosure, a decentralized federated learning system based on personalized local differential privacy is provided, including:
[0162] The initialization module is configured to: initialize the blockchain network for federated learning, including registering the task publisher and several clients participating in the training as blockchain nodes, the task publisher publishing the training task to the blockchain network, and allocating the initial privacy budget of the client;
[0163] The local training module is configured as follows: each client receives the training task from the blockchain, uses the local data set to perform local iterative training on the global model downloaded from the blockchain, and adaptively allocates the privacy budget and performs local differential privacy processing on the local model parameters until the iteration stop condition is met, obtains the trained local model parameters, and uploads them to the blockchain;
[0164] The weighted aggregation module is configured as follows: the blockchain performs weighted aggregation on the local model parameters uploaded by the client, obtains the aggregated global model, and transmits it back to the task publisher;
[0165] The privacy budget adopts a personalized privacy budget allocation strategy, including initial privacy budget allocation based on data set weighting and adaptive privacy budget allocation based on model accuracy.
[0166] Embodiment 3
[0167] The purpose of this embodiment is to provide a computer-readable storage medium.
[0168] A computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps in the decentralized federated learning method based on personalized local differential privacy as described in the first embodiment of the present disclosure.
[0169] Embodiment 4
[0170] The purpose of this embodiment is to provide an electronic device.
[0171] An electronic device includes a memory, a processor, and a program stored in the memory and executable on the processor. When the processor executes the program, the steps in the decentralized federated learning method based on personalized local differential privacy as described in the first embodiment of the present disclosure are implemented.
[0172] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. For those skilled in the art, the present invention may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.
Claims
1. A decentralized federated learning method based on personalized local differential privacy, characterized by: include: Initialize the blockchain network for federated learning, including registering the task publisher and several clients participating in the training as blockchain nodes, the task publisher publishing the training task to the blockchain network, and allocating the initial privacy budget of the client; Each client receives training tasks from the blockchain, uses the local data set to perform local iterative training on the global model downloaded from the blockchain, and adaptively allocates the privacy budget and performs local differential privacy processing on the local model parameters until the conditions for stopping the iteration are met. The trained local model parameters are obtained and uploaded to the blockchain; The blockchain performs weighted aggregation on the local model parameters uploaded by the client to obtain the aggregated global model and sends it back to the task publisher; The privacy budget adopts a personalized privacy budget allocation strategy, including initial privacy budget allocation based on data set weighting and adaptive privacy budget allocation based on model accuracy.
2. The decentralized federated learning method based on personalized local differential privacy as claimed in claim 1, characterized in that: The initial privacy budget of the client is allocated by using the initial privacy budget allocation based on the weight of the data set, specifically: The weight of each client is calculated based on the amount of local data; The initial privacy budget of each client is allocated to the total budget set by the task publisher according to the weight.
3. The decentralized federated learning method based on personalized local differential privacy as claimed in claim 1, characterized in that: The adaptive allocation of the privacy budget is to adopt the adaptive allocation of the privacy budget based on the model accuracy, specifically: If the local model accuracy of the current round is lower than that of the previous round, the privacy budget for the next round is increased by a preset value.
4. The decentralized federated learning method based on personalized local differential privacy as claimed in claim 1, characterized in that: The local differential privacy processing of the local model parameters is specifically as follows: Based on the updated model parameters, calculate the sensitivity of the training data; Calculate the Gaussian noise to add to the gradient based on the privacy budget and sensitivity; Add the calculated noise to the gradient; Based on the gradient, the model parameters are updated using the stochastic gradient descent algorithm.
5. The decentralized federated learning method based on personalized local differential privacy as claimed in claim 1, characterized in that: The local iterative training comprises the following specific steps: (1) Download the latest global model from the blockchain; (2) Input the training data into the model and calculate the model's predicted value through forward propagation; (3) Compare the model’s predicted value with the true label and construct a loss function; (4) Calculate the gradient information of the loss function relative to the model parameters through the back-propagation algorithm; (5) Based on the latest gradient information, local differential privacy processing is performed on the local model parameters to obtain a model with updated parameters; (6) Repeat steps (2) to (5) until the stopping condition is reached.
6. The decentralized federated learning method based on personalized local differential privacy as claimed in claim 1, characterized in that: The blockchain performs weighted aggregation on the local model parameters uploaded by the client, specifically: When the local model parameters in the blockchain reach the preset number, the model aggregation smart contract is triggered, and the local model parameters are aggregated using a parameter-based weighted aggregation algorithm, and the aggregated parameters are used to update the global model.
7. The decentralized federated learning method based on personalized local differential privacy as claimed in claim 6, characterized in that: The parameter-based weighted aggregation algorithm is specifically: Calculate the cosine similarity between the local model uploaded by the client and the global model to obtain the credibility of the client; Calculate aggregation parameters based on the client’s trustworthiness, data volume, and privacy budget; Based on the aggregation parameters, the global model and the local model parameters uploaded by the client are weightedly aggregated.
8. A decentralized federated learning system based on personalized local differential privacy, characterized by: include: The initialization module is configured to: initialize the blockchain network for federated learning, including registering the task publisher and several clients participating in the training as blockchain nodes, the task publisher publishing the training task to the blockchain network, and allocating the initial privacy budget of the client; The local training module is configured as follows: each client receives the training task from the blockchain, uses the local data set to perform local iterative training on the global model downloaded from the blockchain, and adaptively allocates the privacy budget and performs local differential privacy processing on the local model parameters until the iteration stop condition is met, obtains the trained local model parameters, and uploads them to the blockchain; The weighted aggregation module is configured as follows: the blockchain performs weighted aggregation on the local model parameters uploaded by the client, obtains the aggregated global model, and returns it to the task publisher; The privacy budget adopts a personalized privacy budget allocation strategy, including initial privacy budget allocation based on data set weighting and adaptive privacy budget allocation based on model accuracy.
9. An electronic device, comprising: a memory for non-transitory storage of computer readable instructions; a processor for executing the computer-readable instructions; Wherein, when the computer-readable instructions are executed by the processor, the method according to any one of claims 1 to 7 is executed.
10. A storage medium, characterized in that: The computer-readable instructions are non-transitorily stored, wherein when the computer-readable instructions are executed by a computer, the method of any one of claims 1 to 7 is performed.
Citation Information
Patent Citations
Federal learning method based on adaptive differential privacy
CN117874829A
Distributed federated learning aggregation method based on block chain
CN117972744A
Hierarchical privacy protection federated learning method based on core data set
CN118350038A
Communication-efficient federated learning system with data protection and data security
DE202023107184U1
Cited By
Federal element learning privacy protection method based on data set compression
CN119203248A
Compressor running state monitoring method and system based on medical oxygen generation system
CN119207751A
Federal learning method based on local small-batch gradient differential privacy
CN119692499A
Decentralized collaborative learning method and system for privacy protection and computer equipment
CN119783857A
Regular constraint adaptive adjustment method for privacy-preserving heterogeneous decentralized learning
CN119808896A
