Differential privacy online federated learning method based on dimension selection and compression recovery

By introducing a dimensional selection algorithm of local gradient buffer pool and index mechanism in online federated learning, the utility problem of model dimension sampling is solved, the utility and accuracy of differential privacy protection is improved, and more efficient privacy protection and model are achieved train.

CN118940820APending Publication Date: 2024-11-12GUANGXI POWER GRID CORP
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202410992192.2
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-07-23
Publication Date
2024-11-12

AI Technical Summary

Technical Problem

In online federated learning, the utility of model dimension sampling leads to insufficient privacy budgets, noise interferes with data utility, and different dimensions have different impacts on the model, making it difficult to design an effective sampling algorithm to improve the utility of the model.

Method used

Introduce a local gradient buffer pool, use a dimension selection algorithm based on the exponential mechanism to divide the privacy budget to estimate the true value of the top-k dimension, and restore the global model change on the central server side, and control client sampling through the DQN reinforcement learning model. Improve the effectiveness of online federated learning under the guarantee of differential privacy.

Benefits of technology

It improves the utility and privacy protection accuracy of online federated learning, increases the accuracy of global gradient estimation and the efficiency of utilization of local computing resources.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118940820A_ABST
    Figure CN118940820A_ABST
Patent Text Reader

Abstract

The invention discloses a differential privacy online federated learning method based on dimension selection and compression recovery. According to the differential privacy online federated learning method based on dimension selection and compression recovery, the effectiveness problem of a differential privacy mechanism in online federated learning is solved by using a dimension selection method. A local gradient buffer pool is introduced to accumulate the gradient of a local model, a dimension selection algorithm based on an exponential mechanism is used, and a part of privacy budget is divided to be used for estimating the true value of the top-k dimension, so that the variable quantity of the sampled local model is recovered more effectively at a central server side, and the accuracy of the real value of the top-k dimension is improved. And a more accurate global model variation is obtained. According to the method, the effectiveness of online federal learning under a certain differential privacy guarantee is improved.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention belongs to the field of privacy computing and data security, and specifically relates to a differential privacy online federated learning method based on dimension selection and compression recovery. Background Art

[0002] With the rapid popularization of mobile smart devices, the continuous development of technologies such as cloud computing, the Internet of Things, and big data, data analysis, mining, and model training based on private data from various sources (such as medical image data, bank financial data, etc.) have become increasingly common. However, the original data of this type of private data contains a large amount of sensitive information, and directly publishing such data will cause serious data privacy and security issues. In order to protect user privacy and data security, there are currently relevant laws and regulations that prohibit data owners from arbitrarily obtaining and publishing user privacy data. This has created "data islands", which has greatly hindered the development of deep learning, artificial intelligence, etc. In short, how to protect user privacy in various data analysis methods has become an important issue that needs to be solved at present.

[0003] Federated learning, as a new distributed machine learning paradigm, allows multiple clients to jointly train models under the organization of a central server, and the training data does not leave the client. Compared with traditional centralized machine learning, the central server in federated learning does not need to collect all the original data of all users, and can relatively protect the privacy of users. However, current studies still show that using only federated learning may still cause the privacy of users to be leaked during the training process. Differential privacy is a technology that protects the privacy information in the data by perturbing the data. Since differential privacy technology has a strict mathematical definition and its privacy protection degree can be measured using a privacy budget, there are many technologies that use differential privacy technology to protect privacy in federated learning.

[0004] In privacy-preserving online federated learning, the privacy budget is usually limited, and the data uploaded by the client will inevitably be interfered by the noise in the privacy protection mechanism, thus affecting the utility of the data. In localized differential privacy, the more model parameters uploaded, the smaller the privacy budget parameters that can be allocated to each dimension of the model, which will result in greater added noise. In the actual machine learning process, different dimensions have different effects on the model. A simple way to judge the impact of a dimension on the model is to calculate the gradient of the dimension. Therefore, how to design an effective sampling algorithm to control the number of parameters uploaded in federated learning and thus improve the utility of the model has become an important issue. Summary of the invention

[0005] To solve the above problems, the purpose of the present invention is to provide a differential privacy online federated learning method based on dimension selection and compression recovery, so that it can solve the problem of the effectiveness of model dimension sampling in online federated learning. By introducing a local gradient buffer pool to accumulate the gradient of the local model, using a dimension selection algorithm based on an exponential mechanism, a part of the privacy budget is divided to estimate the true value of the top-k dimension, so that the sampled local model changes are effectively restored on the central server side, and a more accurate global model change is obtained. Thereby improving the effectiveness of online federated learning under a certain differential privacy guarantee.

[0006] The present invention is achieved through the following technical solutions:

[0007] A differential privacy online federated learning method based on dimension selection and compression recovery, the method running on a central server and multiple clients, the method comprising:

[0008] Step 1: The central server and client initialize the parameters of online federated learning and initialize the global model;

[0009] Step 2: The central server samples the client using the DQN reinforcement learning model according to the initialized global model to obtain the selected client;

[0010] Step 3: All selected clients update their local models based on the global model and use the data in the local data stream to train the local models. The local model change is obtained based on the difference before and after local model training.

[0011] Step 4: The selected client checks the differential privacy constraints and rejects samples that do not meet the constraints. The selected client updates the local model changes to the local gradient buffer pool, uses the dimension selection mechanism to select the dimension of the local gradient buffer pool, obtains the local upload parameters and uploads them to the central server.

[0012] Step 5: The central server receives the local uploaded parameters from the client, updates the global dimension recovery coefficient, performs compression recovery to obtain the global model change, and uses the global model change to update the global model;

[0013] Step 6: The central server evaluates the updated global model, calculates the reward value of the DQN reinforcement learning model based on the accuracy, trains the DQN reinforcement learning model, and obtains the latest global model;

[0014] Step 7: The central server sends the latest global model to all clients and returns to step 2 and repeats until the termination condition of online federated learning is met.

[0015] A further improvement of the present invention is that in step 1, the central server and the client are initialized, the central server initializes the global model m, the global dimensionality recovery coefficient γ, and sends the global model to all clients, and the client sets the local last sampling time record t′.

[0016] A further improvement of the present invention is that in step 2, the central server reduces the dimension of the global model based on principal component analysis to obtain an embedded representation after dimensionality reduction, and uses the embedded representation after dimensionality reduction and the number of clients that can currently participate in federated learning as inputs of the DQN reinforcement learning model to obtain the sampling interval τ of this round t and the number of client samples φ t , according to the client sampling number φ t Randomly select φ t The clients get the selected client.

[0017] A further improvement of the present invention is that in step 3, the client obtains the global model from the central server, synchronizes the local model with the global model, and the client obtains the global model based on the local t-τ t The data at time t is trained using the gradient descent method for the local model. The parameters after local model training are subtracted from the parameters before local model training to obtain the local model change. is the trained local model, is the local model before training.

[0018] A further improvement of the present invention is that in step 4, the selected client accumulates the model parameters into the local gradient buffer pool of the client:

[0019]

[0020] in, is the updated local gradient buffer pool, β is the attenuation coefficient of the local gradient buffer pool, is the local gradient buffer pool before updating, The local model change amount; the client generates a random symbol amount The value is 1 or -1 if Then sort the dimensions in the local gradient buffer pool from large to small. If Then sort the dimensions in the local gradient buffer pool from small to large, where the first k items are top-k dimensions and the last dk items are non-top-k dimensions;

[0021] The client uses the traversal method to find the optimal solution to the following problem and obtain the optimal dimension selection threshold This optimization problem maximizes the expected value of the number of top-k dimensions among the dimensions selected by the client under a specific index mechanism;

[0022]

[0023] in, d is the number of model parameters, k is the size of the top-k set, h is the number of dimensions finally selected by differential privacy, and v is the number of dimensions selected from the top-k dimensions; the client solves Post-calculation by The value is used as the weight of different values ​​of v. According to the weight, a value is randomly selected from 0 to d as the value of v. The client randomly selects v dimensions from the top-k dimensions and dv dimensions from the non-top-k dimensions to obtain the selected dimensions. The client calculates the mean of the top-k dimensions and adds noise that satisfies the differential privacy constraint to obtain the local dimension recovery coefficient. The last client upload was selected dimension Random symbol amount and the local dimension recovery coefficient

[0024] A further improvement of the present invention is that in step 5, the central server calculates the average value of the local dimension recovery coefficients of all clients to obtain a temporary global dimension recovery coefficient γ t ′ , the central server uses the formula Update the global dimensionality recovery coefficient; where α is a preset update parameter, is the global dimension recovery coefficient of the previous iteration; the central server sets a temporary model for each client and adds The values ​​of the dimensions are all set to The remaining dimension values ​​are set to 0, the mean of the temporary model is calculated and multiplied by the global dimension recovery coefficient to obtain the global model change, and the global model change is added to the global model to update the global model.

[0025] A further improvement of the present invention is that in step 6, the central server evaluates the model on the public data set to obtain the model accuracy ω t , the formula for calculating the reinforcement learning reward value is Ω is the target accuracy of the model. The environment change-behavior-reward value tuple of this round is recorded in the playback buffer of the DQN reinforcement learning model, where: the environment change is the embedding representation after dimensionality reduction and the change in the number of clients that can currently participate in federated learning before and after training, and the behavior refers to the sampling interval τ of this round t and the number of client samples φ t,Then, the central server randomly extracts a batch of environment change-behavior-reward value tuples from the playback buffer to train the DQN reinforcement learning model.

[0026] A further improvement of the present invention is that in step 7, the central server sends the latest global model to all clients, and one execution from step 2 to step 6 is regarded as an online federated learning aggregation. The central server detects whether the online federated learning continues after each round. If not, the central server sends a broadcast to all clients to terminate the federated learning.

[0027] Compared with the prior art, the present invention has at least the following beneficial technical effects:

[0028] The differential privacy online federated learning method based on dimension selection and compression recovery provided by the present invention proposes a more effective differential privacy mechanism based on sampling based on the online federated learning model with w-event-level privacy protection. The method proposed by the present invention proposes the concept of a local gradient buffer pool, selects based on the local gradient buffer pool, and uses an optimized exponential mechanism to select dimensions based on gradients. Unselected gradients are retained to increase the utilization efficiency of local computing resources. A certain privacy budget is allocated to estimate the true value of the top-k gradients of the model, which increases the accuracy of the global gradient estimation. In short, the method proposed by the present invention can effectively increase the effectiveness of the differential privacy mechanism in online federated learning. BRIEF DESCRIPTION OF THE DRAWINGS

[0029] Figure 1 It is a specific flow chart of an embodiment;

[0030] Figure 2 It is a schematic diagram of the modules included in the present invention. DETAILED DESCRIPTION

[0031] In order to make the purpose and technical solution of the present invention clearer and easier to understand, the present invention is further described in detail below in conjunction with the accompanying drawings and embodiments. The specific embodiments described herein are only used to explain the present invention and are not used to limit the present invention.

[0032] In order to make the objectives, technical solutions and advantages of the present invention more clear, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the accompanying drawings in the embodiments of the present invention.

[0033] The differential privacy online federated learning method based on dimension selection and compression recovery provided by the present invention can solve the problem of the effectiveness of model dimension sampling in online federated learning. During the model training process, the DQN model module accumulates the gradient of the local model by introducing a local gradient buffer pool, uses a dimension selection algorithm based on an exponential mechanism, and divides a part of the privacy budget to estimate the true value of the top-k dimension, thereby effectively restoring the sampled local model changes on the central server side and obtaining a more accurate global model change. Thereby improving the effectiveness of online federated learning under certain differential privacy guarantees.

[0034] like Figure 1 The specific flow chart of the embodiment of the present invention is shown, and the specific execution steps are as follows:

[0035] Step 1: During the initialization of the central server and the client, the central server initializes the global model m, the global dimensionality recovery coefficient γ, and sends the global model to all clients. The client sets the local last sampling time record t′ to control whether the client participates in federated learning to meet certain differential privacy constraints.

[0036] Step 2: The central server reduces the dimension of the global model through principal component analysis to obtain the embedded representation after dimensionality reduction, which is convenient for using the embedded representation after dimensionality reduction and the number of clients that can currently participate in federated learning as the input of the DQN reinforcement learning model to obtain the sampling interval τ of this round t and the number of client samples φ t , according to the client sampling number φ t Randomly select φ t The clients get the selected client.

[0037] Step 3: The client obtains the global model from the central server, synchronizes the local model with the global model, and the client uses the local t-τ t The data at time t is trained using the gradient descent method for the local model. The parameters after local model training are subtracted from the parameters before local model training to obtain the local model change. is the trained local model, is the local model before training.

[0038] Step 4: The client accumulates the model parameters into the client's local gradient buffer pool:

[0039]

[0040] in, is the updated local gradient buffer pool, β is the attenuation coefficient of the local gradient buffer pool, is the local gradient buffer pool before updating, The local model change amount; the client generates a random symbol amount The value is 1 or -1 if Then sort the dimensions in the local gradient buffer pool from large to small. If Then sort the dimensions in the local gradient buffer pool from small to large, where the first k items are top-k dimensions and the last dk items are non-top-k dimensions;

[0041] The client uses the traversal method to find the optimal solution to the following problem and obtain the optimal dimension selection threshold This optimization problem maximizes the expected value of the number of top-k dimensions among the dimensions selected by the client under a specific index mechanism;

[0042]

[0043] in, d is the number of model parameters, k is the size of the top-k set, h is the number of dimensions finally selected by differential privacy, and v is the number of dimensions selected from the top-k dimensions; the client solves Post-calculation by The value is used as the weight of different values ​​of v. According to the weight, a value is randomly selected from 0 to d as the value of v. The client randomly selects v dimensions from the top-k dimensions and dv dimensions from the non-top-k dimensions to obtain the selected dimensions. The client calculates the mean of the top-k dimensions and adds noise that satisfies the differential privacy constraint to obtain the local dimension recovery coefficient. The last client upload was selected dimension Random symbol amount and the local dimension recovery coefficient

[0044] Step 5: The central server calculates the average value of the local dimension recovery coefficient of all clients to obtain the temporary global dimension recovery coefficient γ t ′ , the central server uses the formula Update the global dimensionality recovery coefficient; where α is a preset update parameter, is the global dimension recovery coefficient of the previous iteration; the central server sets a temporary model for each client and adds The values ​​of the dimensions are all set to The remaining dimension values ​​are set to 0, the mean of the temporary model is calculated and multiplied by the global dimension recovery coefficient to obtain the global model change, and the global model change is added to the global model to update the global model.

[0045] Step 6: The central server evaluates the model on the public dataset to obtain the model accuracy ω t , the formula for calculating the reinforcement learning reward value is Ω is the target accuracy of the model. The environment change-behavior-reward value tuple of this round is recorded in the playback buffer of the DQN reinforcement learning model, where: the environment change is the embedding representation after dimensionality reduction and the change in the number of clients that can currently participate in federated learning before and after training, and the behavior refers to the sampling interval τ of this round t and the number of client samples φ t ,Then, the central server randomly extracts a batch of environment change-behavior-reward value tuples from the playback buffer to train the DQN reinforcement learning model.

[0046] Step 7: The central server sends the latest global model to all clients. One execution from step 2 to step 6 is regarded as an online federated learning aggregation. The central server detects whether the online federated learning continues after each round. If not, the central server sends a broadcast to all clients to terminate the federated learning.

[0047] like Figure 2 The modules included in the algorithm proposed in the present invention are shown, which mainly include two parts, the central server and the client. The central server includes the global model module and the DQN model module. The global model module is used to update and train the federated learning model, and the DQN model is used to control the federated learning process. The client mainly includes three modules: the model training module is used to obtain client data to train the client's local model, the local gradient buffer pool is used to save the user's local model changes, and the differential privacy mechanism module is used to perform differential privacy processing on the parameters in the local gradient buffer pool, obtain the local upload parameters that meet the differential privacy constraints and upload them to the central server.

[0048] The performance of the algorithm proposed in the present invention is compared through the public data sets MNIST, FMNIST, and CIFAR-10, in which convolutional neural networks are used as the target model for online federated learning training, and the training set of the data set is divided into 100 clients and 1 central server as data streams and verification data sets. The evaluation index of the algorithm is accuracy, which refers to the ratio of the correct results predicted by the model after training to all test results. The parameters of the experiment used are set as follows by default: the algorithm proposed in the present invention is β=0.99, in which 90% of the privacy budget is used for dimension selection and 10% of the privacy budget is used to obtain the local model dimension recovery coefficient. The FedSel algorithm uses 50% of the privacy budget for dimension selection and 50% of the privacy budget for obtaining the estimated value of the dimension value. The dimension recovery coefficient in the SignDS algorithm is set to γ=0.05. Referring to Table 1, from the experimental results, compared with the existing algorithms, the present invention can obtain higher accuracy when using the same privacy budget, and is an effective online federated learning algorithm.

[0049] Table 1 Comparison of the accuracy of differential privacy online federated learning method based on dimension selection and compression recovery and other methods

[0050]

[0051] The above contents are only for explaining the technical idea of ​​the present invention and cannot be used to limit the protection scope of the present invention. Any changes made on the basis of the technical solution in accordance with the technical idea proposed by the present invention shall fall within the protection scope of the claims of the present invention.

Claims

1. A differential privacy online federated learning method based on dimension selection and compression recovery, characterized by: The method is run on a central server and multiple clients, and includes: Step 1: The central server and client initialize the parameters of online federated learning and initialize the global model; Step 2: The central server samples the client using the DQN reinforcement learning model according to the initialized global model to obtain the selected client; Step 3: All selected clients update their local models based on the global model and use the data in the local data stream to train the local models. The local model change is obtained based on the difference before and after local model training. Step 4: The selected client checks the differential privacy constraints and rejects samples that do not meet the constraints. The selected client updates the local model changes to the local gradient buffer pool, uses the dimension selection mechanism to select the dimension of the local gradient buffer pool, obtains the local upload parameters and uploads them to the central server. Step 5: The central server receives the local uploaded parameters from the client, updates the global dimension recovery coefficient, performs compression recovery to obtain the global model change, and uses the global model change to update the global model; Step 6: The central server evaluates the updated global model, calculates the reward value of the DQN reinforcement learning model based on the accuracy, trains the DQN reinforcement learning model, and obtains the latest global model; Step 7: The central server sends the latest global model to all clients and returns to step 2 and repeats until the termination condition of online federated learning is met.

2. The method according to claim 1, characterized in that In step 1, the central server and the client are initialized. The central server initializes the global model m and the global dimensionality recovery coefficient γ, and sends the global model to all clients. The client sets the local last sampling time record t′.

3. The method according to claim 1, characterized in that In step 2, the central server reduces the dimension of the global model based on principal component analysis to obtain the embedded representation after dimensionality reduction. The embedded representation after dimensionality reduction and the number of clients that can currently participate in federated learning are used as the input of the DQN reinforcement learning model to obtain the sampling interval τ for this round t and the number of client samples φ t , according to the client sampling number φ t Randomly select φ t The clients get the selected client.

4. The method according to claim 1, characterized in that: In step 3, the client obtains the global model from the central server, synchronizes the local model with the global model, and the client uses the local t-τ t The data at time t is trained using the gradient descent method for the local model. The parameters after local model training are subtracted from the parameters before local model training to obtain the local model change. is the trained local model, is the local model before training.

5. The method according to claim 1, characterized in that In step 4, the selected client accumulates the model parameters into the client's local gradient buffer pool: in, is the updated local gradient buffer pool, β is the attenuation coefficient of the local gradient buffer pool, is the local gradient buffer pool before updating, The local model change amount; the client generates a random symbol amount The value is 1 or -1 if Then sort the dimensions in the local gradient buffer pool from large to small. If Then sort the dimensions in the local gradient buffer pool from small to large, where the first k items are top-k dimensions and the last dk items are non-top-k dimensions; The client uses the traversal method to find the optimal solution to the following problem and obtain the optimal dimension selection threshold This optimization problem maximizes the expected value of the number of top-k dimensions among the dimensions selected by the client under a specific index mechanism; in, d is the number of model parameters, k is the size of the top-k set, h is the number of dimensions finally selected by differential privacy, and v is the number of dimensions selected from the top-k dimensions; the client solves Post-calculation by The value is used as the weight of different values ​​of v. According to the weight, a value is randomly selected from 0 to d as the value of v. The client randomly selects v dimensions from the top-k dimensions and dv dimensions from the non-top-k dimensions to obtain the selected dimensions. The client calculates the mean of the top-k dimensions and adds noise that satisfies the differential privacy constraint to obtain the local dimension recovery coefficient. The last client upload was selected dimension Random symbol amount and the local dimension recovery coefficient 6. The method according to claim 1, characterized in that In step 5, the central server calculates the average value of the local dimension recovery coefficients of all clients to obtain the temporary global dimension recovery coefficient γ t ′ , the central server uses the formula Update the global dimensionality recovery coefficient; where α is a preset update parameter, is the global dimension recovery coefficient of the previous iteration; the central server sets a temporary model for each client and adds The values ​​of the dimensions are all set to The remaining dimension values ​​are set to 0, the mean of the temporary model is calculated and multiplied by the global dimension recovery coefficient to obtain the global model change, and the global model change is added to the global model to update the global model.

7. The method according to claim 1, characterized in that In step 6, the central server evaluates the model on the public dataset to obtain the model accuracy ω t , the formula for calculating the reinforcement learning reward value is Ω is the target accuracy of the model. The environment change-behavior-reward value tuple of this round is recorded in the playback buffer of the DQN reinforcement learning model, where: the environment change is the embedding representation after dimensionality reduction and the change in the number of clients that can currently participate in federated learning before and after training, and the behavior refers to the sampling interval τ of this round t and the number of client samples φ t ,Then, the central server randomly extracts a batch of environment change-behavior-reward value tuples from the playback buffer to train the DQN reinforcement learning model.

8. The method according to claim 1, characterized in that In step 7, the central server sends the latest global model to all clients. One execution from step 2 to step 6 is regarded as an online federated learning aggregation. The central server detects whether the online federated learning continues after each round. If not, the central server sends a broadcast to all clients to terminate the federated learning.