Diabetic retinopathy image recognition method based on federal learning and differential privacy

Abstract

The invention provides a diabetic retinopathy image recognition method based on federated learning and differential privacy, and belongs to the field of federated learning. The method comprises the following steps: carrying out preprocessing and Poisson sampling on a diabetic retinopathy image data set to initialize a global model located in a central server; the local client obtains parameters of the global model from the central server and constructs a local model; the local client performs training and differential privacy processing on a local model by using local data; the central server receives the parameters uploaded by the clients for secure aggregation, and updates the global model by using the parameters after secure aggregation; if the number of training iterations does not reach the threshold value specified by the user, a result is transmitted to the local model to continue the training process; and when the number of training iterations reaches a threshold value specified by the user, terminating a further iteration process to obtain a trained model. And the problems of gradient leakage and data loss possibly existing in privacy protection by introducing differential privacy in federated learning are balanced.

Description

Technical Field

[0001] The present invention relates to the field of federated learning, and in particular to a method for recognizing diabetic retinopathy images based on federated learning and differential privacy. Background Art

[0002] Diabetic retinopathy is an eye disease caused by diabetes and is one of the most common complications for people with diabetes. The disease damages the retina inside the eye, which is the tissue at the back of the eyeball that is responsible for receiving light and converting it into nerve signals, which are then transmitted to the brain for image recognition. Ophthalmologists need to spend a lot of time to accurately grade diabetic retinopathy, which is a huge challenge for beginners in ophthalmology.

[0003] Federated learning is a new type of machine learning method that aims to overcome the privacy and data security issues in traditional centralized learning methods. The core idea is to achieve global training of the model by training the model locally on distributed data sources without sharing the original data, and then transmitting the local model parameter updates to the central server. However, although federated learning implicitly provides a certain degree of privacy protection, the architecture technology based on federated learning still has the risk of gradient leakage, and the hospital (local client) may leak patient information when updating the model. In this regard, the usual solution is to introduce differential privacy for data encryption based on federated learning. Differential privacy is a technology that protects individual privacy in data processing. It introduces controllable noise during data release or analysis to prevent the leakage of sensitive information about individual data. The core idea of ​​differential privacy technology is to maintain the validity and availability of data as much as possible while protecting data privacy. However, the introduction of differential privacy often means that the loss will also increase, which will lead to a decrease in accuracy and inaccurate recognition. Summary of the invention

[0004] In view of this, the present invention provides an image recognition method for diabetic retinopathy based on federated learning and differential privacy, and performs image recognition of diabetic retinopathy based on federated learning and differential privacy. The relaxed differential privacy with Gaussian noise is introduced to solve the privacy problem existing in federated learning itself, which can better protect data privacy. At the same time, the independently designed federated adaptive threshold optimization method greatly reduces the loss, improves the accuracy, and makes up for the defect of decreased accuracy caused by differential privacy, and obtains good classification accuracy under the premise of differential privacy guarantee.

[0005] To this end, the present invention provides the following technical solutions:

[0006] The present invention provides a diabetic retinopathy image recognition method based on federated learning and differential privacy, characterized by comprising:

[0007] Obtain a dataset of diabetic retinopathy images and preprocess the data in the dataset. The central server uses the preprocessed data to initialize the global model.

[0008] The central server iteratively updates the global model until the specified number of iterations is reached; each iteration includes:

[0009] The local client obtains the parameters of the global model from the central server and builds the local model;

[0010] The local client trains the local model using local data, and performs differential privacy protection on the trained model parameters; the differential privacy is relaxed differential privacy with Gaussian noise;

[0011] The local client uploads the trained and encrypted model parameters to the central server;

[0012] The central server performs weighted averaging on the uploaded model parameters and updates the global model;

[0013] The updated global model is used to perform diabetic retinopathy image recognition to obtain diabetic retinopathy image recognition results.

[0014] Furthermore, the data preprocessing includes: processing the diabetic retinopathy images uniformly into a size of 224×224.

[0015] Furthermore, the data preprocessing also includes: extracting a portion of samples from the diabetic retinopathy data set in each hospital by Poisson sampling.

[0016] Furthermore, the local client trains the local model using local data, including:

[0017] Calculate the gradient values ​​of model parameters;

[0018] Gradient clipping is performed based on the gradient clipping threshold.

[0019] Furthermore, gradient clipping is performed according to a gradient clipping threshold, including:

[0020] The gradient vector g is replaced by Where C is the clipping threshold.

[0021] Furthermore, differential privacy protection is performed on the trained model parameters, including:

[0022] Use Gaussian mechanism to add noise to the clipped gradient.

[0023] Furthermore, it also includes:

[0024] Broadcast the updated global model parameters to each local client;

[0025] The local client updates the local model using the updated parameters.

[0026] Furthermore, the local client updates the local model using the updated parameters, including:

[0027] The drift between the global model of the last round and its adjacent communication round models is quantified using the weight correction loss;

[0028] The corrected model is obtained by minimizing the weight correction loss, and the weighted sum of the corrected model and the previous round model is taken as the final result.

[0029] Furthermore, the modified model is obtained by minimizing the weight modification loss, including:

[0030] The loss is corrected using the model calibration function to obtain a corrected model, where the model calibration function is:

[0031]

[0032] Where μ is a positive number used to balance the weight between the objective function and the regularization term; the regularization term penalizes the variable w t and w t-1 The square of the Euclidean distance between t and w t-1 The proximity between them is constrained.

[0033] Furthermore, an adaptive threshold optimization method is used to set μ, including: if the current loss is greater than the loss in the previous round, then increasing the value of μ; if the current loss has been smaller than the loss in the previous round for λ rounds in a row, then decreasing the value of μ.

[0034] Advantages and positive effects of the present invention:

[0035] 1) It provides a complete multi-hospital collaborative training process, effectively solving the privacy challenges brought about by data dispersion.

[0036] 2) The Poisson sampling method is used to ensure that the statistical characteristics of the data, such as the mean and variance, can be maintained as much as possible while introducing noise, so as to ensure that when the disturbed data is statistically analyzed, the results still have a certain degree of credibility and accuracy.

[0037] 3) By introducing noise through the Gaussian mechanism, it is ensured that differential privacy is achieved by adding less noise to solve the privacy loopholes in federated learning itself, and the data loss caused by differential privacy is reduced.

[0038] 4) A flexible model calibration strategy is proposed to effectively limit the impact of variable local updates and solve the model drift problem.

[0039] 5) A federal adaptive threshold optimization method was designed to effectively reduce the loss, improve the accuracy, and make up for the loss of accuracy caused by differential privacy. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

[0041] Figure 1 It is a flowchart of an image recognition method for diabetic retinopathy based on federated learning and differential privacy in an embodiment of the present invention;

[0042] Figure 2 This is a multi-hospital collaboration flow chart of the present invention. DETAILED DESCRIPTION

[0043] In order to enable those skilled in the art to better understand the scheme of the present invention, the technical scheme in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present invention.

[0044] It should be noted that the terms "first", "second", etc. in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments of the present invention described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units that are clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0045] The present invention provides an image recognition method for diabetic retinopathy based on federated learning and differential privacy, and its concept is as follows: preprocessing operations are performed on the data set of diabetic retinopathy images, and the central server uses the data set to initialize the global model; the local server obtains the global model parameters to train the local model; in each round of training, each hospital extracts a part of the samples from the local diabetic retinopathy data set, and this process uses the Poisson sampling method for sampling, and each data point is independently added to the batch. After that, in each round of training, gradient calculation is performed, a flexible model calibration strategy is executed, and then gradient clipping is performed, and finally the threshold is adaptively updated. Afterwards, in the local training process, for each selected hospital, the privacy budget is calculated, the model parameters are weighted averaged and securely aggregated, and finally the global model is updated and globally broadcasted.

[0046] like Figure 1 As shown, an embodiment of the present invention provides an image recognition method for diabetic retinopathy based on federated learning and differential privacy, and the specific steps are as follows:

[0047] S1. Preprocess and Poisson sample the data, and the central server initializes the global model.

[0048] Specifically, data preprocessing was performed. For different experiments, independent and non-independent distributions were considered for data partitioning. In order to handle changes in image size, all images were uniformly processed to a size of 224×224. 12,576 retinal images in the dataset were used for training, of which 3,144 images (20% of the entire dataset) were used for evaluation testing. Each hospital selected k from the local diabetic retinopathy dataset. A part of the samples are drawn from the batch, which is sampled using Poisson sampling. Each data point is added to the batch independently with a probability equal to q. Represents the batch size. The reason for using Poisson sampling is that Poisson sampling can maintain the statistical characteristics of the data, such as mean value, variance, etc., while introducing noise, so as to ensure that when performing statistical analysis on the disturbed data, the results still have a certain degree of credibility and accuracy.

[0049] S2. Each local client obtains the global model from the central server.

[0050] The local client here refers to the hospital client.

[0051] S3. Each client uses local data for local training and performs differential privacy protection on the trained model parameters.

[0052] The client uses the obtained global model as its initial local model L kFor training, each hospital uses local data D k Train the local model L k .

[0053] In S3, Gaussian noise is added to ensure relaxed differential privacy, preventing attackers from inferring the hospital's local data by attacking the gradient;

[0054] In this process, the federated adaptive threshold optimization method proposed in the present invention is used to reduce the loss and improve the final performance;

[0055] The present invention also proposes a flexible model calibration strategy to effectively limit the impact of variable local updates.

[0056] Specifically, each client uses the local model to perform local training, and then performs differential privacy protection on the trained parameters. In this process, the Differential Privacy Stochastic Gradient Descent (DPSGD) method is preferred to calculate the clipped gradient. After clipping the gradient, the training data set information contained in the gradient can accurately estimate the privacy budget (ε, δ).

[0057] First, gradient clipping is performed. Introducing differential privacy protection usually sacrifices certain model performance. Clipping gradients helps find a balance between privacy protection and model accuracy. By adjusting the clipping threshold, the range of gradient variation can be controlled, thereby affecting the amount of noise added. Let C represent the maximum value of the L2 norm of each sample gradient. Since there is no prior limit on the gradient size, each gradient is clipped within the L2 norm in this invention. In other words, the gradient vector g is replaced by Where C is the clipping threshold, that is, the maximum value of the L2 norm of the gradient of each sample. This clipping ensures that if ||g||2≤C, g remains unchanged, and if ||g||2>C, it is scaled to a vector with norm C.

[0058] Algorithm 1 is as follows:

[0059]

[0060]

[0061]

[0062]

[0063] During the encryption process, preferably, a Gaussian mechanism is added to the relaxed differential privacy. The Gaussian mechanism is used because the vector-valued Laplace mechanism requires the use of L1 sensitivity, while the vector-valued Gaussian mechanism allows the use of L1 or L2 sensitivity. This is a major advantage of the Gaussian mechanism. The Gaussian mechanism does not satisfy pure ε-differential privacy, but satisfies (ε,δ)-differential privacy. For applications where L2 sensitivity is much lower than L1 sensitivity, the Gaussian mechanism allows the addition of less noise, which will improve the final accuracy; the Gaussian noise added in the present invention is:

[0064]

[0065] Here is the proof:

[0066] For a hospital dataset, according to the relaxed differential privacy method, assuming Used by hospital k to replace Become the data set of hospital k, then and There is only one sample difference, so the L2 norm Since each sample is bounded by the L2 norm, and is the difference data set of hospital k in round t, so we have:

[0067]

[0068] Then for all k∈K, we have:

[0069]

[0070] in As the weight, simplify the sensitivity formula to get:

[0071]

[0072] According to the Gaussian mechanism formula, we have Therefore, the Gaussian noise in the present invention satisfies:

[0073]

[0074] S4. Each client uploads the trained and encrypted model parameters to the central server.

[0075] S5. The central server performs weighted averaging on the uploaded model parameters and updates the global model.

[0076] The updated global model is then sent to the local client.

[0077] The weighted average is as follows: The calculation method of the weight wt after obtaining the security aggregation in the t-th federated learning training iteration is as follows:

[0078]

[0079] Due to the inherent data heterogeneity problem, there will always be a big difference between the local model and the global model. Too many local updates may cause the model to diverge due to the underlying heterogeneous data. Therefore, the present invention uses a flexible model calibration strategy to effectively limit the impact of variable local updates. Specifically: first, the weight correction loss is used to quantify the drift between the global model of the last round and its adjacent communication round model. Then the corrected model is obtained by minimizing the weight loss, and the weighted sum of the corrected model and the previous round model is taken as the final result. The model calibration function is shown as follows:

[0080]

[0081] Where μ is a positive number used to balance the weight between the objective function and the regularization term. The regularization term penalizes the variable w t and w t-1 The square of the Euclidean distance between t and w t-1 By adding this term, each device not only optimizes its own local loss function, but also tries to get as close to the global model as possible.

[0082] It can be seen from the above formula that the value of μ is crucial. A large μ may slow down the convergence speed and cause divergence by forcing the update to be close to the starting point, while a suitable μ can constrain the local update trajectory to make it closer to the global model and ensure convergence. For the setting of μ, the present invention proposes an adaptive threshold optimization method, specifically: if the current loss is greater than the loss of the previous round, increase the value of μ; if the current loss has been continuously less than the loss of the previous round λ rounds, reduce the value of μ. This method can make the training loss have a faster convergence speed.

[0083] The above process is repeated until the specified number of rounds is reached. Specifically, if the current number of federated learning training iterations i does not exceed the threshold specified by the user, the training process will continue and the results will be passed to S2. Otherwise, further iterations will be terminated.

[0084] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or replace some or all of the technical features therein with equivalents. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for diabetic retinopathy image recognition based on federated learning and differential privacy, It is characterized in that include: Obtain a dataset of diabetic retinopathy images and preprocess the data in the dataset. The central server uses the preprocessed data to initialize the global model. The central server iteratively updates the global model until the specified number of iterations is reached; Each iteration includes: The local client obtains the parameters of the global model from the central server and builds the local model; The local client trains the local model using local data, and performs differential privacy protection on the trained model parameters; the differential privacy is relaxed differential privacy with Gaussian noise; The local client uploads the trained and encrypted model parameters to the central server; The central server performs weighted averaging on the uploaded model parameters and updates the global model; The updated global model is used to perform diabetic retinopathy image recognition to obtain diabetic retinopathy image recognition results.

2. According to the method for diabetic retinopathy image recognition based on federated learning and differential privacy according to claim 1, It is characterized in that The data preprocessing includes: processing the diabetic retinopathy images uniformly into a size of 224×224.

3. According to the method for diabetic retinopathy image recognition based on federated learning and differential privacy according to claim 2, It is characterized in that The data preprocessing also includes: extracting a portion of samples from the diabetic retinopathy data set in each hospital by Poisson sampling.

4. The diabetic retinopathy image recognition method based on federated learning and differential privacy according to claim 1, It is characterized in that The local client trains the local model using local data, including: Calculate the gradient values ​​of model parameters; Gradient clipping is performed based on the gradient clipping threshold.

5. According to claim 4, the diabetic retinopathy image recognition method based on federated learning and differential privacy, It is characterized in that Gradient clipping is performed based on the gradient clipping threshold, including: The gradient vector g is replaced by Where C is the clipping threshold.

6. The diabetic retinopathy image recognition method based on federated learning and differential privacy according to claim 4, It is characterized in that And perform differential privacy protection on the trained model parameters, including: Use Gaussian mechanism to add noise to the clipped gradient.

7. The diabetic retinopathy image recognition method based on federated learning and differential privacy according to claim 1, It is characterized in that Also includes: Broadcast the updated global model parameters to each local client; The local client updates the local model using the updated parameters.

8. The diabetic retinopathy image recognition method based on federated learning and differential privacy according to claim 7, It is characterized in that The local client updates the local model using the updated parameters, including: The weight correction loss is used to quantify the drift between the global model of the last round and its adjacent communication round models; The corrected model is obtained by minimizing the weight correction loss, and the weighted sum of the corrected model and the previous round model is taken as the final result.

9. The diabetic retinopathy image recognition method based on federated learning and differential privacy according to claim 8, It is characterized in that The corrected model is obtained by minimizing the weight correction loss, including: The loss is corrected using the model calibration function to obtain a corrected model, where the model calibration function is: Where μ is a positive number used to balance the weight between the objective function and the regularization term; the regularization term penalizes the variable w t and w t-1 The square of the Euclidean distance between t and w t-1 The proximity between them is constrained.

10. The diabetic retinopathy image recognition method based on federated learning and differential privacy according to claim 9, It is characterized in that An adaptive threshold optimization method is used to set μ, including: if the current loss is greater than the loss in the previous round, then increase the value of μ; if the current loss has been smaller than the loss in the previous round for λ rounds in a row, then decrease the value of μ.

Images