Data privacy protection method and system based on federated learning and differential privacy

Abstract

The invention discloses a data privacy protection method and system based on federated learning and differential privacy, and relates to the technical field of data security sharing, and the method comprises the steps: transmitting parameters of a federated learning global model to each client, disturbing local data through differential privacy, and transmitting the disturbed local data to the client; training the global model through disturbed local data in each client to obtain local model parameters; updating and obtaining a reputation value according to the model parameters of the clients, judging the reputation value and a threshold value, screening out the clients meeting the conditions, updating the model parameters uploaded by the clients meeting the conditions, aggregating the model parameters by using a weighted average aggregation algorithm, and updating a federated learning global model; and sending the updated global model to a client node, traversing the process of updating the global model until a converged global model is obtained, and performing data security sharing. According to the method, the problem that good balance is difficult to obtain between sensitive data privacy and learning task model performance is solved.

Description

Technical Field

[0001] The present invention relates to the technical field of data security sharing, and in particular to a data privacy protection method and system based on federated learning and differential privacy. Background Art

[0002] With the intensifying global aging trend, smart elderly care is becoming an important approach to addressing the care needs of elderly people living alone. Leveraging technologies such as the Internet of Things (IoT), big data, and artificial intelligence (AI), smart elderly care provides seniors with more intelligent, convenient, and personalized elderly care services. In smart elderly care systems, IoT devices can collect real-time information about the elderly's physiological data, behavioral habits, and living environment. This data provides a solid foundation for health monitoring, disease prediction, and lifestyle support. However, due to the sensitive personal information involved, ensuring data security, privacy protection, and information privacy protection are key challenges in the application and promotion of smart elderly care systems.

[0003] Deep learning has been widely used in the elderly care sector, for example, leveraging pre-trained deep learning models to classify diseases. Traditional deep learning is centralized, uploading all data to the cloud and using the models for centralized learning. This leads to serious privacy and security issues. Furthermore, due to the high privacy and security characteristics of data in the smart elderly care sector, institutions holding these data have been reluctant to share their data for privacy and security reasons, leading to data silos.

[0004] Federated learning and differential privacy are two technologies that have garnered widespread attention in the field of data privacy protection in recent years. Federated learning is a distributed machine learning technology that allows data to remain on the device. By training the model locally, the client only uploads the model parameters, not the original data, thereby achieving data sharing and protecting data privacy. Differential privacy, on the other hand, further reduces the risk of information leakage by adding random noise to the data or model parameters, effectively protecting data privacy. In smart elderly care scenarios, combining federated learning and differential privacy can achieve privacy protection and secure sharing of elderly data distributed across various elderly care devices.

[0005] In traditional smart elderly care data sharing methods, although federated learning and differential privacy have provided certain guarantees for data privacy protection, they still face many challenges in practical applications, such as excessive communication overhead and the presence of malicious clients leading to reduced model accuracy.

[0006] To prevent attackers from stealing model parameters and restoring the client's original private data during federated learning training, existing methods apply differential privacy technology to federated learning. However, improper application of differential privacy in federated learning can lead to problems such as poor model performance and weak privacy. In addition, during the federated learning process, there is a possibility that malicious clients may exist among the participating clients, which may launch poisoning attacks and thus damage the global model performance. Summary of the Invention

[0007] The purpose of the present invention is to address the deficiencies of the above-mentioned prior art and provide a data privacy protection method and system based on federated learning and differential privacy to solve the problems in the prior art.

[0008] The present invention specifically provides the following technical solutions:

[0009] A data privacy protection method based on federated learning and differential privacy, including:

[0010] The global model parameters of federated learning are sent to each client. The local data is perturbed through differential privacy. Each client uses the perturbed local data to train the received global model to obtain the local model and local model parameters. The local model parameters are then compared with the global model parameters to obtain the updated model parameters.

[0011] Obtain a reputation value based on the model parameter updates uploaded by the client, identify clients with a reputation value above the threshold as qualified clients, aggregate the model parameter updates uploaded by qualified clients, and update the global model of federated learning;

[0012] Sending the updated global model to the client node, and traversing the process of updating the global model until a converged global model is obtained;

[0013] Secure data sharing via converged global models.

[0014] Preferably, before sending the global model parameters of the federated learning to each client, the global model parameters of the federated learning are initialized. The global model parameters include the learning rate η, the privacy parameter ε, the feature sensitivity threshold parameter C, the Gaussian noise standard deviation factor And the initial honor value R of each client i , where Δf is the feature change and δ is the failure probability.

[0015] Preferably, after sending the parameters of the global model to each client, the method further includes:

[0016] Each client uses IoT devices to collect data, cleans and standardizes the data, extracts features from the cleaned and standardized data through a convolutional neural network, and perturbs the extracted features using differential privacy methods.

[0017] Preferably, the data cleaning and standardization operations are performed on the data, features are extracted through a convolutional neural network, and the extracted features are perturbed using a differential privacy method, including:

[0018] The image data of the local data is recorded as X, and the image data is converted into a tensor. The image data X′ converted into a tensor is normalized. The specific expression is:

[0019]

[0020] Among them, μ and σ are the mean and standard deviation respectively;

[0021] The normalized image X″ is passed through the convolutional layer f of the deep learning network ResNet Generate feature map F, the specific expression is:

[0022] F=f ResNet (X″,W);

[0023] Among them, W is the weight matrix of the convolutional layer;

[0024] Input the feature map into the fully connected layer g of the deep learning network ResNet , obtain the eigenvector z, the specific expression is z = g ResNet (F);

[0025] The obtained feature vector is clipped using the L2 norm, and Gaussian noise in differential privacy is added to the clipped feature vector to perturb the feature vector.

[0026] Preferably, the global model is trained in each client using the disturbed local data to obtain the local model and local model parameters, including:

[0027] Get the local dataset D of i clients i ={(x j ,y j )}, where x i is the input image data, y j is the corresponding label, y j ∈{0,1}, indicating two categories;

[0028] The client uses the local data set to train the global model and obtain the local model w i , where the parameter set of the local model is

[0029] Based on the true category of the training samples in the local data set and the output category, the loss is calculated, the parameters of the local model are optimized, and the optimal local model parameters are obtained; the loss function calculates the training loss L i The formula is:

[0030]

[0031] in y j denote the predicted probability of the label and the true label respectively.

[0032] Preferably, the comparing of the local model parameters with the global model parameters to obtain the model parameter update is specifically as follows:

[0033] The trained local model parameters are compared with the global model parameters to obtain the model parameter updates. The top k updates are selected based on the k value in the Top-k algorithm. The k value is dynamically updated based on the client update change in this round and the degree of convergence of the global model. Unselected updates are set to 0. The sparse update matrix is obtained and uploaded to the server. The steps for updating the local model parameters are as follows:

[0034] Calculate the trained local model parameters and global model parameters The difference between them is used to get the model parameter update vector, which is specifically expressed as:

[0035]

[0036] Update vector Δθ for each model parameter i Take the absolute value And sort the importance of the model parameter update vector according to the absolute value. The specific expression is:

[0037]

[0038] Among them, Top-k is an adaptive selection algorithm, j represents the jth parameter in the model parameter update vector, {j1,j2,...,j k} is the index of the model parameter, indicating the k updates with the largest absolute value;

[0039] Initialize the k value and dynamically update it in different training rounds based on the client update change and the degree of convergence of the global model. After each round of training, the client calculates the update change between its local model and the global model. If the change is greater than a threshold, select the model parameter updates other than the k updates and upload them.

[0040] Preferably, obtaining a reputation value based on the model parameter updates uploaded by the clients, treating clients with reputation values higher than a threshold as qualified clients, and aggregating the model parameter updates uploaded by the qualified clients to update the global model of federated learning includes:

[0041] Assign an initial reputation value to each client Two adjustment factors α and β, and the minimum similarity threshold θ for increasing credibility sim ;

[0042] Update the model parameters uploaded by the client and use the median model parameter to update the vector Calculate the update deviation of each client;

[0043] Calculate the client's model parameter update vector through cosine similarity The similarity between the update deviation of each client is expressed as:

[0044]

[0045] in, is the model parameter of the i-th client, is the model parameter of the median of the i-th client;

[0046] The similarity is compared with the threshold θ, and the reputation value of each client is dynamically updated after each round of training:

[0047]

[0048] Among them, reputation value The initial value is is the reputation value of the i-th client after update;

[0049] The reputation value of each client is normalized, and a weighting factor is assigned based on the normalized reputation value of the client and the proportion of the data set. The client model parameter update vector is weightedly aggregated using a weighted average algorithm to obtain the updated global model and global model parameters. The updated global model parameters are then distributed to clients whose reputation value is higher than the threshold.

[0050] The present invention provides a data privacy protection system based on federated learning and differential privacy, comprising:

[0051] The training module is used to send the global model parameters of federated learning to each client, perturb the local data through differential privacy, and train the received global model on each client using the perturbed local data to obtain the local model and local model parameters. The local model parameters are then compared with the global model parameters to obtain the model parameter update;

[0052] The update module is used to obtain reputation values based on model parameter updates uploaded by clients, identify clients with reputation values above the threshold as qualified clients, aggregate model parameter updates uploaded by qualified clients, and update the global model of federated learning;

[0053] A traversal module, configured to send the updated global model to the client node and traverse the process of updating the global model until a converged global model is obtained;

[0054] Sharing module for secure data sharing through converged global models.

[0055] The present invention provides a computer device, including a memory and a processor, wherein a program is stored in the memory. When the program is executed by the processor, the processor performs the steps of the above-mentioned data privacy protection method based on federated learning and differential privacy.

[0056] The present invention provides a storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the above-mentioned data privacy protection method based on federated learning and differential privacy.

[0057] Compared with the prior art, the present invention has the following significant advantages:

[0058] The present invention protects local data characteristics through differential privacy, introduces controllable noise to destroy individual sensitive information, and retains the overall statistical characteristics of the data set, thereby improving the privacy protection strength of local data. The local model parameters are compared with the global model parameters to obtain the updated model parameters. The reputation value is obtained according to the updated model parameters of the client. The clients with reputation values higher than the threshold are regarded as qualified clients, and the model parameter updates uploaded by the qualified clients are aggregated to update the federated learning global model. The present invention proposes a secure aggregation scheme based on client reputation. The scheme reduces the possibility of malicious clients destroying the global model to a certain extent, thereby conducting federated learning training more safely and fairly. The updated global model is sent to the client node, and the process of updating the global model is traversed until a converged global model is obtained. The global model is used to protect the privacy of other honest clients, overcoming the problem of finding a good balance between the privacy of sensitive data and the performance of the learning task model, and achieving the effect of reasonably ensuring data privacy and model accuracy. BRIEF DESCRIPTION OF THE DRAWINGS

[0059] Figure 1 The figure is a flowchart of a data privacy protection method based on federated learning and differential privacy in an embodiment of the present invention.

[0060] Figure 2 This is a detailed structural diagram of a data privacy protection method based on federated learning and differential privacy provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0061] The following is a clear and complete description of the technical solutions of the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are part of the embodiments of the present invention, not all of them. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts should fall within the scope of protection of the present invention.

[0062] See also Figure 2 In this embodiment, a smart elderly care data security sharing system based on federated learning and differential privacy is provided, which includes three parts: user end, client end and server end. The user end is a terminal device located in the home of the elderly at home, which is used to collect data. It mainly uses IoT devices to collect the elderly’s voice data. The client end is the terminal device in each community participating in federated learning training, such as desktop computers, laptops, etc. The server end is a medical institution or nursing home center, which is used to remotely monitor the health of the elderly.

[0063] according to Figure 1 , a data privacy protection method based on federated learning and differential privacy provided in this embodiment includes the following steps:

[0064] Step S1: Send the global model parameters of federated learning to each client, perturb the local data through differential privacy, train the received global model with the perturbed local data in each client to obtain the local model and local model parameters, and compare the local model parameters with the global model parameters to obtain the model parameter update.

[0065] Initialize the global model parameters of federated learning, including the learning rate η, privacy parameter ε, feature sensitivity threshold parameter C, and Gaussian noise standard deviation factor. And each client's honor value R i .

[0066] After sending the global model parameters to each client, it also includes:

[0067] Each client uses IoT devices to collect data, pre-processes the data such as data cleaning and standardization, extracts features from the cleaned and standardized data through convolutional neural networks, and perturbs the extracted features using differential privacy methods.

[0068] Perform data cleaning and standardization operations on the data, extract features through convolutional neural networks, and use differential privacy methods to perturb the extracted features, including:

[0069] The image data of the local data is recorded as X, and the image data is converted into a tensor. The image data X′ converted into a tensor is normalized. The specific expression is:

[0070]

[0071] Among them, μ and σ are the mean and standard deviation respectively, and random cropping, rotation, translation, scaling, and flipping data enhancement methods are performed to achieve high-quality output of the data.

[0072] Use the deep learning network to extract image features, and pass the standardized image X″ through the convolution layer f of the deep learning network ResNet Generate feature map F, assuming that the weight matrix of the convolution layer is W, the feature extraction formula is:

[0073] F=f ResNet (X″,W);

[0074] Input the feature map into the fully connected layer g of the deep learning network ResNet , obtain the eigenvector z, the specific expression is z = g ResNet (F), represents the high-dimensional feature representation of image data.

[0075] Differential privacy ensures that a randomized algorithm M satisfies the following conditions for any two inputs x and x′ and all output sets S: Pr[M(x)∈S]≤e ε ·Pr[M(x′)∈S]+δ.

[0076] The obtained feature vector is clipped using the L2 norm, and the Gaussian noise in differential privacy is added to the clipped feature vector to perturb the feature vector. The clipping formula is:

[0077]

[0078] Among them, C is the clipping threshold, and the Gaussian noise N(0,σ 2 ) into the clipped feature vector: Among them, the variance σ is expressed as:

[0079]

[0080] Where ε is the privacy budget and δ is the failure probability.

[0081] Client local training step: In each client, the global model is trained using the perturbed local data to obtain the local model and local model parameters, including:

[0082] The default learning task in this invention is to determine whether the elderly have a certain disease. Get the local data set D of i clients i ={(x j ,y j )}, where x i is the input image data, y j is the corresponding label, y j ∈{0,1}, indicating two categories.

[0083] If there are N participants collaborating to train a deep learning model, {D1,D2,...D n} represent the local dataset of each client respectively. The client uses the local dataset to train the global model and obtains the local model w i ,set up is the parameter set of the local model; based on the true category of the training samples in the local dataset and the output category, the loss is calculated, and then the parameters of the local model are optimized to obtain the optimal local model parameters; when training the local model, the formula for calculating the training loss of the loss function is: in y j denote the predicted probability of the label and the true label respectively.

[0084] The local model parameters are compared with the global model parameters to obtain the updated model parameters. That is, each client uploads the updated local model parameters to the server using the Top-k adaptive selection algorithm. Specifically:

[0085] The client compares the trained local model parameters with the global model parameters to obtain model parameter updates. The top k updates are selected based on the k value in the Top-k algorithm. The k value is dynamically updated based on the client update change in that round and the degree of convergence of the global model. Unselected updates are set to 0, thereby obtaining a sparse update matrix and uploading it to the server. The steps for updating local model parameters are as follows:

[0086] Client i calculates local model parameters and global model parameters The difference between them is used to get the model parameter update vector, which is specifically expressed as:

[0087] Update vector Δθ for each client's model parameters i Take the absolute value Where j represents the jth parameter in the model parameter update vector, and the importance of the model parameter update vector is sorted according to the absolute value, and the top k update items with the largest absolute value are selected to generate the preliminary index set Λ t , the specific expression is:

[0088]

[0089] Among them, Top-k is an adaptive selection algorithm, j represents the jth parameter in the model parameter update vector, {j1,j2,...,j k} are the indices of the model parameters, representing the k updates with the largest absolute values.

[0090] The server initializes the k value, and in different training rounds, the client dynamically updates the k value based on the client update change and the degree of convergence of the global model. After each round of training, the client calculates the update change between its local model and the global model. If the change is greater than the threshold, it indicates that the client update may have made a significant contribution to the global model. In this case, more model parameter updates in addition to the k updates are selected and uploaded. At the same time, as the number of training rounds increases, the degree of convergence of the model gradually improves. At this time, the number of updates uploaded by the client should be appropriately reduced. The specific method is as follows:

[0091] Dynamically calculate the k value based on the client update change: Assume that the model update change of client i in round t is Δθ i , the global model is Set k respectively max ,k min is the maximum and minimum update thresholds, and the k value of the change in the client update in round t is t The choices are as follows:

[0092]

[0093] where α is the regulating factor, This is the maximum value of all client updates.

[0094] Dynamically adjust the k value according to the degree of convergence of the global model: Assume that the parameters of the global model in round t and the global model in the previous round are Adjust k based on the change in the global model:

[0095]

[0096] According to the above two strategies, a weighted average is performed according to the number of training rounds, and the weight coefficient is adjusted so that the k value is suitable for the training target. The final k value can be expressed as:

[0097]

[0098] Where N is the number of clients, t is the current training round, β1, β2, and β3 are weight factors for each strategy, satisfying β1+β2+β3=1. Each weight factor controls the influence of the strategy in the final k value calculation.

[0099] After each round of training, the client calculates k i The value selects the update to be uploaded, and the unselected updates are set to zero to form a sparse update matrix. For the update matrix of the client in round t, select the first k i Updates Set the unselected updates to 0 to get a sparse update matrix:

[0100]

[0101] A Top-K adaptive algorithm is proposed to realize an efficient federated learning process in a distributed environment, reduce the parameter quantity problem when transferring model parameters and reduce certain communication bottlenecks, which is more conducive to the training process of federated learning and reduces the impact of redundant parameters on communication quality.

[0102] Step S2: Obtain a reputation value based on the model parameter updates uploaded by the client, identify clients with a reputation value higher than the threshold as qualified clients, aggregate the model parameter updates uploaded by the qualified clients, and update the global model of federated learning.

[0103] The credibility value is obtained based on the updated model parameters of the client. Clients with a credibility value higher than the threshold are considered qualified clients. The model parameters uploaded by qualified clients are aggregated to update the federated learning global model, including:

[0104] Assign an initial reputation value to each client Two adjustment factors α and β, and the minimum similarity threshold θ for increasing credibility sim .

[0105] The server performs cosine similarity calculation on the model parameter updates uploaded by the client to determine the credibility of each client update.

[0106] The server performs anomaly detection based on the model parameter update uploaded by the client, and uses the median model parameter update vector To calculate the update deviation of each client;

[0107] Calculate the client's model parameter update vector through cosine similarity The similarity between the update deviation of each client is expressed as:

[0108]

[0109] in, is the model parameter of the i-th client, is the model parameter of the median of the i-th client.

[0110] Compare the similarity with the threshold θ and dynamically update the reputation value of each client after each round of training:

[0111]

[0112] Among them, reputation value The initial value is is the reputation value of the i-th client after the update. The reputation values of each client are normalized:

[0113] R i =max(0,min(1,R i ));

[0114] The server assigns a weighting factor based on the client's normalized reputation value and the proportion of the data set. Use the weighted average algorithm to perform weighted aggregation on the client model parameter update vector: An updated global model and global model parameters are obtained, and the updated global model parameters are distributed to clients whose reputation values are higher than a threshold.

[0115] Step S3: Send the updated global model to the client node, and traverse the process of updating the global model until a converged global model is obtained.

[0116] Step S4: Secure data sharing through the converged global model.

[0117] In a specific embodiment, the present invention is specifically:

[0118] Server initialization steps: The server initializes the system and determines the federated learning task, such as identifying Alzheimer's disease. After determining the learning task, the server's hardware and software environment is configured, and the network communication protocol is set to ensure unimpeded communication between all participants.

[0119] Client registration step: The server sends each client's identifier through a secure communication channel It is sent to each client, and the server stores the information in a list format.

[0120] Global model initialization step: After determining the federated learning task, the server selects or designs an appropriate global W based on the learning task. g , and set the initial parameters of the model.

[0121] User-side data collection steps: The user side is the elderly at home in each community, and the smart elderly care data is collected through IoT devices and recorded as x i ,This step is also the step of collecting local smart elderly care privacy data, such as voice data, image data, electronic medical record data, medical imaging data and physiological data, and uploading it to the client through a secure channel.

[0122] Client data preprocessing steps: This example takes voice data as an example. The IoT device in the elderly’s home collects the elderly’s voice signal x(t) in the time domain t. A filter (such as a bandpass filter, mean filter, etc.) is used to remove background noise to obtain the processed signal y[n] = Filter(x[n], f low ,f high ), where f low and f high The speech signal is divided into small frames, each frame is L sampling points long, the frame shift is P sampling points, and the framing process is: Frame k [n]=y[kP+n], where 0≤n≤L-1. The spectrum of each frame signal is extracted through Fourier transform to obtain the spectrum value: Then, the Mel-frequency cepstral coefficient (MFCC) is used to extract the features of the speech signal and obtain the Mel-frequency energy spectrum. Among them H m (f) is the frequency response of the m-th Mel filter.

[0123] Client local training step: Assume that the client's local dataset is where x i is the input feature, y i is the label, n is the number of samples in the local dataset of client k. The client uses D k To train the global model w g Get the local model θ k , and updates its local model parameters through gradient descent in each training iteration. Assume that the learning task is a binary classification problem to identify whether the elderly have a disease, as follows:

[0124] 01: For the i-th sample of client k, the feature Add Gaussian noise Get the features after adding noise

[0125] 01: Define the loss function L(θ,x i ,y i ), which measures the model's performance on a single sample (x i ,y i ), for the two-class problem, the cross entropy loss function is used: L(θ,x i ,y i )=-y i log(f(x i ,θ))-(1-y i )log(1-f(x i ,θ)), where f(x i ,θ) is the model output, y i is the true label.

[0126] 02: Calculate the loss function for model parameters Gradient:

[0127] 03: Update local model parameters: Use gradient descent method to update parameters, and the updated local model parameters for: where η is the learning rate.

[0128] Local model update parameter upload step: When the client uploads the local model update parameters, it updates the top k most important model updates according to the model update. The k value is dynamically adjusted by the number of model training rounds, the update change amount, and the degree of convergence of the global model. The following are the specific steps of this step:

[0129] 01: Calculate model parameter updates and sort: Update vector Δθ for each client's model parameters i Take the absolute value And sort the importance of the model parameter update vector by size: Select the k updates with the largest absolute values.

[0130] 02: The change in the client update in round t, k t The choices are as follows: where α is the regulating factor, This is the maximum value of all client updates.

[0131] 03: Dynamically adjust the k value according to the degree of convergence of the global model:

[0132] 04: Perform weighted averaging based on the number of training rounds and adjust the weight coefficient to obtain the final k value: Where N is the number of clients, t is the current training round, β1, β2, and β3 are weight factors for each strategy, satisfying β1+β2+β3=1. Each weight factor controls the influence of the strategy in the final k value calculation.

[0133] 05: Sparse update matrix: For the update matrix of the client in round t, select the first k i Updates Set the unselected updates to 0 to get a sparse update matrix

[0134]

[0135] Server-side security aggregation steps: After the client uploads the local model update parameters to the server, the server performs security aggregation based on the reputation value of each client. The specific steps are as follows:

[0136] 01: The server detects anomalies in the model update uploaded by the client and uses the median model parameter update vector To calculate the update deviation of each client, use cosine similarity comparison

[0137] 02: Dynamically update the client's reputation value based on cosine similarity:

[0138]

[0139] 03: Normalize the client's reputation value: R i =max(0,min(1,R i )).

[0140] 04: Anomaly detection and removal, if the reputation value R of some clients i Continuously below the threshold R t , the server removes it from the set of clients participating in training.

[0141] 05: The server assigns a weighting factor based on the client's reputation and the proportion of the dataset Use weighted average algorithm for secure aggregation: The updated global model parameters are then distributed to clients with reasonable reputation values.

[0142] Global model evaluation step: Server-side evaluation of the global model if If convergence has not yet occurred, the process starts again from the client-side local model training step and repeats all subsequent steps until the global model converges. If the global model has converged, the global model is distributed to all clients, and the federated learning process ends.

[0143] After learning, individual medical institutions or nursing homes can use local models to implement smart elderly care and improve telemedicine needs. Smart elderly care applications can include assisted diagnosis and treatment, disease prediction, remote monitoring, and health management. This can significantly improve smart elderly care services, help the elderly better care at home, significantly reduce medical costs, monitor patients' health in real time, and enable medical institutions or nursing homes to take treatment actions when necessary. It also provides users with personalized health advice and lifestyle improvement plans.

[0144] Based on the above method, the present invention provides a data privacy protection system based on federated learning and differential privacy, including: a training module, an update module, a traversal module and a sharing module.

[0145] Among them, the training module is used to send the global model parameters of federated learning to each client, perturb the local data through differential privacy, and train the received global model with the perturbed local data in each client to obtain the local model and local model parameters, and compare the local model parameters with the global model parameters to obtain the model parameter update; the update module is used to obtain the reputation value based on the model parameter update uploaded by the client, and regard the client with a reputation value higher than the threshold as a qualified client, and aggregate the model parameter updates uploaded by the qualified clients to update the global model of federated learning; the traversal module is used to send the updated global model to the client node, and traverse the process of updating the global model until a converged global model is obtained; the sharing module is used to securely share data through the converged global model.

[0146] The present invention also provides a computer device, including a memory and a processor, wherein a program is stored in the memory. When the program is executed by the processor, the processor executes the steps of a data privacy protection method based on federated learning and differential privacy.

[0147] According to the disclosed embodiments, a computing device may communicate with one or more external devices (e.g., a keyboard, a pointing device, Bluetooth communications, etc.), or with any device that enables a computing device to communicate with one or more other computing devices (e.g., a router, a modem, etc.).

[0148] The present invention also provides a storage medium on which a computer program is stored. When the computer program is executed by a processor, the steps of a data privacy protection method based on federated learning and differential privacy are implemented.

[0149] According to the disclosed embodiments, the storage medium may be a non-volatile computer-readable storage medium, such as, but not limited to, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination thereof. In the present invention, the storage medium may be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, apparatus, or device.

[0150] The above content is a further detailed description of the present invention in combination with specific preferred embodiments. For those skilled in the art to which the present invention belongs, several simple deductions or replacements can be made without departing from the concept of the present invention, which should be regarded as falling within the scope of protection of the present invention.

Claims

1. A data privacy protection method based on federated learning and differential privacy, characterized in that: include: The global model parameters of federated learning are sent to each client. The local data is perturbed through differential privacy. Each client uses the perturbed local data to train the received global model to obtain the local model and local model parameters. The local model parameters are then compared with the global model parameters to obtain the updated model parameters. Obtain a reputation value based on the model parameter updates uploaded by the client, identify clients with a reputation value above the threshold as qualified clients, aggregate the model parameter updates uploaded by qualified clients, and update the global model of federated learning; Sending the updated global model to the client node, and traversing the process of updating the global model until a converged global model is obtained; Secure data sharing via converged global models.

2. The data privacy protection method based on federated learning and differential privacy according to claim 1, characterized in that: Before sending the global model parameters of the federated learning to each client, the global model parameters of the federated learning are initialized. The global model parameters include the learning rate η, the privacy parameter ε, the feature sensitivity threshold parameter C, the Gaussian noise standard deviation factor And each client's honor value R i , where Δf is the feature change and δ is the failure probability.

3. The data privacy protection method based on federated learning and differential privacy according to claim 1, characterized in that: After sending the parameters of the global model to each client, the method further includes: Each client uses IoT devices to collect data, cleans and standardizes the data, extracts features from the cleaned and standardized data through a convolutional neural network, and perturbs the extracted features using differential privacy methods.

4. The data privacy protection method based on federated learning and differential privacy according to claim 3, characterized in that: The data is cleaned and standardized, features are extracted through a convolutional neural network, and the extracted features are perturbed using a differential privacy method, including: The image data of the local data is recorded as X, and the image data is converted into a tensor. The image data X′ converted into a tensor is normalized. The specific expression is: Among them, μ and σ are the mean and standard deviation respectively; The normalized image X″ is passed through the convolutional layer f of the deep learning network ResNet Generate feature map F, the specific expression is: F=f ResNet (X″,W); Among them, W is the weight matrix of the convolutional layer; Input the feature map into the fully connected layer g of the deep learning network ResNet , obtain the eigenvector z, the specific expression is z = g ResNet (F); The obtained feature vector is clipped using the L2 norm, and Gaussian noise in differential privacy is added to the clipped feature vector to perturb the feature vector.

5. The data privacy protection method based on federated learning and differential privacy according to claim 1, characterized in that: The global model is trained in each client using the disturbed local data to obtain a local model and local model parameters, including: Get the local dataset D of i clients i ={(x i ,y j )}, where x i is the input image data, y j is the corresponding label, y j ∈{0,1}, indicating two categories; The client uses the local dataset to train the global model and obtains the local model w i , where the parameter set of the local model is Based on the true category of the training samples in the local data set and the output category, the loss is calculated, the parameters of the local model are optimized, and the optimal local model parameters are obtained; the loss function calculates the training loss L i The formula is: in y j denote the predicted probability of the label and the true label respectively.

6. The data privacy protection method based on federated learning and differential privacy according to claim 5, characterized in that: The comparison of the local model parameters with the global model parameters to obtain the model parameter update is specifically as follows: The trained local model parameters are compared with the global model parameters to obtain the model parameter updates. The top k updates are selected based on the k value in the Top-k algorithm. The k value is dynamically updated based on the client update change in this round and the degree of convergence of the global model. Unselected updates are set to 0. The sparse update matrix is obtained and uploaded to the server. The steps for updating the local model parameters are as follows: Calculate the trained local model parameters and global model parameters The difference between them is used to get the model parameter update vector, which is specifically expressed as: Update vector Δθ for each model parameter i Take the absolute value And sort the importance of the model parameter update vector according to the absolute value. The specific expression is: Among them, Top-k is an adaptive selection algorithm, j represents the jth parameter in the model parameter update vector, {j1,j2,...,j k } is the index of the model parameter, indicating the k updates with the largest absolute value; Initialize the k value and dynamically update it in different training rounds based on the client update change and the degree of convergence of the global model. After each round of training, the client calculates the update change between its local model and the global model. If the change is greater than a threshold, select the model parameter updates other than the k updates and upload them.

7. The data privacy protection method based on federated learning and differential privacy according to claim 1, characterized in that: The method obtains a reputation value based on the model parameter updates uploaded by the client, identifies the client with a reputation value higher than a threshold as a qualified client, aggregates the model parameter updates uploaded by the qualified client, and updates the global model of the federated learning, including: Assign an initial reputation value to each client Two adjustment factors α and β, and the minimum similarity threshold θ for increasing credibility sim ; Update the model parameters uploaded by the client and use the median model parameter to update the vector Calculate the update deviation of each client; Calculate the client's model parameter update vector through cosine similarity The similarity between the update deviation of each client is expressed as: in, is the model parameter of the i-th client, is the model parameter of the median of the i-th client; The similarity is compared with the threshold θ, and the reputation value of each client is dynamically updated after each round of training: Among them, reputation value The initial value is is the reputation value of the i-th client after update; The reputation value of each client is normalized, and a weighting factor is assigned based on the normalized reputation value of the client and the proportion of the data set. The client model parameter update vector is weightedly aggregated using a weighted average algorithm to obtain the updated global model and global model parameters. The updated global model parameters are then distributed to clients whose reputation value is higher than the threshold.

8. A data privacy protection system based on federated learning and differential privacy, characterized in that: include: The training module is used to send the global model parameters of federated learning to each client, perturb the local data through differential privacy, and train the received global model on each client using the perturbed local data to obtain the local model and local model parameters. The local model parameters are then compared with the global model parameters to obtain the model parameter update; The update module is used to obtain reputation values based on model parameter updates uploaded by clients, identify clients with reputation values above the threshold as qualified clients, aggregate model parameter updates uploaded by qualified clients, and update the global model of federated learning; A traversal module, configured to send the updated global model to the client node and traverse the process of updating the global model until a converged global model is obtained; Sharing module for secure data sharing through converged global models.

9. A computer device, characterized in that: The invention comprises a memory and a processor, wherein a program is stored in the memory, and when the program is executed by the processor, the processor performs the steps of a data privacy protection method based on federated learning and differential privacy as described in any one of claims 1 to 7.

10. A storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of a data privacy protection method based on federated learning and differential privacy according to any one of claims 1 to 7 are implemented.

Images