A privacy protection method, system, device and medium for federated learning

By combining differential privacy and secret sharing technology in federated learning, noise is generated and secret sharing shares aggregation is solved, the problem of client privacy leakage is achieved, and the privacy protection of data security sharing is reduced, and communication and computing complexity is reduced.

CN118590332BActive Publication Date: 2024-11-15SHANGHAI INFORMATION MANAGEMENT NETWORK TECH CO LTD
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202411074686.9
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2024-08-07
Publication Date
2024-11-15
Estimated Expiration
2044-08-07

AI Technical Summary

Technical Problem

When the existing federated learning technology uploads the training results to the server, the client's private information may be inferred by the attacker through the output distribution, resulting in privacy leakage problems.

Method used

Using the combination of differential privacy and secret sharing technology, noise data parameters are generated through the ratio of data sensitivity to privacy budget, noise is added to each data point, and a secret shared share is generated, uploaded to the server for aggregation, and updated the secret shared share of the client. Ensure that each round of aggregation process meets differential privacy and prevents privacy leakage.

Benefits of technology

Effectively prevent attackers from inferring sensitive information of a data by outputting distribution differences, protecting the privacy of global information, reducing communication and computing complexity, and optimizing data security sharing.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118590332B_ABST
    Figure CN118590332B_ABST
Patent Text Reader

Abstract

The present invention discloses a privacy protection method, system, device and medium for federated learning, which relates to the field of information security technology, and includes the following steps: multiple participants install initialized models on their respective clients; obtain and segment the original data to be protected; input the segmented original data into each initialized model respectively, obtain noise data parameters, and add noise to each data point through the noise data parameters; generate #imgabs0# secret sharing shares through each noise-added data point, and upload #imgabs1# secret sharing shares to a server to obtain aggregated shares, and update the secret sharing shares of each client through the aggregated shares. The present invention uploads the secret sharing share set to a server to obtain aggregated shares, so that each round of aggregation process satisfies differential privacy, prevents the adversary from inferring sensitive information of a piece of data through the difference in output distribution, and protects the privacy of global information.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the field of information security technology, and in particular to a privacy protection method, system, device and medium for federated learning. Background Art

[0002] In the current digital age, with the rapid development of big data, data security and privacy issues are becoming increasingly prominent. While pursuing data-driven decision-making and innovation, businesses and research institutions must address the challenge of secure data sharing. Although many companies have launched AI-based security services, such as content security auditing and intrusion detection, concerns about user privacy and commercial confidentiality make it difficult for companies to share raw data, leading to isolated technologies and services.

[0003] In recent years, federated learning, a cutting-edge distributed machine learning method, has garnered widespread attention worldwide. Its core concept is to allow multiple participants to jointly train a machine learning model while maintaining data privacy and security. With large amounts of data and numerous nodes participating in computation, optimizing algorithms to reduce computing resource consumption and communication costs has become a research hotspot.

[0004] While federated learning protects user privacy to a certain extent, existing attack scenarios indicate that it still suffers from privacy leakage issues, making additional privacy protection techniques essential. To alleviate this privacy leakage problem, many researchers have proposed solutions using the following three technologies: secure multi-party computation (SMC), homomorphic encryption (HE), and differential privacy (DP). Secure multi-party computation focuses on how to securely compute a specified function without a trusted third party, but at the same time, each participant needs to obtain relevant information about other participants beyond the sum of gradients. Since the learning process involves multi-party interactions, their models incur significant communication overhead. Therefore, scholars have proposed a federated learning privacy protection scheme based on homomorphic encryption, which utilizes additional homomorphic encryption to protect gradients from server attacks. However, their scheme fails to protect user privacy if users holding the same key collude with each other.

[0005] Given the limited communication and computing resources of smart IoT devices and the widespread applicability of differential privacy in deep learning, differential privacy technology has been widely used in federated learning. A user-level differential privacy federated learning algorithm has been proposed. The idea is that the server adds noise to the clipped gradients at each iteration. However, this work does not consider privacy protection during the parameter upload phase. When the client uploads training results to the server, private information about the client could be intercepted by a hidden attacker by inferring sensitive information about a piece of data through the output distribution, leading to privacy leakage. Summary of the Invention

[0006] The purpose of the present invention is to address the deficiencies of the above-mentioned existing technologies and provide a privacy protection method, system, device and medium for federated learning, so as to solve the problem in the existing technology that when the client uploads the training results to the server, the client's private information may be intercepted by a hidden attacker through the output distribution to infer sensitive information of a piece of data, thereby performing differential interception and causing privacy leakage.

[0007] The present invention specifically provides the following technical solution: a privacy protection method for federated learning, comprising the following steps:

[0008] Obtain multiple original data to be protected and divide the multiple original data into multiple groups of data; each group of data after division contains the number of participants , privacy budget and data sensitivity ;

[0009] The divided data sets are input into multiple initialized models respectively, based on the sensitivity of the data and privacy budgets The noise data parameters are obtained by the ratio of , and noise is added to each data point in the original data according to the noise data parameters; multiple initialized models are installed on the client of each participant respectively;

[0010] Generate a number matching the number of participants using each data point after adding noise secret sharing shares, and The secret sharing shares of each client are uploaded to the server to obtain the aggregated share, and the secret sharing shares of each client are updated through the aggregated share.

[0011] Preferably, based on the sensitivity of the data and privacy budgets The noise data parameters are obtained by the ratio of , and the specific expression is:

[0012]

[0013] in, is the noise data parameter.

[0014] Preferably, adding noise to each data point in the original data according to the noise data parameter comprises the following steps:

[0015] The noise type is selected based on the noise data parameters, and the noise type is selected according to the noise data sensitivity. Get the noise level by which noise is added to each data point.

[0016] Preferably, the noise type is selected based on the noise data parameters, and the noise type is selected according to the noise data sensitivity. Obtaining the noise level also includes the following steps:

[0017] Data sensitivity based on noisy data and the user's privacy preferences to generate a privacy budget ;

[0018] Set the initial privacy budget for the original data in each initialized model , the privacy budget and initial privacy budget Compare and obtain the initial remaining privacy budget ;

[0019] Generate noise through a Laplace distributed random number with a center of 0 and a scale of scale. If the noise is greater than the initial remaining privacy budget When the remaining privacy budget is As the noise level at this time; otherwise, the remaining privacy budget at this time The difference between the privacy budget parameter and the above mentioned privacy budget is taken as the remaining privacy budget at this time.

[0020] Preferably, based on the sensitivity of the data and privacy budgets The noise data parameters are obtained by the ratio of , which also includes the following steps:

[0021] Set the total privacy budget of the original data in each initialized model , and initialize the remaining privacy budget ;

[0022] Get the data sensitivity based on the noise data each time and the user's privacy preferences to generate a privacy budget Individual privacy budgets allocated in operations ;

[0023] The remaining privacy budget is updated by the difference between the total privacy budget and the individual privacy budget allocated each time, and the rejection or implementation judgment of the operation is made based on the updated remaining privacy budget.

[0024] Preferably, The process of uploading the secret sharing shares to the server to obtain the aggregated shares includes the following steps:

[0025] Use linear aggregation, nonlinear aggregation or aggregation by precision to Aggregating the secret sharing share sets, and generating a global model through the aggregated shares;

[0026] The linear polymerization process includes the following steps:

[0027] set up For the The model parameters of each client, For the client The number of data points, then the global model parameters The calculation expression is:

[0028]

[0029] in, is the number of clients, is the total number of data points;

[0030] The nonlinear polymerization process includes the following steps:

[0031] set up It is The model parameters of each client, is the corresponding weight, then the global model parameter The calculation expression is:

[0032]

[0033] Among them, the weight Dynamically adjust based on client performance and data volume;

[0034] The precision aggregation process includes the following steps:

[0035] set up It is The model parameters of each client, is the weight determined based on performance evaluation, then the global model parameters The update expression is:

[0036]

[0037] Among them, the weight Determined by the model's accuracy on local data.

[0038] Preferably, the updating of the secret sharing shares of each client by the aggregated share further includes the following steps:

[0039] The client downloads the secret sharing shares of each server to perform secret recovery and obtain the updated model.

[0040] The present invention provides a privacy protection system for federated learning, comprising:

[0041] The data acquisition module is used to obtain multiple original data to be protected and divide the multiple original data into multiple groups of data; each group of data after division contains the number of participants , privacy budget and data sensitivity ;

[0042] The noise adding module is used to input the segmented multiple groups of data into multiple initialized models respectively, based on the sensitivity of the data and privacy budgets The noise data parameters are obtained by the ratio of , and noise is added to each data point in the original data according to the noise data parameters; multiple initialized models are installed on the client of each participant respectively;

[0043] The secret sharing module is used to generate a secret that matches the number of participants using each data point after adding noise. secret sharing shares, and The secret sharing shares of each client are uploaded to the server to obtain the aggregated share, and the secret sharing shares of each client are updated through the aggregated share.

[0044] The present invention provides a computer device, comprising a memory and a processor, wherein a program is stored in the memory, and when the program is executed by the processor, the processor performs the steps of the above-mentioned privacy protection method for federated learning.

[0045] The present invention provides a storage medium having a computer program stored thereon, characterized in that when the computer program is executed by a processor, the steps of the above-mentioned privacy protection method for federated learning are implemented.

[0046] Compared with the prior art, the present invention has the following significant advantages:

[0047] The present invention divides the original data into multiple parts and inputs the divided original data into each initialized model respectively. Each participant can only access a part of the data allocated to it. The model obtains the noise data parameter by the ratio of data sensitivity to privacy budget, adds noise to each data point, and generates The client uploads the secret sharing share set to the server to obtain the aggregated share, so that each round of aggregation process satisfies differential privacy, preventing the adversary from inferring sensitive information of a piece of data through the difference in output distribution, thereby preventing the aggregation model from being exploited by the adversary and protecting the privacy of global information. BRIEF DESCRIPTION OF THE DRAWINGS

[0048] Figure 1 A framework flowchart of a privacy protection method for federated learning in an embodiment of the present invention. DETAILED DESCRIPTION

[0049] The following is a clear and complete description of the technical solutions of the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are part of the embodiments of the present invention, not all of them. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts should fall within the scope of protection of the present invention.

[0050] This invention combines federated learning and differential privacy technology to enhance the privacy protection effect of data security sharing. The core components include the fusion application of secret sharing mechanism and differential privacy technology, such as Figure 1 As shown in the figure, the system architecture design emphasizes optimizing communication efficiency and balancing privacy and efficiency. This includes reducing communication times, applying data compression techniques, implementing adaptive privacy protection strategies, and conducting performance evaluations. The key is to effectively train and optimize the model while ensuring data privacy, minimizing communication costs and computational complexity. Figure 1 In the figure, client A and client B are all clients in the system. Local model training means training the local model through the client's data. Differential privacy and secret sharing represent the data privacy protection processing. Model share means aggregating the secret sharing share using various aggregation methods. The aggregated result share is obtained by uploading the aggregated secret sharing share set to the server. The client downloads the aggregated result share to restore the global model.

[0051] In terms of technical implementation, the solution combines the Laplace mechanism for differential privacy with a secret sharing mechanism. This involves adding random noise to the gradient on the client side and using Lagrange interpolation for data encryption in secret sharing. The solution also emphasizes a balance between communication and computational overhead in practical applications, ensuring privacy protection without excessively sacrificing model performance and efficiency. Implementation details and technical parameters, such as noise control and secret sharing parameters, are adjusted based on specific application scenarios and data characteristics to achieve optimal privacy protection and data processing.

[0052] The main goal of designing a privacy protection strategy that integrates secret sharing and differential privacy is to leverage the technical advantages of secret sharing and differential privacy to build a federated learning environment that ensures data security and maintains privacy.

[0053] like Figure 1 As shown, the present invention addresses the shortcomings of the existing technology and proposes a privacy protection method for federated learning to solve the problems of the existing technology. Specifically, the method includes the following steps:

[0054] Step S1: Obtain multiple original data to be protected and divide the multiple original data into multiple groups of data; each group of data after division contains the number of participants , privacy budget and data sensitivity .

[0055] Step S2: Input the segmented data into multiple initialized models respectively, based on the data sensitivity and privacy budgets The noise data parameters are obtained by the ratio of , and noise is added to each data point in the original data according to the noise data parameters; multiple initialized models are installed on the client of each participant respectively.

[0056] Specifically:

[0057] Noise addition is a core component of the differential privacy framework. The appropriate noise type and magnitude are determined based on the data sensitivity and the desired privacy budget. There are two main types of noise: Laplace noise and Gaussian noise.

[0058] The process of adding noise is as follows:

[0059] Input: Dataset , For the data points, batch size , generating a network , discriminant network , privacy budget , data sensitivity , Adam optimizer parameters 、 and , noise variance .

[0060] Output: trained generative network .

[0061] 1: Initialize the generative network and discriminant network Parameters .

[0062] 2: For each cycle implement, For any time, is the maximum time.

[0063] 3: For each batch implement, is the batch size under maximum time.

[0064] 4: From the original data sample mid-sampled dataset Any data point in .

[0065] 5: For each ,calculate Gradient .

[0066] 6: Yes Add Laplace noise to get the gradient with Laplace noise added .

[0067] 7: Update the generation network Parameters .

[0068] 8: Yes conduct Updates using gradient descent.

[0069] 9: Update noise variance ; is the noise variance difference, is the initialization noise variance.

[0070] 10: If there is a remaining privacy budget <Single Privacy Budget ,but:

[0071] 11: Terminate training.

[0072] 12: Return to the generated network .

[0073] The above process first defines a function for generating Laplace noise, and then defines a function for adding noise to the gradient. In the example usage section, the present invention sets the privacy budget and the data sensitivity of the gradient, and then adds noise to a simulated gradient value.

[0074] Data sensitivity determination: First, evaluate the data sensitivity of the dataset, which is a key factor in determining the noise level. Datasets with high data sensitivity require stronger privacy protection.

[0075] Noise type selection: based on privacy budget and data sensitivity , select the appropriate noise type. Laplace noise is suitable for scenarios with low data sensitivity, while Gaussian noise is more suitable for scenarios with high data sensitivity.

[0076] Noise level calculation: Based on the selected noise type and data sensitivity , calculate the magnitude of the noise. The method for calculating the magnitude varies depending on the type of noise.

[0077] Data Noising: Add a calculated amount of noise to each data point to ensure the data meets the privacy budget.

[0078] In this step, based on the data sensitivity and privacy budgets The noise data parameters are obtained by the ratio of , and the specific expression is:

[0079]

[0080] in, is the noise data parameter, Budget for privacy.

[0081] In this step, noise is added to each data point in the original data using the noise data parameter, and the following steps are also included:

[0082] The noise type is selected based on the noise data parameter and the noise sensitivity is calculated based on the selected noise type and data sensitivity. Get the noise magnitude by which to add noise to each data point.

[0083] The noise type is selected based on the noise data parameters, and the noise type is selected based on the noise data sensitivity. Obtaining the noise level also includes the following steps:

[0084] Data sensitivity based on noisy data and the user's privacy preferences to generate a privacy budget .

[0085] Set the initial privacy budget for the original data in each initialized model , the privacy budget and initial privacy budget Compare and obtain the initial remaining privacy budget .

[0086] Generate noise through a Laplace distribution random number with a center of 0 and a scale of scale. If the noise is greater than the initial remaining privacy budget When the remaining privacy budget is As the noise level at this time; otherwise, the remaining privacy budget at this time The difference between the privacy budget parameter and the privacy budget is taken as the remaining privacy budget at this time.

[0087] This step uses an adaptive privacy protection mechanism algorithm, which aims to automatically adjust the noise level in differential privacy technology according to the complexity of the data processing task and the data sensitivity. It analyzes the characteristics of the data query or processing task to evaluate the data sensitivity and complexity. Based on these evaluation algorithms, an appropriate privacy budget is determined, which is a key parameter of differential privacy technology that affects the noise level and the strength of privacy protection. The algorithm calculates an appropriate noise level and adds this noise to data processing or queries to protect individual privacy. The final output is processed and privacy-protected data, maintaining data usefulness as much as possible. This mechanism allows for flexible privacy protection in different data processing scenarios, ensuring that user privacy is protected without unduly impacting data validity and usability. The privacy protection mechanism operates as follows:

[0088] Input: original data data, data sensitivity data_sensitivity, user preference user_preference, initial privacy budget .

[0089] Output: protected data.

[0090] 1: Initialize the remaining privacy budget .

[0091] 2: Laplace noise function LaplaceNoise(scale):

[0092] 3: Returns a Laplace distributed random number noise_scale with center at 0 and scale scale.

[0093] 4: Privacy function AdaptPrivacy(data_sensitivity, user_preference):

[0094] 5: noise_scale←data_sensitivity*user_preference.

[0095] 6: If ,but:

[0096] 7: noise_scale← .

[0097] 8: ←0.

[0098] 9: Otherwise:

[0099] 10: .

[0100] 11: Return noise_scale.

[0101] 12: Privacy mechanism function ApplyPrivacyMechanism(data,data_sensitivity,user_preference):

[0102] 13: scale←AdaptPrivacy(data_sensitivity,user_preference).

[0103] 14: noise←LaplaceNoise(scale).

[0104] 15: Returns the protected data data+noise.

[0105] The above process implements an AdaptivePrivacyMechanism class that dynamically adjusts the amount of noise added to the data based on the data sensitivity, the user's privacy preferences, and the remaining privacy budget. The ApplyPrivacyMechanism method applies this adaptive mechanism to the given data.

[0106] Based on the sensitivity of the data and privacy budgets The noise data parameters are obtained by the ratio of , which also includes the following steps:

[0107] Set the total privacy budget of the original data in each initialized model , and initialize the remaining privacy budget .

[0108] Get the data sensitivity based on the noise data each time and the user's privacy preferences to generate a privacy budget Individual privacy budgets allocated in operations .

[0109] The remaining privacy budget is updated by the difference between the total privacy budget and the individual privacy budget allocated each time, and the rejection or implementation judgment of the operation is made based on the updated remaining privacy budget.

[0110] The dynamic privacy budget adjustment algorithm flexibly adjusts the value by considering the sensitivity, complexity, and frequency of each query, thereby providing customized privacy protection levels for different types of data queries. This approach helps find a balance between protecting individual privacy and maintaining data utility. The dynamic privacy budget process is as follows:

[0111] Input: Total privacy budget .

[0112] Output: Boolean value indicating whether the budget is allocated successfully.

[0113] 1: Initialize the remaining privacy budget .

[0114] 2: Budget allocation function AllocateBudget( ):

[0115] 3: if ,but:

[0116] 4: .

[0117] 5: return True.

[0118] 6: else:

[0119] 7: return False.

[0120] 8: Remaining budget check function CheckRemainingBudget():

[0121] 9: return .

[0122] During the dynamic privacy budgeting process, the PrivacyBudgetManager class manages the total privacy budget and the remaining budget. It provides a method called allocate_budget to allocate a budget for each operation. If the remaining budget is sufficient, this method returns True and reduces the remaining budget accordingly to find a balance between data utilization and user privacy protection. The steps are as follows:

[0123] Initialize the total privacy budget: Set the total privacy budget for the entire data processing process .

[0124] Set budget allocation per operation: Allocate a privacy budget per operation based on the importance or expected frequency of the operation .

[0125] Update the remaining privacy budget: After each data operation, subtract the privacy budget of that operation from the total budget.

[0126] Ensure no overspending: Before performing any data operation, check whether there is sufficient privacy budget. If the remaining budget is insufficient for the operation, reject the operation or readjust the budget allocation.

[0127] Optional budget recovery strategies: In some cases, consider strategies to “recover” a portion of the privacy budget, for example, gradually increasing the budget over time or increasing it when certain conditions are met.

[0128] Step S4: Generate a number matching the number of participants using each data point after adding noise. secret sharing shares, and The secret sharing shares of each client are uploaded to the server to obtain the aggregated share, and the secret sharing shares of each client are updated through the aggregated share.

[0129] Specific implementation of data protection combining secret sharing and differential privacy:

[0130] The combination of secret sharing and differential privacy takes advantage of the communication efficiency of secret sharing and the strong privacy protection capabilities of differential privacy. In this combination, data is divided into multiple parts and shared among multiple participants, while each participant can only access a portion of the data allocated to it. The client calls the additive secret sharing algorithm to upload the gradient share to the server. The server obtains the aggregated share, and the client downloads the aggregated share of each server for secret recovery to obtain the updated model. This segmented data adds a certain perturbation to the local model obtained by local training, and the perturbed model is uploaded to the server, so that each round of aggregation process satisfies differential privacy, eliminating the adversary from inferring sensitive information of a piece of data through the difference in output distribution, thereby preventing the aggregate model from being exploited by the adversary and protecting the privacy of global information. The data protection processing process combining secret sharing and differential privacy is as follows:

[0131] Algorithm: Data protection combining secret sharing and differential privacy.

[0132] Input: Raw data , number of participants , privacy budget , data sensitivity .

[0133] Output: Share of secret share .in For the secret sharing shares.

[0134] 1: Calculate Laplace noise parameters .

[0135] 2: For the dataset Each data point in :

[0136] 3: Generate Laplace noise .

[0137] 4: Adding noise to the data .

[0138] 5: Dataset to which noise will be added Apply a secret sharing algorithm.

[0139] 6: For Each noise data point in :

[0140] 7: Generate Secret sharing shares .

[0141] 8: Return the share set of each participant .

[0142] Quantitative analysis method for performance and privacy balance:

[0143] A comprehensive evaluation strategy is adopted to accurately quantify the balance between differential privacy protection and model performance. This paper designs a series of experiments, selects standard datasets and applies different privacy budgets ( values) and noise types, such as Laplace noise and Gaussian noise. These experiments aim to observe how these variables affect model performance, particularly in terms of key metrics such as accuracy and recall. The present invention also conducts an in-depth analysis of the effectiveness of these noise addition strategies and privacy budget settings on personal data privacy protection to ensure the effectiveness of the present invention's approach in protecting user privacy. By comparing model performance and privacy protection levels under different settings, an optimal balance is successfully found. This not only demonstrates the effectiveness of differential privacy technology, but also demonstrates the optimal trade-off that can be achieved between protecting privacy and maintaining high model performance. The experimental results and analysis provide important theoretical and practical guidance for the practical application of differential privacy technology, demonstrating how to effectively utilize differential privacy technology to protect user privacy while maintaining high efficiency in data processing and analysis in the current data security environment.

[0144] In the field of secure data sharing, the core of the adaptive privacy protection strategy is to flexibly adjust the intensity of privacy protection according to the sensitivity of the data and the usage scenario. In the specific implementation, a dynamic noise adjustment strategy is adopted, which means that according to the characteristics of the dataset (for example, the diversity of the dataset and the different levels of user privacy requirements), the system will automatically adjust the amount of noise added. For example, for datasets containing more sensitive information, the system will increase the amount of noise to improve the level of privacy protection, and vice versa, it will reduce the noise to maintain the practicality of the data. This solution uses differential privacy technology to implement this strategy. Differential privacy protects individual information by adding noise to the dataset, avoiding inferring the specific information of any individual by aggregating data. During the implementation process, the system adjusts the noise level according to the preset privacy budget ( The noise distribution (such as Laplace or Gaussian distribution) is determined based on the noise value and data sensitivity. This way, personal information can be effectively protected even when the data is being analyzed.

[0145] This paper considers the core principle of differential privacy: while protecting data privacy, introducing a certain amount of noise to prevent the leakage of personal information. The key to the dynamic noise adjustment strategy lies in how to flexibly adjust the noise level based on the specific data conditions and privacy protection requirements.

[0146] In differential privacy, the addition of noise is usually based on two main types: Laplace noise and Gaussian noise. Laplace noise is suitable for satisfying In the case of differential privacy, the amount of noise is directly related to the data sensitivity and privacy budget. Gaussian noise is used to meet ( ) differential privacy is applicable to more complex data analysis scenarios. The key to dynamic noise adjustment strategies lies in flexibly adjusting the amount and type of noise based on the sensitivity and frequency of data queries and the allocation of the privacy budget. For example, for frequent low-sensitivity queries, a smaller amount of noise can be used to maintain data usability while achieving basic privacy protection. Conversely, for highly sensitive queries, the amount of noise should be increased to provide stronger privacy protection. The concept of privacy budget is crucial in differential privacy. Each data operation consumes a certain amount of privacy budget, and the overall privacy budget is limited. Therefore, how to reasonably allocate the privacy budget across different queries becomes the key to dynamic noise adjustment strategies.

[0147] In this paper, we experimentally validated the effectiveness of the dynamic noise adjustment strategy by testing different datasets, including structured and unstructured data. The experimental results show that by dynamically adjusting the amount of noise, not only can privacy be effectively protected, but the practicality and accuracy of the data can also be maintained to a certain extent. Although the dynamic noise adjustment strategy has demonstrated advantages in many aspects, challenges still exist when processing large-scale datasets. Future research may focus on optimizing the noise generation algorithm, improving the algorithm's computational efficiency, and maximizing the practicality of the data while ensuring privacy.

[0148] In summary, the dynamic noise adjustment strategy provides a flexible and effective privacy protection mechanism for differential privacy. By fine-tuning the amount of noise, it can better meet the privacy protection requirements of different data environments while maintaining data availability and analytical accuracy. The implementation of this strategy is of great significance for promoting the popularization and development of differential privacy technology in practical applications.

[0149] This paper explores the design of customized algorithms for diverse data aggregation in the context of secure data sharing. As can be seen from the table, the design of diverse data aggregation focuses on how to flexibly select and adjust data aggregation algorithms based on data characteristics and processing requirements in secure data sharing, as shown in Table 1.

[0150] Table 1 Data aggregation method

[0151]

[0152] In this step, the secret sharing share set is uploaded to the server to obtain the aggregated share. The aggregation process includes the following steps:

[0153] Use linear aggregation, nonlinear aggregation or aggregation by precision to The secret sharing shares are aggregated and the global model is generated by the aggregated shares.

[0154] Linear aggregation uses the FedAvg algorithm, a common federated learning algorithm primarily used for linear aggregation. It updates the global model by summing and averaging model updates from different clients. Clients first train the model on local data and then send the model parameters to a central server. The central server collects these parameters, sums them, and then averages them to update the global model. The linear aggregation process includes the following steps:

[0155] set up For the The model parameters of each client, For the client The number of data points, then the global model parameters The calculation expression is:

[0156]

[0157] in, is the number of clients, is the total number of data points.

[0158] Nonlinear aggregation: Weighted average. In nonlinear aggregation, client performance or data volume can be taken into account using a weighted average approach. Each client is assigned a different weight based on its performance (such as computing power) and data volume. Clients with higher performance or larger data volumes receive a higher weight during aggregation. The nonlinear aggregation process includes the following steps:

[0159] set up It is The model parameters of each client, is the corresponding weight, then the global model parameter (after weighted average) The calculation expression is:

[0160]

[0161] Among them, the weight Dynamically adjust based on client performance and data volume.

[0162] Aggregation by accuracy allows the central server to adjust weights based on the model performance reported by the clients, prioritizing model updates with higher performance. Clients evaluate the performance of their models locally and send this information to the central server. The central server then adjusts the weights of the clients in global model updates based on these performance metrics. The aggregation by accuracy process includes the following steps:

[0163] set up It is The model parameters of each client, is the weight determined based on performance evaluation, then the global model parameters The update expression is:

[0164]

[0165] Among them, the weight Determined by the model's accuracy on local data.

[0166] The selection and design of these aggregation algorithms need to be customized according to the specific application scenario and data characteristics. By flexibly applying these algorithms, the performance and accuracy of the federated learning model can be effectively improved while taking into account the balance between computing resources and data privacy.

[0167] The client downloads the secret sharing shares of each server to perform secret recovery and obtain the updated model.

[0168] The design of the data aggregation strategy employs a dynamic adjustment mechanism to ensure high efficiency across various data characteristics and usage scenarios. This strategy takes into account the diversity and nonlinear characteristics of data, automatically selecting the most appropriate aggregation method, such as linear or nonlinear aggregation, based on the characteristics and processing requirements of different datasets. The strategy also includes an adaptive mechanism that adjusts the aggregation algorithm parameters based on real-time performance feedback to achieve an optimal balance. To ensure the efficiency and accuracy of the data aggregation process, this paper proposes a performance-based strategy. This strategy utilizes multiple evaluation metrics, such as accuracy, processing speed, and memory usage, to monitor and evaluate the performance of the data aggregation process in real time. This approach enables timely identification and resolution of potential performance bottlenecks, thereby optimizing the entire data aggregation process. Furthermore, advanced mathematical models and algorithms are introduced to improve the efficiency of data aggregation. For example, efficient linear algebra operations and optimized data structures are utilized to reduce computational overhead, while machine learning algorithms are employed to predict and adjust the aggregation strategy to suit different data distributions and user needs. To demonstrate the effectiveness of this performance-based data aggregation strategy, a series of experiments were designed. These experiments used different datasets, such as image data, text data, and structured data, to evaluate the performance of the strategy in various scenarios. Compared with traditional data aggregation methods, this strategy has significant improvements in many aspects, including higher data processing speed, better accuracy and lower resource consumption.

[0169] Performance-based data aggregation is an effective approach for improving the overall performance of federated learning systems while ensuring secure data sharing. Through continuous optimization and iteration, this strategy is expected to play an even greater role in future secure data sharing.

[0170] Based on the above method, the present invention provides a privacy protection system for federated learning, including: a data acquisition module, a noise addition module and a secret sharing module.

[0171] The data acquisition module is used to obtain multiple original data to be protected and divide the multiple original data into multiple groups of data; each group of data after division contains the number of participants. , privacy budget and data sensitivity ; The noise adding module is used to input the multiple groups of segmented data into multiple initialized models respectively, based on the data sensitivity and privacy budgets The noise data parameters are obtained by the ratio of , and noise is added to each data point in the original data according to the noise data parameters; multiple initialized models are installed on the client of each participant respectively; the secret sharing module is used to generate a secret data point that matches the number of participants using each data point after adding noise secret sharing shares, and The secret sharing shares of each client are uploaded to the server to obtain the aggregated share, and the secret sharing shares of each client are updated through the aggregated share.

[0172] The present invention also provides a computer device, including a memory and a processor, wherein a program is stored in the memory. When the program is executed by the processor, the processor performs the steps of a privacy protection method for federated learning.

[0173] According to the disclosed embodiments, a computing device may communicate with one or more external devices (e.g., keyboards, pointing devices, Bluetooth communications, etc.), or any device (e.g., routers, modems, etc.) that enables a computing device to communicate with one or more other computing devices.

[0174] The present invention also provides a storage medium on which a computer program is stored. When the computer program is executed by a processor, the steps of a privacy protection method for federated learning are implemented.

[0175] According to the disclosed embodiments, the storage medium may be a non-volatile computer-readable storage medium, such as, but not limited to, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination thereof. In the present invention, the storage medium may be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, apparatus, or device.

[0176] The above content is a further detailed description of the present invention in combination with specific preferred embodiments. For those skilled in the art to which the present invention belongs, several simple deductions or replacements can be made without departing from the concept of the present invention, which should be regarded as falling within the scope of protection of the present invention.

Claims

1. A privacy protection method for federated learning, characterized in that: include: Acquire a plurality of original data to be protected, and divide the plurality of original data into a plurality of groups of data; Each group of data after segmentation contains the number of participants n, privacy budget ε and data sensitivity Δf; The multiple groups of segmented data are input into multiple initialized models respectively, a noise data parameter is obtained based on the ratio of the data sensitivity Δf to the privacy budget ε, and noise is added to each data point in the original data according to the noise data parameter; the multiple initialized models are respectively installed on the client of each participant; Generate n secret sharing shares that match the number of participants using each data point after adding noise, upload the n secret sharing shares to the server to obtain the aggregated share, and update the secret sharing shares of each client through the aggregated share; Adding noise to each data point in the original data according to the noise data parameter comprises the following steps: Selecting a noise type based on the noise data parameters, and obtaining a noise magnitude according to the selected noise type and the data sensitivity Δf, and adding noise to each data point by using the noise magnitude; Selecting a noise type based on the noise data parameters and obtaining a noise magnitude according to the selected noise type and data sensitivity Δf comprises the following steps: Generate a privacy budget ε based on the data sensitivity Δf of the noise data and the user’s privacy preference; Set the initial privacy budget ε of the original data in each initialized model initial , the privacy budget ε is divided into the initial privacy budget ε initial Compare and obtain the initial residual privacy budget ε remain ; Noise is generated by a Laplace distributed random number with a center of 0 and a scale of scale. If the noise is greater than the initial remaining privacy budget ε remain When the remaining privacy budget ε remain As the noise level at this time; otherwise, the remaining privacy budget ε at this time remain The difference between the privacy budget parameter and the privacy budget parameter is taken as the remaining privacy budget at this time.

2. A privacy protection method for federated learning as claimed in claim 1, characterized in that: The noise data parameter is obtained based on the ratio of the data sensitivity Δf to the privacy budget ε. The specific expression is: λ=Δf / ε Among them, λ is the noise data parameter.

3. A privacy protection method for federated learning as claimed in claim 1, characterized in that: Obtaining the noise data parameter based on the ratio of the data sensitivity Δf to the privacy budget ε also includes the following steps: Set the total privacy budget ε of the original data in each initialized model total , and initialize the remaining privacy budget ε remain ; Get the single privacy budget ε allocated in each privacy budget ε generation operation based on the data sensitivity Δf of the noisy data and the user's privacy preference top ; The remaining privacy budget is updated by the difference between the total privacy budget and the individual privacy budget allocated each time, and the rejection judgment or implementation judgment of the operation is made by using the updated remaining privacy budget.

4. A privacy protection method for federated learning as claimed in claim 1, characterized in that: Uploading n secret sharing shares to the server to obtain the aggregated share includes the following steps: Aggregating the n secret sharing shares by linear aggregation, nonlinear aggregation or precision aggregation, and generating a global model by the aggregated shares; The linear polymerization process includes the following steps: Assume w i is the model parameter of the i-th client, n i is the number of data points of client i, then the calculation expression of the global model parameter W is: Where K is the number of clients and N is the total number of data points; The nonlinear aggregation process includes the following steps: Assume w i is the model parameter of the i-th client, α i is the corresponding weight, then the global model parameter W is calculated as: Among them, the weight α i Dynamically adjust according to the client's performance and data volume; The precision aggregation process includes the following steps: Assume w i is the model parameter of the i-th client, β i is the weight determined based on performance evaluation, then the update expression of the global model parameter W is: Among them, the weight β i Determined by the accuracy of the model on local data.

5. A privacy protection method for federated learning as claimed in claim 1, characterized in that: The method of updating the secret sharing shares of each client by aggregating the shares further includes the following steps: The client downloads the secret sharing shares of each server to perform secret recovery and obtain the updated model.

6. A privacy protection system for federated learning, characterized in that: include: A data acquisition module, used for acquiring a plurality of original data to be protected, and dividing the plurality of original data into a plurality of groups of data; Each group of data after segmentation contains the number of participants n, privacy budget ε and data sensitivity Δf; A noise adding module, used to input the divided multiple groups of data into multiple initialized models respectively, obtain noise data parameters based on the ratio of the data sensitivity Δf to the privacy budget ε, and add noise to each data point in the original data according to the noise data parameters; the multiple initialized models are respectively installed on the client of each participant; The secret sharing module is used to generate n secret sharing shares matching the number of participants using each data point after adding noise, and upload the n secret sharing shares to the server to obtain the aggregated share, and update the secret sharing shares of each client through the aggregated share; Adding noise to each data point in the original data according to the noise data parameter comprises the following steps: Selecting a noise type based on the noise data parameters, and obtaining a noise magnitude according to the selected noise type and the data sensitivity Δf, and adding noise to each data point by using the noise magnitude; Selecting a noise type based on the noise data parameters and obtaining a noise magnitude according to the selected noise type and data sensitivity Δf comprises the following steps: Generate a privacy budget ε based on the data sensitivity Δf of the noise data and the user’s privacy preference; Set the initial privacy budget ε of the original data in each initialized model initial , the privacy budget ε is divided into the initial privacy budget ε initial Compare and obtain the initial residual privacy budget ε remain ; Noise is generated by a Laplace distributed random number with a center of 0 and a scale of scale. If the noise is greater than the initial remaining privacy budget ε remain When the remaining privacy budget ε remain As the noise level at this time; otherwise, the remaining privacy budget ε at this time remaian The difference between the privacy budget parameter and the privacy budget parameter is taken as the remaining privacy budget at this time.

7. A computer device, characterized in that: It includes a memory and a processor, wherein a program is stored in the memory, and when the program is executed by the processor, the processor executes the steps of a privacy protection method for federated learning as described in any one of claims 1 to 5.

8. A storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of a privacy protection method for federated learning described in any one of claims 1 to 5 are implemented.