Personalized Differential Privacy Federated Learning Method and System

Abstract

This application discloses a personalized differential privacy federated learning method and system, belonging to the field of big data security. The personalized differential privacy federated learning method includes: distributing global model parameters and privacy types to pre-selected clients; receiving local model parameters and privacy protection levels sent by the clients, wherein the local model parameters are model parameters obtained by the clients after perturbation processing of local data, and the privacy protection level is a level among the privacy types; aggregating the local model parameters and the privacy protection level to obtain aggregated model parameters and aggregated privacy protection levels; processing the aggregated model parameters according to the aggregated privacy protection levels to obtain target global model parameters; and obtaining the target global model based on the target global model parameters.

Description

Technical Field

[0001] This application belongs to the field of big data security, specifically involving a personalized differential privacy federated learning method and system. Background Technology

[0002] Federated learning is a data sharing solution for big data scenarios, designed to address the widespread data silo problem in large enterprises—the inability to share and mine data due to reasons such as data security and data value. By deploying a federated learning architecture in a distributed cluster, data sharing within the group can be effectively promoted.

[0003] In related technologies, hybrid differential privacy algorithms for federated learning data sharing divide users into two groups: ordinary users, who are assumed to have high privacy needs and are therefore protected using a localized differential privacy algorithm; and trusted users, who have confirmed their unconditional trust in the server. The algorithm considers these users to trust the server and thus reduces or even eliminates data protection for them, using the raw data directly. This division is relatively simplistic and can easily lead to a significant waste of privacy budget and reduced data efficiency. Summary of the Invention

[0004] This application provides a personalized differential privacy federated learning method and system, which can solve the problems of data waste and low data efficiency caused by the unified privacy protection of related technologies.

[0005] In a first aspect, embodiments of this application provide a personalized differential privacy federated learning method applied to a server, the method comprising:

[0006] Distribute global model parameters and privacy types to pre-selected clients;

[0007] The system receives local model parameters and a privacy protection level sent by the client. The local model parameters are model parameters obtained by the client after perturbing local data, and the privacy protection level is one of the privacy types.

[0008] The local model parameters and the privacy protection level are aggregated to obtain aggregated model parameters and aggregated privacy protection level;

[0009] Based on the aggregated privacy protection level, the aggregated model parameters are processed to obtain the target global model parameters;

[0010] The target global model is obtained based on the target global model parameters.

[0011] Secondly, embodiments of this application provide a personalized differential privacy federated learning system applied to a client, the method comprising:

[0012] Receive global model parameters and privacy type sent by the server;

[0013] Based on the global model parameters and the privacy type, the local model parameters and privacy protection level are obtained by perturbing the global model parameters using local data.

[0014] The local model parameters and the privacy protection level are sent to the client.

[0015] Thirdly, embodiments of this application provide a personalized differential privacy federated learning system applied to a server, the system comprising:

[0016] The first sending module is used to send global model parameters and privacy type to pre-selected clients;

[0017] The first receiving module is used to receive local model parameters and privacy protection level sent by the client. The local model parameters are model parameters obtained by the client after perturbing local data, and the privacy protection level is a level among the privacy types.

[0018] The aggregation module is used to aggregate the local model parameters and the privacy protection level to obtain aggregated model parameters and aggregated privacy protection level;

[0019] The processing module is used to process the aggregated model parameters according to the aggregated privacy protection level to obtain the target global model parameters;

[0020] The determination module is used to obtain the target global model based on the target global model parameters.

[0021] Fourthly, embodiments of this application provide a personalized differential privacy federated learning system applied to a client, the system comprising:

[0022] The second receiving module is used to receive global model parameters and privacy types sent by the server;

[0023] The perturbation module is used to perturb the global model parameters using local data based on the global model parameters and the privacy type, so as to obtain the local model parameters and the privacy protection level.

[0024] The second sending module is used to send the local model parameters and the privacy protection level to the client.

[0025] Fifthly, embodiments of this application provide a readable storage medium on which a program or instructions are stored, which, when executed by a processor, implement the steps of the method described in the first aspect, or implement the steps of the method described in the second aspect.

[0026] In this embodiment, the global model parameters and privacy type are first sent to a pre-selected client. Then, the local model parameters and privacy protection level sent by the client are received. The local model parameters are obtained by the client after perturbing local data, and the privacy protection level is one of the privacy types. Next, the local model parameters and privacy protection level are aggregated to obtain aggregated model parameters and aggregated privacy protection level. Based on the aggregated privacy protection level, the aggregated model parameters are processed to obtain the target global model parameters. Finally, the target global model is obtained based on the target global model parameters. This embodiment obtains the local model parameters after the user has perturbed their local data using a privacy protection algorithm, ensuring that the perturbed data meets the user's personalized privacy needs, thereby reducing the waste of data availability caused by a uniform protection level. Attached Figure Description

[0027] Figure 1 This is a flowchart of a personalized differential privacy federated learning method provided in one embodiment of this application;

[0028] Figure 2 This is a diagram of a personalized differential privacy federated learning architecture under data sharing provided in one embodiment of this application;

[0029] Figure 3 This is a flowchart of a personalized differential privacy federated learning method provided in one embodiment of this application;

[0030] Figure 4 This is a structural block diagram of a personalized differential privacy federated learning system provided in one embodiment of this application;

[0031] Figure 5 This is a structural block diagram of a personalized differential privacy federated learning system provided in one embodiment of this application. Detailed Implementation

[0032] The technical solutions of the embodiments of this application will be clearly described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this application. All other embodiments obtained by those skilled in the art based on the embodiments of this application are within the scope of protection of this application.

[0033] The terms "first," "second," etc., used in the specification and claims of this application are used to distinguish similar objects and not to describe a specific order or sequence. It should be understood that such use of data can be interchanged where appropriate so that embodiments of this application can be implemented in orders other than those illustrated or described herein. Furthermore, in the specification and claims, "and / or" indicates at least one of the connected objects, and the character " / " generally indicates that the preceding and following objects are in an "or" relationship.

[0034] The following description, in conjunction with the accompanying drawings, details a personalized differential privacy federated learning method and system provided in this application through specific embodiments and application scenarios.

[0035] like Figure 1 As shown, this application provides a personalized differential privacy federated learning method, such as... Figure 1 As shown, this personalized differential privacy federated learning method is applied to the server. This personalized differential privacy federated learning method may include the content shown in S101 to S105.

[0036] In S101, global model parameters and privacy types are sent to pre-selected clients.

[0037] The privacy type can include high privacy level, low privacy level, intermediate privacy level, etc., and the specific classification can be made according to the actual situation. This embodiment does not limit it.

[0038] It's worth noting that when federated learning is running for the first time, the server will first initialize a basic, general model as global model parameters; as federated learning progresses, the server will aggregate the model parameters from the previous round and use them as global model parameters. The server can be a central aggregation server.

[0039] In S102, the local model parameters and privacy protection level sent by the client are received.

[0040] Among them, the local model parameters are the model parameters obtained by the client after perturbing the local data, and the privacy protection level is one of the levels in the privacy type.

[0041] In S103, the local model parameters and privacy protection level are aggregated to obtain aggregated model parameters and aggregated privacy protection level.

[0042] In S104, the aggregated model parameters are processed according to the aggregated privacy protection level to obtain the target global model parameters.

[0043] In S105, the target global model is obtained based on the target global model parameters.

[0044] In this embodiment, the global model parameters and privacy type are first sent to a pre-selected client. Then, the local model parameters and privacy protection level sent by the client are received. The local model parameters are obtained by the client after perturbing local data, and the privacy protection level is one of the privacy types. Next, the local model parameters and privacy protection level are aggregated to obtain aggregated model parameters and aggregated privacy protection level. Based on the aggregated privacy protection level, the aggregated model parameters are processed to obtain the target global model parameters. Finally, the target global model is obtained based on the target global model parameters. This embodiment obtains the local model parameters after the user has perturbed their local data using a privacy protection algorithm, ensuring that the perturbed data meets the user's personalized privacy needs, thereby reducing the waste of data availability caused by a uniform protection level.

[0045] In one possible implementation of this application, before distributing the global model parameters and privacy type to pre-selected clients, the personalized differential privacy federated learning method may further include: randomly selecting a preset number of clients, each client possessing the training data required for the federated learning task, and training to obtain local model parameters; in the case of the first run of federated learning, generating an initial global model, the parameters of which include the global model parameters; and in the case of the run of federated learning, updating the global model parameters with the target global model parameters from the previous round.

[0046] In other words, global model parameters and privacy type can be distributed to a randomly selected preset number of clients so that the clients can use their local data to train global model parameters and obtain local model parameters. Global model parameters can be continuously updated according to federated learning to obtain global model parameters that meet the requirements, i.e., target global model parameters.

[0047] In one possible implementation of this application, aggregating local model parameters and privacy protection levels to obtain aggregated model parameters and aggregated privacy protection levels may include: aggregating parameters according to the following formula to obtain aggregated model parameters:

[0048] w = p1w1 + p2w2 + ... + p i w i +...+p k w k

[0049] Where w represents the aggregation model parameters; w i p represents the global model parameters for the i-th client. i These are the local model parameters for the i-th client.

[0050] In other words, the server can aggregate model parameters and privacy levels uploaded by user clients. After aggregation, a new set of model parameters is obtained.

[0051] In one possible implementation of this application, the aggregated model parameters are processed according to the aggregated privacy protection level to obtain target global model parameters. This may include: adding noise equal to the global differential privacy budget threshold to the aggregated model parameters if the aggregated privacy protection level is greater than the global differential privacy budget threshold; using the aggregated model parameters as target global model parameters if the aggregated privacy protection level is less than the global differential privacy budget threshold; and sending the target global model parameters as global model parameters for the next round to the next selected client.

[0052] In other words, if the aggregated privacy protection level is greater than the global differential privacy budget threshold, it means that the user has high requirements for the privacy level. In this case, noise of the budget threshold needs to be added to the aggregated model parameters to protect user data. If the aggregated privacy protection level is less than the global differential privacy budget threshold, it means that the user has low requirements for the privacy level. In this case, the user feedback data can be used directly.

[0053] In one possible implementation of this application, when the aggregated privacy protection level is greater than the global differential privacy budget threshold, adding noise equal to the global differential privacy budget threshold to the aggregated model parameters may include: the noise of the global differential privacy budget threshold is as shown in the following formula:

[0054]

[0055]

[0056]

[0057] Where, σ D Noise is the threshold value for the global differential privacy budget; σ U The noise corresponding to the smallest local dataset; σ A For the client, use noise from a dataset of the same size; m is the minimum local dataset; C is the pruning threshold for the local model parameters; c is the privacy budget; δ is the probability value; ε k ε represents the client's privacy protection level; ε represents the aggregated privacy protection level; max is the global differential privacy budget threshold; N is the number of clients.

[0058] like Figure 2 The diagram shown illustrates the structure of a personalized differential privacy federated learning system under data sharing. Figure 2As shown, the server sends global model parameters and privacy type to each client. After adding noise, the client forms local model parameters and sends them to the server. The server chooses whether to continue adding noise based on the client's privacy protection level. Clients with high privacy protection levels need to continue adding noise, while clients with low privacy protection levels can choose not to add noise.

[0059] This embodiment uses a privacy protection algorithm to perturb and protect the parameters trained locally by the user, meeting the user's personalized privacy needs and minimizing the waste of data availability caused by a uniform protection level. After obtaining the privacy protection parameters uploaded by the user, the server determines the user's overall privacy protection level. If the requirements are not met, a second round of fine-grained protection is performed, quantifying the data security level of the entire system.

[0060] like Figure 3 As shown, this application provides a personalized differential privacy federated learning method, such as... Figure 1 As shown, this personalized differential privacy federated learning method is applied to the client. This personalized differential privacy federated learning method may include the content shown in S301 to S303.

[0061] In S301, the global model parameters and privacy type are sent by the receiving server.

[0062] In S302, based on the global model parameters and privacy type, the local data is used to perturb the global model parameters to obtain the local model parameters and privacy protection level.

[0063] In S303, local model parameters and privacy protection levels are sent to the client.

[0064] In this embodiment, the client first receives the global model parameters and privacy type from the server. Then, based on the global model parameters and privacy type, it uses local data to perturb the global model parameters to obtain local model parameters and a privacy protection level. Finally, the local model parameters and privacy protection level are sent to the client. This embodiment uses a privacy protection algorithm to perturb the user's local data to obtain local model parameters, ensuring that the perturbed data meets the user's personalized privacy needs, thereby reducing the waste of data availability caused by a uniform protection level.

[0065] In other words, the personalized differential privacy federated learning algorithm under data sharing initializes the personalized differential privacy module based on the privacy preferences in the local settings, adding Gaussian noise that meets the user's privacy requirements. Then, the perturbed model parameters and the user's privacy preferences are uploaded to the server.

[0066] In one possible implementation of this application, the global model parameters are perturbed using local data based on global model parameters and privacy type to obtain local model parameters and privacy protection level. This may include: determining a privacy protection level based on privacy preferences, wherein the privacy protection level is a level of the privacy type; and perturbing the global model parameters using the privacy protection level and local data to obtain local model parameters.

[0067] In other words, the privacy level can be selected based on the user's client privacy preferences, and the local model parameters that need to be uploaded can be perturbed according to the privacy level.

[0068] In one possible implementation of this application, perturbing the global model parameters using privacy protection levels and local data to obtain local model parameters may include: perturbing according to the following formula:

[0069]

[0070] in, For local model parameters; w i For the i-th client, w represents the global model parameter; D represents the global model parameter. i For the local data of the i-th client; D i,j D i The j-th sample in M; i This is a randomized algorithm; n is the number of clients; Y is a constant; ||·|| p It is a p-norm.

[0071] The sensitivity can be expressed as:

[0072]

[0073] Among them, D′ i D i Neighboring datasets; D′ i,j D′ i The j-th sample in the dataset.

[0074] The sensitivity of the upload channel can be expressed as:

[0075] To ensure the upload channel satisfies (ε,δ)-DP (Differential Privacy), Gaussian noise is added. Assume the smallest local dataset is , where is the pruning threshold for the weights. The specific noise addition is as follows:

[0076]

[0077] The server aggregates model parameters based on the model parameters and privacy level uploaded by the user client. After aggregation, a new set of model parameters is obtained. The server's aggregation process is as follows:

[0078] w = p1w1 + p2w2 + ... + p i w i +...+p k w k

[0079] Sensitivity is:

[0080] The server aggregates the privacy protection scale of participating users to determine whether further aggregation at the central server is needed. If the privacy scale of participating users reaches a set threshold, no further noise is added to ensure the quality of model training. If the privacy scale of participating users does not reach the set threshold, a new round of noise needs to be added at the central server. Therefore, the sensitivity is:

[0081]

[0082] To quantify the results, we assume that user clients use the same size dataset, i.e. Therefore, the amount of noise that should be added globally is as follows:

[0083]

[0084] When the global differential privacy budget threshold is not reached, the noise that the central aggregation server should add is:

[0085]

[0086] The noise mechanism used for the perturbation is Gaussian noise. Its formal definition is as follows:

[0087] Suppose there is a function f: D→R D The sensitivity is Δf. When N ~ Lap(Δf / ε), the random algorithm M = f(D) + N provides privacy protection for (ε,δ)-DP. The parameters... The probability density function formula for the Gaussian distribution is as follows:

[0088]

[0089] Where σ represents the standard deviation of the Gaussian distribution and μ represents the expected value of the Gaussian distribution, and sensitivity f is the query function, and ε is the privacy budget. The Gaussian mechanism uses L2 sensitivity, which is well-suited for the training process of federated learning.

[0090] For two datasets D1 and D2, if D1 and D2 have exactly one different data point, that is... D1 and D2 are referred to as adjacent datasets. This represents the symmetric difference operation.

[0091] The formulaic definition of differential privacy used is as follows:

[0092] A random algorithm M: X→R, where X is the domain and R is the range. If the random algorithm M satisfies the following equation on any adjacent datasets D and D', then the random algorithm M satisfies (ε,δ)-DP. In particular, when δ=0, the algorithm M satisfies ε-differential privacy.

[0093] Pr(M(D)∈S)≤e ε ×Pr(M(D′)∈S)+δ

[0094] The parameter ε is the privacy budget, and its value determines the degree of privacy protection. The smaller ε is, the higher the privacy protection; the larger ε is, the higher the data availability.

[0095] To further prove that the proposed personalized differential privacy federated learning algorithm under data sharing satisfies (ε) max δ) - Global differential privacy, meaning the algorithm has specific quantitative metrics. We provide the corresponding proof, as follows:

[0096]

[0097] It can be obtained set up Proof is required:

[0098]

[0099] Furthermore, the boundary conditions for Pr(x≥η) are as follows:

[0100]

[0101] That is, to prove:

[0102]

[0103] When the privacy settings of participating users reach a predetermined threshold, Substitution achievable That concludes the proof.

[0104] When the privacy protection level of participating users does not reach the predetermined threshold, Substitution achievable That concludes the proof.

[0105] like Figure 4 As shown in the figure, this application embodiment also provides a personalized differential privacy federated learning system, which is applied to a server and may include: a first sending module 401, a first receiving module 402, an aggregation module 403, a processing module 404, and a determining module 405.

[0106] The system comprises the following modules: a first sending module 401, used to send global model parameters and privacy type to a pre-selected client; a first receiving module 402, used to receive local model parameters and privacy protection level sent by the client, wherein the local model parameters are model parameters obtained by the client after perturbing local data, and the privacy protection level is a level among the privacy types; an aggregation module 403, used to aggregate the local model parameters and privacy protection level to obtain aggregated model parameters and aggregated privacy protection level; a processing module 404, used to process the aggregated model parameters according to the aggregated privacy protection level to obtain target global model parameters; and a determining module 405, used to obtain the target global model according to the target global model parameters.

[0107] In this embodiment, the first sending module 401 first sends global model parameters and privacy type to a pre-selected client; then, the first receiving module 402 receives the local model parameters and privacy protection level sent by the client. The local model parameters are model parameters obtained by the client after perturbing local data, and the privacy protection level is one of the privacy types; the aggregation module 403 then aggregates the local model parameters and privacy protection level to obtain aggregated model parameters and aggregated privacy protection level; the processing module 404 processes the aggregated model parameters according to the aggregated privacy protection level to obtain the target global model parameters; finally, the determining module 405 obtains the target global model based on the target global model parameters. This embodiment obtains the local model parameters after the user has perturbed the user's local data using a privacy protection algorithm, so that the perturbed data meets the user's personalized privacy needs, thereby reducing the waste of data availability caused by a uniform protection level.

[0108] In one possible implementation of this application, the personalized differential privacy federated learning system may further include: a selection module, a generation module, and an update module.

[0109] The selection module is used to randomly select a preset number of clients, each of which has the training data required for the federated learning task and has trained to obtain local model parameters. The generation module is used to generate an initial global model when federated learning is run for the first time. The parameters of the initial global model include the global model parameters. The update module is used to update the global model parameters with the target global model parameters from the previous round when federated learning is run.

[0110] In one possible implementation of this application, the aggregation module 403 can be used to: aggregate parameters according to the following formula to obtain aggregated model parameters:

[0111] w = p1w1 + p2w2 + ... + p i w i +...+p k w k

[0112] Where w represents the aggregation model parameters; w i p represents the global model parameters for the i-th client. i These are the local model parameters for the i-th client.

[0113] In one possible implementation of this application, the processing module 404 can be used to: add noise equal to the global differential privacy budget threshold to the aggregated model parameters when the aggregated privacy protection level is greater than the global differential privacy budget threshold; use the aggregated model parameters as the target global model parameters when the aggregated privacy protection level is less than the global differential privacy budget threshold; and send the target global model parameters as the global model parameters for the next round to the next selected client.

[0114] In one possible implementation of this application, the processing module 404 can be used to: determine the noise of the global differential privacy budget threshold as shown in the following formula:

[0115]

[0116]

[0117]

[0118] Where, σ D Noise is the threshold value for the global differential privacy budget; σ U The noise corresponding to the smallest local dataset; σ A For the client, use noise from a dataset of the same size; m is the minimum local dataset; C is the pruning threshold for the local model parameters; c is the privacy budget; δ is the probability value; ε k ε represents the client's privacy protection level; ε represents the aggregated privacy protection level; max is the global differential privacy budget threshold; N is the number of clients.

[0119] The personalized differential privacy federated learning system in this application embodiment can be a device, or a component, integrated circuit, or chip in a terminal. The device can be a mobile electronic device or a non-mobile electronic device. For example, mobile electronic devices can be mobile phones, tablets, laptops, PDAs, in-vehicle electronic devices, wearable devices, ultra-mobile personal computers (UMPCs), netbooks, or personal digital assistants (PDAs), etc., while non-mobile electronic devices can be servers, network-attached storage (NAS), personal computers (PCs), televisions (TVs), ATMs, or self-service machines, etc. This application embodiment does not impose specific limitations.

[0120] The personalized differential privacy federated learning system in this application embodiment can be a device with an operating system. This operating system can be Android, iOS, or other possible operating systems; this application embodiment does not specifically limit it.

[0121] The personalized differential privacy federated learning system provided in this application embodiment can achieve... Figure 1 The various processes implemented in the method embodiments achieve the same technical effect, and will not be described again here to avoid repetition.

[0122] like Figure 5 As shown in the figure, this application embodiment also provides a personalized differential privacy federated learning system, which is applied to a client and may include: a second receiving module 501, a perturbation module 502 and a second sending module 503.

[0123] The second receiving module 501 is used to receive global model parameters and privacy type sent by the server; the perturbation module 502 is used to perturb the global model parameters using local data according to the global model parameters and privacy type to obtain local model parameters and privacy protection level; and the second sending module 503 is used to send the local model parameters and privacy protection level to the client.

[0124] In this embodiment, the second receiving module 501 first receives the global model parameters and privacy type sent by the server. Then, the perturbation module 502 perturbs the global model parameters using local data based on the global model parameters and privacy type to obtain local model parameters and a privacy protection level. Finally, the second sending module 503 sends the local model parameters and privacy protection level to the client. This embodiment uses a privacy protection algorithm to perturb the user's local data to obtain local model parameters, ensuring that the perturbed data meets the user's personalized privacy needs and reducing the waste of data availability caused by a uniform protection level.

[0125] In one possible implementation of this application, the perturbation module 502 can be used to: determine a privacy protection level based on privacy preferences, wherein the privacy protection level is a level of privacy type; and perturb the global model parameters using the privacy protection level and local data to obtain local model parameters.

[0126] In one possible embodiment of this application, the perturbation module 502 can be used to perform perturbation processing as shown in the following formula:

[0127]

[0128] in, For local model parameters; w i For the i-th client, w represents the global model parameter; D represents the global model parameter. i For the local data of the i-th client; D i,j D i The j-th sample in M; i This is a randomized algorithm; n is the number of clients; Y is a constant; ||·|| p It is a p-norm.

[0129] The personalized differential privacy federated learning system in this application embodiment can be a device, or a component, integrated circuit, or chip in a terminal. The device can be a mobile electronic device or a non-mobile electronic device. For example, mobile electronic devices can be mobile phones, tablets, laptops, PDAs, in-vehicle electronic devices, wearable devices, ultra-mobile personal computers (UMPCs), netbooks, or personal digital assistants (PDAs), etc., while non-mobile electronic devices can be servers, network-attached storage (NAS), personal computers (PCs), televisions (TVs), ATMs, or self-service machines, etc. This application embodiment does not impose specific limitations.

[0130] The personalized differential privacy federated learning system in this application embodiment can be a device with an operating system. This operating system can be Android, iOS, or other possible operating systems; this application embodiment does not specifically limit it.

[0131] The personalized differential privacy federated learning system provided in this application embodiment can achieve... Figure 3 The various processes implemented in the method embodiments achieve the same technical effect, and will not be described again here to avoid repetition.

[0132] This application also provides a readable storage medium on which a program or instructions are stored. When executed by a processor, the program or instructions implement the various processes of the personalized differential privacy federated learning method embodiments provided in any of the above embodiments. Furthermore, it achieves the same technical effects, and to avoid repetition, it will not be described again here.

[0133] The readable storage medium includes computer-readable storage media, such as computer read-only memory (ROM), random access memory (RAM), magnetic disk, or optical disk.

[0134] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element. Furthermore, it should be noted that the scope of the methods and apparatuses in the embodiments of this application is not limited to performing functions in the order shown or discussed, but may also include performing functions substantially simultaneously or in the reverse order, depending on the functions involved. For example, the described methods may be performed in a different order than described, and various steps may be added, omitted, or combined. Additionally, features described with reference to certain examples may be combined in other examples.

[0135] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods of the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, can be embodied in the form of a computer software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) and includes several instructions to cause a terminal (which may be a mobile phone, computer, server, or network device, etc.) to execute the methods described in the various embodiments of this application.

[0136] The embodiments of this application have been described above with reference to the accompanying drawings. However, this application is not limited to the specific embodiments described above. The specific embodiments described above are merely illustrative and not restrictive. Those skilled in the art can make many other forms under the guidance of this application without departing from the spirit and scope of the claims, and all of these forms are within the protection scope of this application.

Claims

1. A personalized differential privacy federated learning method, characterized in that, Applied to a server, the method includes: Distribute global model parameters and privacy types to pre-selected clients; The system receives local model parameters and a privacy protection level sent by the client. The local model parameters are model parameters obtained by the client after perturbing local data, and the privacy protection level is one of the privacy types. The local model parameters and the privacy protection level are aggregated to obtain aggregated model parameters and aggregated privacy protection level; Based on the aggregated privacy protection level, the aggregated model parameters are processed to obtain the target global model parameters; Based on the target global model parameters, the target global model is obtained; The step of aggregating the local model parameters and the privacy protection level to obtain aggregated model parameters and aggregated privacy protection level includes: The parameters are aggregated according to the following formula to obtain the aggregated model parameters: in, These are the parameters for the aggregation model; These are the global model parameters for the i-th client; These are the local model parameters for the i-th client; The step of processing the aggregated model parameters according to the aggregated privacy protection level to obtain the target global model parameters includes: If the aggregated privacy protection level is greater than the global differential privacy budget threshold, noise of the global differential privacy budget threshold is added to the aggregated model parameters; If the aggregated privacy protection level is less than the global differential privacy budget threshold, the aggregated model parameters are used as the target global model parameters. The target global model parameters are used as the global model parameters for the next round and sent to the clients selected in the next round.

2. The method according to claim 1, characterized in that, Before sending the global model parameters and privacy type to the pre-selected clients, the method further includes: A preset number of clients are randomly selected. These clients have the training data required for the federated learning task and are used to train local model parameters. In the case of the first run of the federated learning, an initial global model is generated, the parameters of which include global model parameters; In the case of federated learning, the global model parameters are updated using the target global model parameters from the previous round.

3. The method according to claim 1, characterized in that, When the aggregated privacy protection level is greater than the global differential privacy budget threshold, adding noise of the global differential privacy budget threshold to the aggregated model parameters includes: The noise of the global differential privacy budget threshold is given by the following formula: in, Noise representing the global differential privacy budget threshold; The noise corresponding to the smallest local dataset; The client uses noise from a dataset of the same size; m is the minimum local dataset; C is the pruning threshold for the local model parameters; c is the privacy budget. This is a probability value; The privacy protection level for the client; To aggregate privacy protection levels; is the global differential privacy budget threshold; N is the number of clients.

4. A personalized differential privacy federated learning method, characterized in that, Applied to a client, the method includes: Receive global model parameters and privacy type sent by the server; Based on the global model parameters and the privacy type, the local model parameters and privacy protection level are obtained by perturbing the global model parameters using local data. Send the local model parameters and the privacy protection level to the client; The step of perturbing the global model parameters using local data based on the global model parameters and the privacy type to obtain local model parameters and a privacy protection level includes: Based on privacy preferences, a privacy protection level is determined, wherein the privacy protection level is a level of the privacy type; The global model parameters are perturbed using the privacy protection level and local data to obtain local model parameters.

5. The method according to claim 4, characterized in that, The step of perturbing the global model parameters using the privacy protection level and local data to obtain local model parameters includes: Perform the perturbation as shown in the following formula: in, For local model parameters; These are the global model parameters for the i-th client; These are global model parameters; This represents the local data of the i-th client; for The j-th sample in; This is a random algorithm; n is the number of clients; Y is a constant. It is a p-norm.

6. A personalized differential privacy federated learning system, characterized in that, The system, applied to a server, includes: The first sending module is used to send global model parameters and privacy type to pre-selected clients; The first receiving module is used to receive local model parameters and privacy protection level sent by the client. The local model parameters are model parameters obtained by the client after perturbing local data, and the privacy protection level is a level among the privacy types. The aggregation module is used to aggregate the local model parameters and the privacy protection level to obtain aggregated model parameters and aggregated privacy protection level; The processing module is used to process the aggregated model parameters according to the aggregated privacy protection level to obtain the target global model parameters; The determination module is used to obtain the target global model based on the target global model parameters; The aggregation module is used to aggregate parameters according to the following formula to obtain the aggregation model parameters: in, These are the parameters for the aggregation model; These are the global model parameters for the i-th client; These are the local model parameters for the i-th client; The processing module is configured to: add noise equal to the global differential privacy budget threshold to the aggregated model parameters when the aggregated privacy protection level is greater than the global differential privacy budget threshold; use the aggregated model parameters as target global model parameters when the aggregated privacy protection level is less than the global differential privacy budget threshold; and send the target global model parameters as global model parameters for the next round to the next selected client.

7. A personalized differential privacy federated learning system, characterized in that, The system, applied to a client, includes: The second receiving module is used to receive global model parameters and privacy types sent by the server; The perturbation module is used to perturb the global model parameters using local data based on the global model parameters and the privacy type, so as to obtain the local model parameters and the privacy protection level. The second sending module is used to send the local model parameters and the privacy protection level to the client; The perturbation module is used to determine a privacy protection level based on privacy preferences, wherein the privacy protection level is a level of the privacy type; and to perturb the global model parameters using the privacy protection level and local data to obtain local model parameters.

Images