A method, system and terminal for privacy protection in two-party federated learning

Abstract

The application discloses a method, system and terminal for privacy protection in two-party federated learning. The method first defines two parties participating in privacy protection. A first participant establishes a global parameter model, determines a loss function thereof, and sends the loss function to a second participant. The two parties respectively train the global parameter model locally to obtain local gradient data or local model parameters, a local loss function, and a weighted average of the local gradient data or the local model parameters and the local loss function by using a two-party weighted average algorithm. The parameters in the global parameter model are updated. Iteration is performed until final parameters are obtained. Finally, a global parameter model adopting the final parameters is output to predict and analyze a set problem in a privacy protection scenario. The application can solve the problem that the server-client mode in the prior art is not applicable to two-party federated learning and improve the confidentiality of privacy data.

Description

Technical Field

[0001] This application relates to the field of privacy data protection technology, and in particular to a method, system and terminal for privacy protection in two-party federated learning. Background Technology

[0002] With the rapid popularization of smart terminal devices, a large amount of personal information data is collected by various online platforms. Therefore, the protection of personal privacy data is receiving increasing attention, and how to protect privacy is an important technical issue.

[0003] Current methods for protecting personal information privacy typically employ a server-side and client-side approach, enhancing the confidentiality of federated learning through homomorphic encryption systems based on non-interactive zero-knowledge proofs. Specifically, the model parameters are encrypted on the local client and then sent to the server. The server aggregates the encrypted model parameters, decrypts the aggregated data, and finally calculates a new global model. For example, all clients hold both public and private keys, while the server only holds the public key. Each time, a subset of clients is randomly selected for training. After the local clients send their encrypted data to the server, the server aggregates the encrypted model parameters, and the aggregated global private parameters are directly sent to the client, which then decrypts the encrypted global private parameters.

[0004] However, current methods for protecting personal information privacy primarily employ a server-side and client-side approach. These scenarios typically involve multiple users and a server. In reality, there are many scenarios with only two participants who communicate directly without going through a server. Existing methods are unsuitable for these two-participant situations, namely, two-party federated learning scenarios. Therefore, there is an urgent need to provide a method for privacy protection in two-party federated learning. Summary of the Invention

[0005] This application provides a method, system, and terminal for privacy protection in two-party federated learning, to solve the problem that the existing server-client approach is not suitable for two-party federated learning.

[0006] To address the aforementioned technical problems, the embodiments of this application disclose the following technical solutions:

[0007] A method for privacy protection in two-party federated learning, the method comprising:

[0008] S1: Define the two parties involved in privacy protection as: the first party and the second party. The privacy data includes: the number of data of the first party, the number of data of the second party, and federated learning parameters. The federated learning parameters are local gradient data or local model parameters.

[0009] S2: The first participant establishes a global parameter model, determines the loss function of the global parameter model, and sends the global parameter model and the loss function to the second participant;

[0010] S3: The first participant and the second participant train the global parameter model locally, obtain local gradient data or local model parameters, and determine the local loss function.

[0011] S4: Using the algorithm for calculating the weighted average of the two sides, calculate the weighted average of the local gradient data or local model parameters, as well as the weighted average of the local loss function.

[0012] S5: Update the parameters in the global parameter model based on the calculated local gradient data or the weighted average of the local model parameters, and the weighted average of the local loss function.

[0013] Based on the updated parameters, repeat steps S3-S5 until the set iteration stop condition is met, and obtain the final parameters in the global parameter model;

[0014] S6: Output a global parameter model that takes the final parameters, which is used to predict and analyze the set problems in the privacy protection scenario.

[0015] Optionally, the set iteration stopping condition is: the number of iterations reaches a preset number of iterations, or the global loss function converges.

[0016] Optionally, the step of calculating the weighted average of local gradient data or local model parameters and the weighted average of local loss function using the algorithm for calculating the weighted average of two sides includes:

[0017] Define the first participant as having N1 data points and the second participant as having N2 data points. Each data point has multiple components. When calculating the weighted average of the local gradient data, the multiple components are the local gradient data and the local loss function. When calculating the weighted average of the local model parameters, the multiple components are the local model parameters and the local loss function.

[0018] By multiplying by the same factor, the first and second participants respectively convert their relevant data into integers;

[0019] The first participant generates a probabilistic public-key encryption algorithm key and sends the corresponding public key to the second participant;

[0020] The first participant encrypted its data count and the data converted to integers before sending it to the second participant.

[0021] The second participant encrypts its data count and the data converted to integers, and randomly processes the product of the corresponding data counts of the first and second participants and the product of the corresponding converted integer data in the ciphertext, and sends the randomly processed data to the first participant.

[0022] The first participant decrypts the randomly processed data and performs an approximate quotient calculation before sending it to the second participant;

[0023] Based on the quotient result and the correspondence of the randomly processed data, the second participant calculates the approximate average value of all data components and sends the approximate average value to the first participant.

[0024] Optionally, the second participant encrypts its data count and the data converted to integers, and randomly processes the product of the corresponding data counts of the first and second participants and the product of the corresponding converted integer data on the ciphertext, and sends the randomly processed data to the first participant, including:

[0025] The second participant encrypts the number of data items and the data converted to integers to obtain ciphertext;

[0026] Calculate the random number power of the product of the number of data from the first participant and the number of data from the second participant on the ciphertext, and the random number power of the product of the data from the first participant converted to integers and the data from the second participant converted to integers, to obtain the randomly processed data, wherein any of the random numbers contains multiple prime factors;

[0027] The randomly processed data is sent to the first participant.

[0028] Optionally, after calculating the random power of the product of the number of data items from the first participant and the number of data items from the second participant on the ciphertext, and the random power of the product of the data items from the first participant converted to integers and the data items from the second participant converted to integers, to obtain the randomly processed data, the method further includes:

[0029] The order of the random numbers is obfuscated.

[0030] Optionally, the size of the random number is limited so that the sizes of the different randomly processed data obtained by decrypting the randomly processed data using the first parameter are approximately equal.

[0031] Optionally, the first participant decrypts and approximates the quotient of the randomly processed data before sending it to the second participant, including:

[0032] The first participant decrypts the randomly processed data;

[0033] Based on the homomorphism of probabilistic public-key encryption algorithms, the product of the sum of the corresponding data of the first participant and the second participant, and the product of the sum of the corresponding integer data multiplied by different random numbers, is calculated and then confused.

[0034] Based on the set error range, the product result after confusion is approximated and the quotient is calculated.

[0035] The approximate quotient result is sent to the second participant.

[0036] Optionally, the step of calculating an approximate average value of all data components based on the quotient result and the correspondence of the randomly processed data, and sending the approximate average value to the first participant, includes:

[0037] Based on the multiplier used by the first and second participants when converting the relevant data into integers, the calculation of any random number of the data after random processing by both parties, and the results of encrypted and obfuscated data, the second participant uses the component calculation formula to calculate the approximate average value of all components of the data.

[0038] The approximate average value is sent to the first participant.

[0039] A system for privacy protection in two-party federated learning, the system comprising:

[0040] The definition module is used to define the two parties involved in privacy protection as: the first participant and the second participant. The privacy data includes: the number of data of the first participant, the number of data of the second participant, and the federated learning parameters, wherein the federated learning parameters are local gradient data or local model parameters.

[0041] A global parameter model building module is set in the first participant, used to build a global parameter model, determine the loss function of the global parameter model, and send the global parameter model and loss function to the second participant;

[0042] The model training module is set up in the first participant and the second participant, and is used by the first participant and the second participant to train the global parameter model locally, obtain local gradient data or local model parameters, and determine the local loss function.

[0043] The weighted average calculation module is used to calculate the weighted average of local gradient data or local model parameters, as well as the weighted average of local loss function, using an algorithm for calculating the weighted average of two sides.

[0044] The parameter update module is used to update the parameters in the global parameter model based on the calculated local gradient data or the weighted average of the local model parameters, as well as the weighted average of the local loss function.

[0045] The iteration module is used to repeatedly start the model training module, the weighted average calculation module, and the parameter update module in sequence according to the updated parameters until the set iteration stopping condition is reached, and to obtain the final parameters in the global parameter model.

[0046] The output module is used to output a global parameter model that takes the final parameters, which is used to predict and analyze the set problems in the privacy protection scenario.

[0047] A terminal, the terminal comprising: a processor and a memory communicatively connected to the processor, wherein,

[0048] The memory stores instructions that can be executed by the processor, which, when executed by the processor, enables the processor to perform a privacy-preserving method in two-party federated learning as described in any of the above descriptions.

[0049] The technical solutions provided by the embodiments of this application may include the following beneficial effects:

[0050] This application provides a privacy protection method for two-party federated learning. The method first defines the two parties involved in privacy protection as a first party and a second party. The first party establishes a global parameter model for setting the problem, determines the loss function of the model, and sends the global parameter model and loss function to the second party. Both parties train the global parameter model locally, obtaining local gradient data or local model parameters, and determining their local loss functions. Then, using an algorithm for calculating the weighted average of the two parties' data, the weighted average of the local gradient data or local model parameters, and the weighted average of the local loss function are calculated. These weighted averages are used to update the parameters in the global parameter model, and this process is iterated multiple times until the final parameters are determined. A global parameter model using the final parameters is then output, and this model is used for problem prediction and analysis. This embodiment provides a privacy protection method for two-party federated learning scenarios. It uses an algorithm for calculating the weighted average of the federated learning parameters and the weighted average of the loss function. This algorithm can more securely calculate the weighted average of the two parties when the data has multiple components, thereby effectively improving the confidentiality of privacy data. Moreover, this method is based on two-party federated learning, which can solve the problem that the existing server-client approach is not suitable for two-party federated learning.

[0051] This application also provides a privacy-preserving system for two-party federated learning. The system mainly includes: a definition module, a global parameter model building module, a model training module, a weighted average calculation module, a parameter update module, an iteration module, and an output module. This system is designed for two-party federated learning. Through the global parameter model building module, a privacy-preserving model is established. Through the model training module, weighted average calculation module, parameter update module, and iteration module, the final parameters for the global parameter model are obtained. These final parameters are then used to predict and analyze a given problem, thereby effectively solving the problem that existing technologies using server-side and client-side approaches are unsuitable for two-party federated learning.

[0052] By setting up a weighted average calculation module, this system uses an algorithm for calculating the weighted average of two parties to calculate the weighted average of federated learning parameters and the weighted average of the loss function. This algorithm for calculating the weighted average of two parties can more securely calculate the weighted average of two parties when the data has multiple components, thereby effectively improving the confidentiality of privacy data.

[0053] This application also provides a terminal that has the same technical effects as the above-mentioned methods and systems for privacy protection in federated learning, which will not be elaborated here.

[0054] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and do not limit this application. Attached Figure Description

[0055] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application.

[0056] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, for those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0057] Figure 1 A flowchart illustrating a method for privacy protection in two-party federated learning provided in this application embodiment;

[0058] Figure 2 This is a schematic diagram of the structure of a system for privacy protection in two-party federated learning, provided as an embodiment of this application. Detailed Implementation

[0059] To enable those skilled in the art to better understand the technical solutions in this application, the technical solutions in the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments in this application, all other embodiments obtained by those of ordinary skill in the art without creative effort should fall within the scope of protection of this application.

[0060] To better understand this application, the embodiments of this application will be explained in detail below with reference to the accompanying drawings.

[0061] Example 1

[0062] See Figure 1 , Figure 1 This is a flowchart illustrating a method for privacy protection in two-party federated learning, provided as an embodiment of this application. Figure 1 As can be seen, the method for privacy protection in two-party federated learning in this embodiment mainly includes the following processes:

[0063] S1: Define the two parties involved in privacy protection as the first participant and the second participant.

[0064] The privacy data in this embodiment includes: the number of data from the first participant, the number of data from the second participant, and federated learning parameters, where the federated learning parameters are local gradient data or local model parameters.

[0065] Privacy protection of personal information occurs in many scenarios, particularly in practical applications involving data exchange between two parties that require privacy protection. Examples include two hospitals collaborating to study the relationship between the effectiveness of a drug and patient characteristics such as age, physical condition, and dosage; or two shopping malls collaborating to study the sales levels of different clothing styles each season. In such scenarios, the two users are defined as the first and second participants. This method, while ensuring the confidentiality of the data volume, local gradient data, or local model parameters between the two parties, allows for the prediction and analysis of results for specific scenarios, such as: obtaining the relationship between the effectiveness of a drug and patient characteristics like age, physical condition, and dosage; obtaining the relationship between sales levels and seasons / clothing styles; and predicting sales levels.

[0066] S2: The first participant establishes a global parameter model, determines the loss function of the global parameter model, and sends the global parameter model and loss function to the second participant.

[0067] S3: The first and second participants train the global parameter model locally, obtain local gradient data or local model parameters, and determine the local loss function.

[0068] In this embodiment, the appropriate federated learning parameters are selected according to the specific application scenario. Either local gradient data or local model parameters can be selected.

[0069] S4: Using the algorithm for calculating the weighted average of the two sides, calculate the weighted average of the local gradient data or local model parameters, as well as the weighted average of the local loss function.

[0070] Specifically, step S4 includes the following process:

[0071] S41: Define the first participant as having N1 data points and the second participant as having N2 data points. Each data point has multiple components. When calculating the weighted average of the local gradient data, the multiple components are the local gradient data and the local loss function. When calculating the weighted average of the local model parameters, the multiple components are the local model parameters and the local loss function.

[0072] Step S4 can also be understood as: allocating weights according to the ratio of N1 and N2, and applying them to the data (a1, a2, ..., a d ),(b1,b2,…,b d Calculate the weighted average based on the components.

[0073] S42: By multiplying by the same factor, the first and second participants respectively convert their relevant data into integers.

[0074] According to step S4, assuming that both the first and second participants have d components, the relevant data of the first participant is (a1N1, a2N1, ..., a...). d N1), the relevant data of the second participant is (b1N2,b2N2,…,b d N2), through step S42, the relevant data are multiplied by the same multiple s, and correspondingly transformed into (X1,X2,…,X). d (Y1,Y2,…,Y) and (Y1,Y2,…,Y) d ).

[0075] S43: The first participant generates a probabilistic public-key encryption algorithm key and sends the corresponding public key to the second participant.

[0076] In this embodiment, the probabilistic public key encryption algorithm uses the Paillier algorithm. The first participant generates the Paillier algorithm key (n,g,λ) and sends the public key (n,g) to the second participant.

[0077] S44: The first participant sends the number of its data items and the data converted to integers to the second participant after encryption.

[0078] The first participant encrypts data N1, (X1,X2,…,X...).d ), thus obtaining E(N1),(E(X1),E(X2),…,E(X d The encrypted data is then sent to the second participant.

[0079] S45: The second participant encrypts its data count and the data converted to integers, and randomly processes the product of the corresponding data counts of the first and second participants and the product of the corresponding converted integer data on the ciphertext, and sends the randomly processed data to the first participant.

[0080] Furthermore, step S45 includes the following process:

[0081] S451: The second participant encrypts the number of its data items and the data converted to integers to obtain the ciphertext.

[0082] S452: Calculate the random power of the product of the number of data from the first participant and the number of data from the second participant on the ciphertext, and the random power of the product of the data from the first participant converted to integers and the data from the second participant converted to integers, to obtain the randomly processed data, wherein any random number contains multiple prime factors.

[0083] S454: Send the randomly processed data to the first participant.

[0084] Furthermore, step S452 is followed by step S453: obfuscating the order of the random numbers.

[0085] From steps S451-S454 above, it can be seen that the second participant encrypts the data N2,(Y1,Y2,…,Y…). d ), get E(N2),(E(Y1),E(Y2),…,E(Y d Then calculate the result on the ciphertext. Where r0, r1, r2, ..., r d It is a random number containing multiple prime factors, which are then shuffled to obtain (w′0, w′1, w′2, ..., w′). d The random numbers are sent to the first participant. In this embodiment, the obfuscation can be achieved by adjusting the order, that is, adjusting the order of the random numbers.

[0086] Furthermore, this embodiment also includes: limiting the size of the random numbers so that the sizes of the different decrypted randomly processed data obtained by decrypting the randomly processed data using the first parameter are approximately equal. Specifically, this can be applied to r0, r1, r2, ..., r d The size is appropriately limited, for example, r0≈r1·s·b1≈r2·s·b2≈…≈r d ·s·bd This allows the first participant to decrypt r0(N1+N2), r1(X1+Y1), r2(X2+Y2), ..., r in the next step. d (X d +Y d The sizes are roughly the same.

[0087] S46: The first participant decrypts the randomly processed data and performs an approximate quotient calculation before sending it to the second participant.

[0088] Specifically, step S46 includes:

[0089] S461: The first participant decrypts the randomly processed data.

[0090] S462: Based on the homomorphism of the Paillier algorithm, calculate the product of the sum of the corresponding data of the first participant and the second participant, the sum of the corresponding integer data, and different random numbers, and then confuse the results.

[0091] S463: Based on the set error range, approximate the quotient of the product after confusion.

[0092] S463: Send the approximate quotient result to the second participant.

[0093] The first participant has w0′, w1′, w2′, ..., w′ d Decryption: Based on the homomorphism of Paillier's algorithm, we can obtain the values ​​for r0(N1+N2), r1(X1+Y1), r2(X2+Y2), ..., r d (X d +Y d Let the results after obfuscation be D(w0′), D(w1′), D(w2′), ..., D(w′). d Choose a very small value ε based on the error requirement (e.g., ε = 10). -6 min{D(w0′),D(w1′),D(w2′),…,D(w′ d )}, and calculate It is then sent to the second participant.

[0094] By multiplying the above data, we can obtain... The value of , in this embodiment, is determined by setting a smaller approximation error limit ε, if the value of , in this embodiment, is , if the value of , in this embodiment, is , in this embodiment, by setting a smaller approximation error limit ε ... Once converted to decimals, it is very difficult to restore the original data form. Therefore, this method can effectively prevent participants from speculating on private data through fraction simplification, thereby enhancing the protection of privacy data.

[0095] S47: Based on the quotient result and the correspondence of the randomly processed data, the second participant calculates the approximate average value of all data components and sends the approximate average value to the first participant.

[0096] Specifically, step S47 includes:

[0097] S471: Based on the multiplier used by the first and second participants when converting the relevant data into integers, the calculation of any random number of the data after random processing by both parties, and the encrypted and obfuscated data, the second participant uses the component calculation formula to calculate the approximate average value of all components of the data.

[0098] Specifically, the second participant uses the formula The approximate average value of all components of the data is calculated, where i = 1, 2, ..., d, k i r is the approximate average value of the i-th component. i It is any random number calculated from the data after random processing by both parties, where s is the multiplier used by the first and second participants when converting the relevant data into integers, {w0,w1,w2,…,w d} represents the encrypted data, {w′0,w′1,w′2,…,w′ d} is {w0,w1,w2,…,w d The result of obfuscation is that the data in the two sets is the same but in a different order. For {w′0,w′1,w′2,…,w′ d The decryption result at index i0 in} is {w′0,w′1,w′2,…,w′ d The index in} is the decryption result of j0, and ε is the approximate error limit.

[0099] S472: Send the approximate average value to the first participant.

[0100] From steps S471-S472 above, it can be seen that the second participant, through w0, w1, w2, ..., w d and w′0, w′1, w′2, ..., w′ d The correspondence, and combined with r0,r1,r2,…,r d The values ​​are used to approximate the average values ​​k1, k2, ..., k of all data components. d And approximate average values ​​k1,k2,…,k d Send to the first participant.

[0101] In this embodiment, if w0 and Correspondingly, the first calculation is as follows: If w iand Correspondingly, we can obtain Where i = 1, 2, ..., d.

[0102] See also Figure 1 As can be seen, after calculating the weighted average of the local gradient data or local model parameters and the weighted average of the local loss function using the algorithm for calculating the weighted average of the two sides, step S5 is executed: based on the calculated weighted average of the local gradient data or local model parameters and the weighted average of the local loss function, the parameters in the global parameter model are updated.

[0103] Based on the updated parameters, repeat steps S3-S5 until the set iteration stop condition is met, and obtain the final parameters in the global parameter model.

[0104] In this embodiment, the iteration stopping condition is set as follows: the number of iterations reaches a preset number of iterations, or the global loss function converges. The global loss function is the weighted average of the local loss functions.

[0105] After obtaining the final parameters in the global parameter model, execute step S6: output the global parameter model with the final parameters, which is used to predict and analyze the setting issues in the privacy protection scenario.

[0106] The following example details the process of privacy protection in two-party federated learning using the method described in this embodiment:

[0107] 1) The first participant establishes the global parameter model and loss function, and sends them to the second participant, assuming the parameters are θ1, θ2, ..., θ d .

[0108] 2) The first and second participants train locally to obtain local gradient data or local model parameters and calculate the local loss function. If local gradient data is selected as the federated learning parameter, then a) is executed to calculate the gradient weighted average; if local model parameters are selected as the federated learning parameter, then b) is executed to calculate the model parameter weighted average, which is the weighted average.

[0109] a) Gradient-weighted average: Let g be the local gradient data obtained by the first participant. 11 ,g 12 ,…,g 1d The second participant obtains the local gradient data as g. 21 ,g 22 ,…,g 2d .

[0110] b) Weighted average: Let θ be the local model parameters obtained by the first participant. 11 ,θ12 ,…,θ 1d The second participant obtains the local gradient data as θ. 21 ,θ 22 ,…,θ 2d .

[0111] Suppose that the local loss function calculated by the first participant is F1 = F1(θ1,θ2,…,θ). d The second participant calculates the local loss function as F2 = F2(θ1,θ2,…,θ). d ).

[0112] 3) The first and second participants use an algorithm to calculate the weighted average of the local gradient data or local model parameters, as well as the weighted average of the local loss function, which is the global loss function.

[0113] a) Gradient-weighted average: Calculate g 11 ,g 12 ,…,g 1d F1 and g 21 ,g 22 ,…,g 2d The weighted average of F2

[0114] b) Weighted average: Calculate and weighted average

[0115] Let N1 be the number of data points possessed by the first participant and N2 be the number of data points possessed by the second participant. The weights are determined based on the ratio of N1±n1 to N2±n2. Here, n1 is the random number chosen by the first participant in this round, and n2 is the random number chosen by the second participant in this round, satisfying n1 << N1 and n2 << N2. This embodiment, by adding random numbers n1 and n2, effectively prevents users from slightly altering a component and exploiting the difference between two results to attack privacy data when a component is relatively stable. Adding random numbers removes the correlation between any two results, thereby effectively preventing users from attacking using the difference between two results and improving the confidentiality and security of privacy data.

[0116] 4) Both the first and second participants update the global model parameters.

[0117] Gradient-weighted average: Where η represents the learning rate;

[0118] Weighted average:

[0119] Then repeat steps 2)-4) until the required number of iterations is reached, or the global loss function is correct. Stop when convergence occurs.

[0120] Example 2

[0121] exist Figure 1 Based on the illustrated embodiment, see also Figure 2 , Figure 2 This is a schematic diagram of the structure of a system for privacy protection in two-party federated learning, provided as an embodiment of this application. Figure 2 As can be seen, the privacy protection system in this embodiment of two-party federated learning mainly includes: a definition module, a global parameter model building module, a model training module, a weighted average calculation module, a parameter update module, an iteration module, and an output module.

[0122] The system includes the following modules: a definition module, which defines the two parties involved in privacy protection as a first participant and a second participant; privacy data includes the number of data points for the first participant and the second participant, as well as federated learning parameters, which are either local gradient data or local model parameters; a global parameter model building module, located on the first participant, is used to build a global parameter model, determine the loss function of the global parameter model, and send the global parameter model and loss function to the second participant; a model training module, located on both the first and second participants, is used for the first and second participants to train the global parameter model locally, obtain local gradient data or local model parameters, and determine the local loss function; and a weighted average calculation module. The system comprises four modules: a module for calculating the weighted average of local gradient data or local model parameters and the weighted average of local loss functions using a two-way weighted average algorithm; a parameter update module for updating the parameters in the global parameter model based on the calculated weighted average of local gradient data or local model parameters and the weighted average of local loss functions; an iteration module for repeatedly starting the model training module, weighted average calculation module, and parameter update module sequentially based on the updated parameters until a set iteration stopping condition is met, and obtaining the final parameters in the global parameter model; and an output module for outputting the global parameter model with the final parameters, used for prediction and analysis of specific problems in privacy protection scenarios.

[0123] Furthermore, the weighted average calculation module includes:

[0124] The definition unit is used to define that the first participant has N1 data points and the second participant has N2 data points. Each data point has multiple components. When calculating the weighted average of the local gradient data, the multiple components are the local gradient data and the local loss function. When calculating the weighted average of the local model parameters, the multiple components are the local model parameters and the local loss function.

[0125] The format conversion unit is used to convert the relevant data of the first participant and the second participant into integers by multiplying by the same factor.

[0126] The algorithm key generation unit is used to generate the Paillier algorithm key by the first participant and send the corresponding public key to the second participant;

[0127] The first encryption unit is set in the first participant and is used to encrypt the number of data items of the first participant and the data converted into integers before sending them to the second participant;

[0128] The second encryption unit is set in the second participant and is used to encrypt the number of data of the second participant and the data converted into integers. The unit also performs random processing on the product of the corresponding number of data of the first participant and the second participant and the product of the corresponding data converted into integers in the ciphertext, and sends the randomly processed data to the first participant.

[0129] The decryption unit, located in the first participant, is used to decrypt the randomly processed data and perform approximate quotient calculations before sending it to the second participant.

[0130] An approximate average value calculation unit is located in the second participant. It is used to calculate the approximate average value of all components of the data based on the quotient result and the correspondence of the randomly processed data, and then send the approximate average value to the first participant.

[0131] This embodiment describes the working principle and method of a privacy-preserving system in two-party federated learning. Figure 1 The embodiments shown have been described in detail, and the two embodiments can be referred to each other, so they will not be repeated here.

[0132] Example 3

[0133] This application also provides a terminal, which includes: a processor and a memory communicatively connected to the processor, wherein the memory stores instructions executable by the processor, and the instructions are executed by the processor to enable the processor to perform the privacy protection method in the above two-party federated learning.

[0134] The following methods are used to protect privacy during two-party federated learning executed by the processor:

[0135] S1: Define the two parties involved in privacy protection as: the first party and the second party. The privacy data includes: the number of data of the first party, the number of data of the second party, and the federated learning parameters. The federated learning parameters are local gradient data or local model parameters.

[0136] S2: The first participant establishes a global parameter model, determines the loss function of the global parameter model, and sends the global parameter model and loss function to the second participant;

[0137] S3: The first and second participants train the global parameter model locally, obtain local gradient data or local model parameters, and determine the local loss function.

[0138] S4: Using the algorithm for calculating the weighted average of the two sides, calculate the weighted average of the local gradient data or local model parameters, as well as the weighted average of the local loss function.

[0139] S5: Update the parameters in the global parameter model based on the calculated local gradient data or the weighted average of the local model parameters, and the weighted average of the local loss function.

[0140] Based on the updated parameters, repeat steps S3-S5 until the set iteration stop condition is met, and obtain the final parameters in the global parameter model;

[0141] S6: Outputs a global parameter model with final parameters, used for prediction and analysis of setting issues in privacy protection scenarios.

[0142] For parts not described in detail in this embodiment, please refer to [link / reference]. Figure 1 and Figure 2 The embodiments shown will not be described in detail here.

[0143] The above description is merely a specific embodiment of this application, enabling those skilled in the art to understand or implement this application. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of this application. Therefore, this application is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A method for privacy protection in two-party federated learning, characterized in that, The method includes: S1: Define the two parties involved in privacy protection as: the first party and the second party. The privacy data includes: the number of data of the first party, the number of data of the second party, and federated learning parameters. The federated learning parameters are local gradient data or local model parameters. S2: The first participant establishes a global parameter model, determines the loss function of the global parameter model, and sends the global parameter model and the loss function to the second participant; S3: The first participant and the second participant train the global parameter model locally, obtain local gradient data or local model parameters, and determine the local loss function. S4: Using the algorithm for calculating the weighted average of the two sides, calculate the weighted average of the local gradient data or local model parameters, as well as the weighted average of the local loss function. The algorithm for calculating the weighted average of two sides, which calculates the weighted average of local gradient data or local model parameters, and the weighted average of local loss functions, includes: Define the first participant as follows: There are data points, and the second participant has... Each data point has multiple components. When calculating the weighted average of the local gradient data, the multiple components are the local gradient data and the local loss function. When calculating the weighted average of the local model parameters, the multiple components are the local model parameters and the local loss function. By multiplying by the same factor, the first and second participants respectively convert their relevant data into integers; The first participant generates a probabilistic public-key encryption algorithm key and sends the corresponding public key to the second participant; The first participant encrypted its data count and the data converted to integers before sending it to the second participant. The second participant encrypts its data count and the data converted to integers, and randomly processes the product of the corresponding data counts of the first and second participants and the product of the corresponding converted integer data in the ciphertext, and sends the randomly processed data to the first participant. The first participant decrypts the randomly processed data and performs an approximate quotient calculation before sending it to the second participant; Based on the quotient result and the correspondence of the randomly processed data, the second participant calculates the approximate average value of all data components and sends the approximate average value to the first participant. S5: Update the parameters in the global parameter model based on the calculated local gradient data or the weighted average of the local model parameters, and the weighted average of the local loss function. Based on the updated parameters, repeat steps S3-S5 until the set iteration stop condition is met, and obtain the final parameters in the global parameter model; S6: Output a global parameter model that takes the final parameters, which is used to predict and analyze the set problems in the privacy protection scenario.

2. The method for privacy protection in two-party federated learning according to claim 1, characterized in that, The set iteration stopping conditions are: the number of iterations reaches the preset number of iterations, or the global loss function converges.

3. The method for privacy protection in two-party federated learning according to claim 1, characterized in that, The second participant encrypts its data count and the data converted to integers, and randomly processes the product of the corresponding data counts of the first and second participants and the product of the corresponding converted integer data on the ciphertext, and sends the randomly processed data to the first participant, including: The second participant encrypts the number of data items and the data converted to integers to obtain ciphertext; Calculate the random number power of the product of the number of data from the first participant and the number of data from the second participant on the ciphertext, and the random number power of the product of the data from the first participant converted to integers and the data from the second participant converted to integers, to obtain the randomly processed data, wherein any of the random numbers contains multiple prime factors; The randomly processed data is sent to the first participant.

4. The method for privacy protection in two-party federated learning according to claim 3, characterized in that, After calculating the random power of the product of the number of data items from the first participant and the number of data items from the second participant on the ciphertext, and the random power of the product of the data items from the first participant converted to integers and the data items from the second participant converted to integers, and obtaining the randomly processed data, the method further includes: The order of the random numbers is obfuscated.

5. The method for privacy protection in two-party federated learning according to claim 3, characterized in that, The size of the random number is limited so that the sizes of the different randomly processed data obtained by the first participant after decrypting the randomly processed data are approximately equal.

6. The method for privacy protection in two-party federated learning according to claim 1, characterized in that, The first participant decrypts and approximates the quotient of the randomly processed data, then sends it to the second participant, including: The first participant decrypts the randomly processed data; Based on the homomorphism of the probabilistic public key encryption algorithm, the sum of the corresponding data of the first participant and the second participant, and the product of the sum of the corresponding integer data multiplied by different random numbers are calculated and then confused. Based on the set error range, the product result after confusion is approximated and the quotient is calculated. The approximate quotient result is sent to the second participant.

7. The method for privacy protection in two-party federated learning according to claim 1, characterized in that, The step of calculating the approximate average value of all data components based on the quotient result and the correspondence of the randomly processed data, and sending the approximate average value to the first participant, includes: Based on the multiplier used by the first and second participants when converting the relevant data into integers, the calculation of any random number of the data after random processing by both parties, and the results of encrypted and obfuscated data, the second participant uses the component calculation formula to calculate the approximate average value of all components of the data. The approximate average value is sent to the first participant.

8. A system for privacy protection in two-party federated learning, characterized in that, The system is used to implement the method of claim 1, and the system comprises: The definition module is used to define the two parties involved in privacy protection as: the first participant and the second participant. The privacy data includes: the number of data of the first participant, the number of data of the second participant, and the federated learning parameters, wherein the federated learning parameters are local gradient data or local model parameters. A global parameter model building module is set in the first participant, used to build a global parameter model, determine the loss function of the global parameter model, and send the global parameter model and loss function to the second participant; The model training module is set up in the first participant and the second participant, and is used by the first participant and the second participant to train the global parameter model locally, obtain local gradient data or local model parameters, and determine the local loss function. The weighted average calculation module is used to calculate the weighted average of local gradient data or local model parameters, as well as the weighted average of local loss function, using an algorithm for calculating the weighted average of two sides. The parameter update module is used to update the parameters in the global parameter model based on the calculated local gradient data or the weighted average of the local model parameters, as well as the weighted average of the local loss function. The iteration module is used to repeatedly start the model training module, the weighted average calculation module, and the parameter update module in sequence according to the updated parameters until the set iteration stopping condition is reached, and to obtain the final parameters in the global parameter model. The output module is used to output a global parameter model that takes the final parameters, which is used to predict and analyze the set problems in the privacy protection scenario.

9. A terminal, characterized in that, The terminal includes: a processor and a memory communicatively connected to the processor, wherein... The memory stores instructions that can be executed by the processor, which, when executed by the processor, enables the processor to perform the privacy protection method in two-party federated learning as described in any one of claims 1 to 7.

Images