Federal learning privacy protection method and system

Abstract

The invention discloses a federated learning privacy protection method and system, and belongs to the technical field of data security, and the method comprises the steps: segmenting a secret into a plurality of secret shares through employing a multi-secret reusable secret sharing method, and transmitting the plurality of secret shares to corresponding users; the users perform mask encryption on respective local model parameters and send local model parameter ciphertexts to the server, and the server aggregates the received local model parameter ciphertexts to obtain model parameter aggregation ciphertexts; the user establishes a secret share combination according to the secret share of the current online user and sends the secret share combination to the server, and the server reconstructs all the received secret share combinations to obtain a secret combination, decrypts the model parameter aggregation ciphertext by using the secret combination to obtain a global model parameter, and sends the global model parameter to the user; and the user performs local training by adopting the global model parameters to obtain updated local model parameters, performs mask encryption on the local model parameters again, and iteratively updates the global model. According to the method, the problem of low calculation efficiency caused by repeated secret sharing operation in multi-round federal aggregation is solved, then, the user data privacy is protected by adopting random single mask encryption, and federal aggregation can be efficiently completed when part of users are offline.

Description

Technical Field

[0001] The present invention relates to the field of data security technology, and in particular to a method and system for protecting privacy in federated learning. Background Art

[0002] In the application of military intelligence, a large amount of military sensitive data needs to be shared and processed between different departments and institutions. Intelligence intelligent analysis systems, battlefield situation awareness platforms, etc. all require a large amount of data for training and optimization, such as intelligence data, weapon equipment performance data, battlefield situation awareness data, equipment system data, etc. Because it involves national and military secrets, it is more sensitive and private than personal information. In order to prevent the loss of secrets, it is difficult to share the original data directly, forming a military data island; Federated learning, as a specific artificial intelligence technology, is originally intended to solve the data island problem and strengthen data flow and sharing on the premise of achieving data privacy protection. It can achieve distributed collaboration between multiple military departments or institutions without sharing the original data, which can solve the problem of protecting military sensitive data to a certain extent and promote the development of military intelligence.

[0003] However, with the advancement of technology, federated learning itself also has privacy and security issues. Plaintext gradient information is a key parameter for model training. Although it does not directly contain the original data, it may also be used by attackers to leak the data of a single user or the privacy of the model. The current privacy protection technologies in federated learning mainly include cryptographic technology, differential privacy technology, and trusted hardware technology. Differential privacy technology protects privacy by adding noise to the data. Although it does not cause a lot of communication and computing overhead, the availability of data will be affected, resulting in a decrease in training accuracy and model number, and it is not suitable for application scenarios with high-precision data. Especially in horizontal federated learning, too much noise will make it difficult for the model to converge. Trusted hardware technology mainly protects the privacy of data through a trusted execution environment. Its memory is limited and it is vulnerable to side channel attacks, which greatly reduces the application effect in federated learning. Cryptographic technology can effectively hide intermediate parameters such as gradients in federated learning, making it impossible for adversaries to obtain plaintext gradient information and restore data, thereby effectively protecting the data privacy of federated learning. Therefore, cryptographic technology is the mainstream way and research hotspot to achieve privacy protection in federated learning.

[0004] Federated learning protection based on cryptographic technology mainly includes mask encryption technology, homomorphic encryption technology, and secure multi-party computing technology. The federated learning protection method based on homomorphic encryption has good accuracy, but the computational and storage overheads are too large, and it has limitations in the distributed data training of federated learning. Federated learning based on secure multi-party computing has good privacy, but due to too many communication rounds, the communication overhead is large, and it is difficult to ensure the availability of the global model in scenarios with malicious users. The privacy protection of federated learning based on mask encryption is simple in structure and has high computational efficiency. Therefore, this study mainly focuses on the privacy protection of federated learning based on mask encryption and its application in military intelligence.

[0005] The privacy protection scheme of federated learning based only on mask encryption is simple in structure and highly efficient in computation, but its robustness is relatively poor. In particular, when the user is offline, the server cannot aggregate the encrypted gradient, and thus cannot obtain the correct aggregation result. In order to deal with the situation where some users are offline, the current solution adopts the SecAgg scheme proposed by Bonawitz et al. in combination with the secret sharing strategy, which is only applicable to a single round of federated aggregation. Federated learning requires multiple rounds of aggregation to converge, and the secret sharing before each round of federated aggregation brings huge computational and communication overheads, which affects the efficiency of federated learning. How to reduce the computational and communication volume of multiple secret sharing is an urgent problem to be solved. Summary of the invention

[0006] In view of the privacy protection problem in multi-round federated learning model aggregation, the present invention provides a federated learning privacy protection method and system, which solves the problem of large computing and communication volume caused by too many secret sharing times in multi-round federated learning data aggregation.

[0007] The present invention is achieved through the following technical solutions:

[0008] In a first aspect, the present application provides a method for protecting privacy in federated learning, comprising the following steps:

[0009] The user adopts a multi-secret reusable secret sharing method to divide the secret into multiple secret shares and send the multiple secret shares to the corresponding users respectively;

[0010] The users use their own secrets as masks to encrypt their own local model parameters, and send the local model parameter ciphertexts to the server. The server aggregates the received local model parameter ciphertexts to obtain the model parameter aggregate ciphertext.

[0011] The user establishes a secret share combination based on the secret share of the current online user and sends the secret share combination to the server. The server reconstructs all the received secret share combinations to obtain the secret combination, and uses it to decrypt the model parameter aggregation ciphertext to obtain the global model parameters, and sends the global model parameters to the user; the user uses the global model parameters for local training to obtain updated local model parameters, and mask-encrypts the local model parameters again, and iteratively updates the global model until the global model converges.

[0012] Preferably, the method of using a multi-secret reusable secret sharing method to divide a secret into multiple secret shares comprises:

[0013] Determine multiple secret shares of the secret according to the secret share distribution function, and send the multiple secret shares to the corresponding multiple users respectively;

[0014] The secret share distribution function is as follows:

[0015]

[0016] Among them, s i For user u i The secret, For user u i Send to user u n The secret share.

[0017] Preferably, the cloud server aggregates the received local model parameter ciphertexts to obtain model parameter aggregate ciphertexts, including:

[0018] The server determines the number of online users based on the received local model parameter ciphertext. When the number of online users is greater than the set threshold, the server aggregates the local model parameter ciphertexts of all online users.

[0019] The method for ciphertext aggregation of local model parameters is as follows:

[0020]

[0021] Among them, C is the aggregated ciphertext of model parameters, m i Represents user u i The model parameter ciphertext, represents the online user list consisting of users who successfully sent the local model parameter ciphertext, H represents the hash function, T S Represents a time interval.

[0022] Preferably, the user uses the global model parameters for local training to obtain updated local model parameters, and mask and encrypt the local model parameters again, and iteratively update the global model until the global model converges, including the following process:

[0023] The server sends the global model parameters to the current online user, and the user uses the global model parameters for local training to obtain updated local model parameters;

[0024] After the local model is updated, the updated local model parameters are masked and encrypted and sent to the server. The server aggregates and decrypts to obtain the global model. The above process is repeated until the global model converges.

[0025] Preferably, the server reconstructs all received secret share combinations to obtain the secret combination in the following manner:

[0026] The user determines the online users according to the online user list, calculates the secret share combination of all online users' secret shares and sends it to the server;

[0027] The server determines the current online user based on the received secret share combination. When the number of online users is greater than a threshold, the server uses the Lagrange interpolation method combined with the secret reconstruction function to aggregate and reconstruct all secret share combinations to obtain a secret combination.

[0028] Preferably, the expression of the secret share combination is as follows:

[0029]

[0030] The aggregation reconstruction method of the secret combination is as follows:

[0031]

[0032] The expression of the secret reconstruction function is as follows:

[0033]

[0034] Among them, Y i is the combination of secret shares received by the user from the current user list, s i For user u i The secret, H represents the hash function, T S Represents a time interval.

[0035] Preferably, the method for determining the global model parameters is as follows:

[0036] The exponential form of the global model parameters is obtained by aggregating the ciphertext according to the secret combination and the model parameters, and the discrete logarithm of the exponential form is calculated to obtain the global model parameters.

[0037] Preferably, the expression of the global model parameter z is as follows:

[0038]

[0039] in, The online user list consisting of users who successfully sent the local model parameter ciphertext, m i Represents user u i The model parameter ciphertext, g represents the cyclic group The generator on .

[0040] In a second aspect, the present application provides a federated learning privacy protection system, comprising the following steps:

[0041] A secret share module, used for users to adopt a multi-secret reusable secret sharing method, divide the secret into multiple secret shares, and send the multiple secret shares to corresponding users respectively;

[0042] A model parameter aggregation module is used for users to use their own secrets as masks to encrypt their own local model parameters, and send the local model parameter ciphertext to the server. The server aggregates the received local model parameter ciphertext to obtain the model parameter aggregate ciphertext;

[0043] The global model training module is used for users to establish secret share combinations based on the secret shares of current online users, and send the secret share combinations to the server. The server reconstructs all received secret share combinations to obtain secret combinations, and uses them to decrypt the model parameter aggregation ciphertext to obtain global model parameters, and sends the global model parameters to the user; the user uses the global model parameters for local training to obtain updated local model parameters, and mask-encrypts the local model parameters again, and iteratively updates the global model until the global model converges.

[0044] In a third method, the present application provides an electronic device, including:

[0045] Memory for storing computer programs;

[0046] A processor is used to implement the steps of the federated learning privacy protection method when executing the computer program.

[0047] Compared with the prior art, the present invention has the following beneficial technical effects:

[0048] This application proposes a method for privacy protection of federated learning. Aiming at the problem of low computational efficiency caused by repeated secret sharing operations in multiple rounds of federated aggregation, a computationally efficient federated learning privacy protection scheme under the semi-honest security assumption is constructed. This method adopts a multi-secret reusable secret sharing mechanism to reduce the number of secret sharing in multiple rounds of federated aggregation. Secondly, a federated learning privacy protection scheme under the semi-honest assumption is constructed based on the multi-secret reusable secret sharing mechanism. This scheme uses random single-mask encryption to protect user data privacy and can efficiently complete federated aggregation when some users are offline.

[0049] This application also proposes a federated learning privacy protection system, an electronic device and a computer storage medium, which have all the advantages of the above-mentioned federated learning privacy protection method. BRIEF DESCRIPTION OF THE DRAWINGS

[0050] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required for use in the embodiments will be briefly introduced below. It should be understood that the following drawings only show certain embodiments of the present application and therefore should not be regarded as limiting the scope. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without creative work.

[0051] Figure 1 This is an architecture diagram of the federated learning privacy protection system based on multi-secret reusable secret sharing of the present invention;

[0052] Figure 2 A flowchart of a schematic diagram of multi-secret reusable secret sharing of the present invention;

[0053] Figure 3 The communication model diagram of the present invention;

[0054] Figure 4 This is a diagram of the computing efficiency of a single user under different disconnection rates according to an embodiment of the present invention;

[0055] Figure 5 This is a diagram of the computing efficiency of a single user under different numbers of users in an embodiment of the present invention;

[0056] Figure 6 This is a diagram of the computational efficiency of a single user under different vector sizes according to an embodiment of the present invention; DETAILED DESCRIPTION

[0057] In order to make the purpose, technical solution and advantages of the embodiments of the present application clearer, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, rather than all the embodiments. The components of the embodiments of the present application described and shown in the drawings here can be arranged and designed in various different configurations.

[0058] Therefore, the following detailed description of the embodiments of the present application provided in the accompanying drawings is not intended to limit the scope of the present application for which protection is sought, but merely represents selected embodiments of the present application. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in the field without creative work are within the scope of protection of the present application.

[0059] It should be noted that similar reference numerals and letters represent similar items in the following drawings. Therefore, once an item is defined in one drawing, it does not need to be further defined and explained in the subsequent drawings. The symbol interpretation is defined as follows:

[0060]

[0061]

[0062] The federated learning model is a special distributed machine learning model that can protect user data privacy, solve the problem of data silos, and promote data sharing and cooperation across units and fields. Federated learning allows multiple users to jointly train a global model in a collaborative manner, coordinated by a central server, without the need to centralize the data for training. The initial local model is published by the server, and each user trains the current local model on his or her own local data set to ensure that the original data does not leave the local area, and sends the updated parameters of the local model to the server to update the global model.

[0063] A method for preserving privacy in federated learning includes the following steps:

[0064] Step 1: The user uses a multi-secret reusable secret sharing method to generate multiple secret shares from a secret, and sends the multiple secret shares to other users respectively.

[0065] Step 2: Each user uses the secret as a mask to encrypt their own local model parameters, and sends the local model parameter ciphertext to the cloud server. The cloud server aggregates the local model parameter ciphertext to obtain the model parameter aggregate ciphertext.

[0066] Step 3: The user determines the secret share combination of the current online user and sends the secret share combination to the cloud server. The cloud server reconstructs all received secret share combinations using the Lagrange interpolation method and aggregates the ciphertext with the model parameters to obtain the global model parameters.

[0067] Step 4: The cloud server sends the global model parameters to the current online user and trains and updates the user's local model.

[0068] Step 5: Repeat steps 2-4, iteratively train the local model of each user, and aggregate to obtain the global model under privacy protection until the global model converges.

[0069] Example 1

[0070] See also Figure 2-3 , a privacy protection method for federated learning, comprising the following steps:

[0071] Step 1: Initialize the federated learning model.

[0072] The server is the participant and user collections Set the number of users and the online user threshold t (t<n). Let p be a large prime number, q be a large prime number that satisfies Where β is a positive integer, is the Euler function of q. is a cyclic group, and g is Generator of upper order p. Construct a t-1 degree polynomial The coefficient a0 is the secret that needs to be shared, and the other coefficients 0≤j≤t-1. Denote the hash function SHA-256 as H, T S Represents a time interval. There is a trusted channel between each user and the server.

[0073] Step 2: User u i Calculate the secret s according to the secret share distribution function i The secret share And the secret share is sent to all users.

[0074] It should be noted that the number of secret shares depends on the number of users, that is, the number of secret shares is calculated according to the number of users, and the secret shares are sent to the corresponding users.

[0075] Secret sharing is completed in the initialization phase. Each user u i From the cyclic group The secret s is randomly generated on i , and then use the multi-secret reusable secret sharing scheme to distribute the secret shares through the secret share function Calculate n secret shares and Send to corresponding users

[0076]

[0077] in, is a user set, representing all users participating in federated learning. This user set is constructed by the cloud server.

[0078] The secret share is calculated as follows:

[0079] User i The secret to be shared will be selected Let a0 = s i ,according to Calculating secret shares j∈{1,2,…,n}, and the secret share Send to the corresponding user u j .

[0080] The secret share distribution function is:

[0081] Step 3: The user uses the secret as a mask to encrypt their own local model parameters and sends the local model parameter ciphertext to the cloud server. The cloud server aggregates the local model parameter ciphertext to obtain the model parameter aggregate ciphertext. The details are as follows:

[0082] S3.1. Each user u i Share your secrets i As a mask, the mask is used to encrypt each local model parameter to obtain the corresponding local model parameter ciphertext Each user encrypts the local model parameter c i Send to the server.

[0083] The calculation method of the local model parameter ciphertext is as follows:

[0084] S3.2, the cloud server receives all the local model parameter ciphertexts c i Determine current online users when Send the online user list to each user, otherwise terminate this round of aggregation.

[0085] S3.3. The cloud server aggregates the local model parameter ciphertexts of all users to obtain the model parameter aggregate ciphertext C. The aggregation method of the local model parameter ciphertext is as follows:

[0086]

[0087] Step 4: The user calculates the secret share combination of all online users based on the online user list, and sends the secret share combination to the cloud server. The cloud server uses the Lagrange interpolation method to reconstruct all received secret share combinations, and combines the model parameter aggregation ciphertext to obtain the global model parameters. The details are as follows:

[0088] S4.1. User ui receives the online user list from the server, determines the online users based on the online user list, and calculates the secret share combination of all online users and send it to the server.

[0089] The secret share combination is calculated as follows:

[0090] User i After receiving the secret shares sent by all users uj, first press the formula The secret share combination is obtained by summing up all the received secret shares, and the secret share combination is converted into an exponential form and sent to the cloud serverless server.

[0091] The method of converting the secret share combination into exponential form is as follows:

[0092]

[0093] S4.2. The cloud server determines the current online users based on the received secret share combination. When the number of online users is less than the online user threshold, the current round of aggregation is terminated, otherwise S4.3 is executed.

[0094] S4.3. Use Lagrange interpolation method and secret reconstruction function to aggregate and reconstruct the secret share combination to obtain the secret combination

[0095] The expression of the secret share combination is as follows:

[0096]

[0097] The aggregation reconstruction method of the secret combination Y is as follows:

[0098]

[0099] The expression of the secret reconstruction function is as follows:

[0100]

[0101] S4.3. According to the secret combination and the model parameter aggregation ciphertext, the exponential form of the global model parameter is obtained, and the discrete logarithm of the exponential form of the global model parameter is calculated to obtain the global model parameter z.

[0102] The exponential form of the global model parameter is as follows:

[0103] calculate

[0104] The expression of the global model parameter z is as follows:

[0105]

[0106] Step 5: The cloud server sends the global model parameters to the current online user and trains and updates the user's local model.

[0107] Step 6: Repeat steps 3-5, and each user iteratively trains the local model until the federated learning model of each user converges to obtain the trained federated learning model.

[0108] The correctness of the above-mentioned federated learning privacy protection method, Shamir secret sharing, is based on Lagrange interpolation reconstruction. i , according to t value points (x1,f 1 (x1)+f 2(x1)+…+f n (x1)), (x2,f 1 (x2)+f 2 (x2)+…+f n (x2)),…,(x t ,f 1 (x t )+f 2 (x t )+…+f n (x t )), where x i ≠x j (i,j∈{1,2,…,t}), the following formula can be reconstructed by applying the Lagrange interpolation formula.

[0109]

[0110] Let x = 0, and the sum of all secrets can be reconstructed as:

[0111]

[0112] When reconstructing the MSRSS mechanism, what needs to be reconstructed is the combined value of multiple secrets. Instead of simply reconstructing the sum of multiple secrets. Therefore, for all users u i , according to t value points

[0113] where x i ≠x j , i,j∈{1,2,…,t}, the Lagrange interpolation formula can be used to reconstruct:

[0114]

[0115] Let x = 0, and we can reconstruct As shown in the following formula.

[0116]

[0117] In summary, the multi-secret reusable secret sharing can be reconstructed as required and meets the correctness requirements.

[0118] The following is a security analysis of the above-mentioned federated learning privacy protection method.

[0119] The main analysis is the security of the constructed scheme under the semi-honesty assumption. The participants in the scheme are mainly a server and a group of users The server is always online, but users may be offline at any time due to network reasons. In the scheme, the online users at each stage are represented as Assume that the adversary is composed of a subset of users whose number is less than a threshold t, and distinguish two situations based on whether the adversary contains the server: The adversary is mainly divided into two situations: (1) the server is honest, the users are semi-honest, and some users can collude with each other; (2) the server is semi-honest, the users are semi-honest, and some users can collude with the server.

[0120] According to the privacy security requirements, an adversary composed of any subset of users cannot obtain the model parameter information of other normal users. Specifically, given any adversary set in Other normal users The result of the model parameter aggregation should be random and uniform.

[0121] Let the random variable The privacy protection scheme of federated learning based on multi-secret reusable secret sharing under the semi-honesty assumption is actually implemented The combination of users in , let the random variable Indicates that when the constructed scheme is simulated by uniformly generating the input of honest users at random The combination of users in . Assume and is indistinguishable, then the constructed scheme is secure.

[0122] The security analysis of the constructed scheme is carried out according to the two situations: whether the adversary includes the server or not.

[0123] (1) The server is honest

[0124] For all in and There exists a probabilistic polynomial-time algorithm such that the following holds, where ≡ means the distributions are indistinguishable.

[0125]

[0126] Proof: Because the server is honest, The combination of users in and not in The user combinations in are independent and unrelated. That is, by letting all semi-honest users execute according to the original scheme, while other users generate inputs randomly, the simulator SIM can completely simulate Since only the lists of specific users are shown to semi-honest users, the simulator SIM can The inputs of the honest participants in are generated uniformly at random. Therefore, the simulated The users in are exactly the same as in REAL.

[0127] (2) The server is semi-honest

[0128] For all in and There exists a probabilistic polynomial-time algorithm such that the following holds, where ≡ means the distributions are identical.

[0129]

[0130] Proof: The security of the scheme is proved by a series of security games. If any two consecutive games are computationally indistinguishable, then the scheme is secure.

[0131] ①Game 0: In Game 0, the SIM simulates a safe game according to the original REAL scheme.

[0132] ②Game 1: In Game 1, for users Randomly pick a value to replace At this time, the enemy gathers Users in the More secret shares of The adversary has a secret share of the secret si. According to the security property of secret sharing, the adversary set cannot obtain any information about the secret si. Therefore, the distributions in Game 1 and Game 0 are indistinguishable.

[0133] ③Game 2: In Game 2, for honest users, a randomly generated value of equal length is used to replace the result of masked encryption. Since the secret si is uniformly randomly selected, Game 2 and Game 1 are indistinguishable.

[0134] ④Game 3: In Game 3, for honest users Randomly select m i ′ replace m i , where m i ' is The masked encrypted m is sent to the server. i 'Results Instead of sending Obviously, and The distribution of is the same. Therefore, Game 3 and Game 2 are indistinguishable.

[0135] In summary, the security games Game 0, Game 1, Game 2, and Game 3 are indistinguishable from each other, i.e. Therefore, the constructed scheme is secure under the semi-honesty assumption.

[0136] This application adopts multi-secret reusable secret sharing technology to solve the problem of large computation and communication volume caused by too many secret sharing times in multi-round federated learning data aggregation. Secondly, it has scalability. If a new user joins, the new user only needs to execute the multi-secret reusable secret sharing algorithm and send the secret share to other old users, while other old users do not need to re-execute the secret sharing algorithm.

[0137] The performance of the federated learning privacy protection method is evaluated below.

[0138] The influencing factors of three aspects, namely, disconnection rate, number of users and vector size, were tested and analyzed.

[0139] (1) Experimental setup

[0140] The solution is implemented in Python programming language, the operating environment is Sugon W560-G630, and the operating system is Ubuntu 20.04 with 62.6GB memory. The time in this section of the test mainly considers the computing time in the aggregation process of the federated learning model, and does not consider the time consumed in the communication process. Because MSRSS is mainly to improve the efficiency of users, it mainly tests the computing time of users in the initialization, mask encryption, aggregation decryption and other stages.

[0141] The Homoagg scheme is a typical federated learning privacy protection scheme based on random single mask and homomorphic pseudo-random function. To facilitate data comparison, the Homoagg scheme under the semi-honest assumption is run under the same experimental conditions and parameter settings.

[0142] (1) Experimental results and analysis

[0143] ① Comparison of running time under different disconnection rates

[0144] The user calculation time of a single user was tested when the disconnection rate was 0%, 5%, 10%, 15% and 20% respectively. At this time, the number of users was fixed to 500 and the vector size was fixed to 50000. The results are shown in Table 1.

[0145] Table 1 Computational efficiency of a single user under different disconnection rates (10 rounds)

[0146]

[0147] Since only the computation time is considered, the trends of the running time of the two solutions with the drop rate are as follows: Figure 4 As shown in the figure, in the initialization phase, the running time of the two schemes changes with the disconnection rate as shown in Figure 4(a) As shown. Taking the disconnection rate of 20% as an example, the running time of this scheme is about 1.553 seconds, while the running time of the Homoagg scheme is as high as 15.319 seconds. The reason for such a large gap is that in the aggregation of multiple rounds of federated learning models, the proposed scheme adopts multi-secret reusable secret sharing. Users only need to share secrets in the first round of initialization, while the Homoagg scheme needs to share secrets in each round of initialization.

[0148] In the mask encryption stage, this section's scheme also has obvious advantages, such as Figure 4 (b) As shown in the figure, the user in this case directly uses the secret generated in the initialization phase as the mask, while the Homoagg scheme needs to use the secret generated in the initialization phase as the seed and then generate the mask through a homomorphic pseudo-random generator, so the user mask encryption time is quite different.

[0149] In the aggregate decryption phase, the time spent by users in the two schemes is almost the same, such as Figure 4 (c) The reason is that in the aggregate decryption phase, the operations performed by users in the two schemes are similar, and they only need to sum up the secret shares and send them to the server.

[0150] In terms of total time, Figure 4 As shown in (d), this scheme has obvious advantages. The change of disconnection rate does not have much impact on the computing time of the user end. However, in terms of computing time, this scheme has obvious advantages over the Homoagg scheme.

[0151] ② Comparison of running time under different user scales

[0152] The running time of a single user was tested when the number of users was set to 100, 200, 300, 400, and 500 respectively. At this time, the drop rate was fixed at 20% and the vector size was fixed at 50,000. The test results are shown in Table 2.

[0153] Table 2 Computational efficiency of a single user under different numbers of users (10 rounds)

[0154]

[0155]

[0156] The trends of the running time of the two solutions as the number of users changes are as follows: Figure 5 shown.

[0157] From the perspective of initialization time, the time of this scheme is much lower than that of the Homoagg scheme, and the growth rate is small as the number of users increases, such as Figure 5(a) shows the initialization time. The initialization time is mainly the time for secret sharing. The scheme in this section adopts multi-secret reusable secret sharing. In the multi-round federated model data aggregation, secret sharing only needs to be performed in the first round.

[0158] From the perspective of mask encryption time, the mask encryption time of the two schemes remains relatively stable under different numbers of users, such as Figure 5 (b) shows that the mask encryption time of the Homoagg scheme fluctuates between 20.407 seconds and 20.617 seconds, while the mask encryption time of the scheme in this section fluctuates between 3.384 seconds and 3.407 seconds. Although there is no obvious growth trend for both, the mask encryption time of the scheme in this section is significantly shorter, showing higher encryption efficiency.

[0159] From the perspective of aggregate decryption time, the aggregate decryption time of the two solutions is very short, and increases slightly with the increase in the number of users, but the increase is very small. Figure 5 (c) This shows that the aggregate decryption process is insensitive to changes in the number of users.

[0160] As the number of users increases from 100 to 500, the total running time of both solutions increases, but the growth rates are significantly different. Figure 5 (d) shows that the total running time of the Homoagg solution increased from 22.717 seconds when the number of users was 100 to 36.194 seconds when the number of users was 500, with a growth rate of about 59%. The total running time of the solution in this section increased relatively slowly, from 3.656 seconds when the number of users was 100 to 5.187 seconds when the number of users was 500, with a growth rate of only 42%. This shows that the total running time of the solution in this section is effectively controlled when dealing with larger-scale users, and has better scalability.

[0161] From the perspective of growth trend, this scheme shows better efficiency in processing multi-round federated aggregation tasks. As the number of users increases, the total time of this scheme increases slowly, while the total time of the Homoagg scheme increases rapidly. This shows that this scheme is more suitable for user scenarios with limited computing resources.

[0162] The running time of a single user was tested when the vector size was set to 10000, 20000, 30000, 40000, and 50000 respectively. At this time, the number of users was fixed to 500 and the disconnection rate was fixed to 20%. The test results are shown in Table 3.

[0163] Table 3 Computational efficiency of a single user under different vector sizes (10 rounds)

[0164]

[0165] The trends of the running time of the two solutions as the vector size changes are as follows Figure 6 As shown in Figure 2, the initialization time of both schemes is relatively stable under different vector sizes, with no obvious growth trend, as shown in Figure 2. Figure 6 (a) This is mainly because the initialization phase is mainly secret sharing, which has little to do with the vector size.

[0166] From the perspective of mask encryption time, as the vector size increases, the mask encryption time of both schemes shows an increasing trend, such as Figure 6 (b) This is because the encryption process needs to process each element in the vector, so the processing time will increase as the vector size increases. However, the mask encryption time of this scheme increases relatively slowly, showing higher encryption efficiency.

[0167] From the perspective of aggregate decryption time, the aggregate decryption time has little relationship with the vector size. The aggregate decryption time of both schemes is very short and remains stable under different vector sizes.

[0168] As the vector size increases, the total time of both schemes shows an increasing trend, e.g. Figure 6 (d) is shown. However, the growth of this scheme is significantly smaller than that of the Homoagg scheme. For example, when the vector size increases from 10,000 to 50,000, the total time of the Homoagg scheme increases from 19.732 seconds to 36.251 seconds. The total time of this scheme increases from 2.432 seconds to 5.207 seconds, with a smaller absolute growth and a lower initial cardinality. This shows that when processing larger-scale vector data, the growth of the total time of this scheme is effectively controlled, showing better scalability and efficiency.

[0169] In practical applications, when large-scale vector data needs to be processed, the scheme in this section can be used for federated learning. At the same time, the algorithm implementation and parallel processing strategy of the scheme in this section can be further studied and optimized to further improve its performance. In addition, other optimization methods, such as data compression and transmission optimization, can be explored to further reduce the time overhead in the federated learning process.

[0170] This federated learning privacy protection method can be applied in comprehensive investigation systems, see Figure 1 The comprehensive reconnaissance system includes 1 central server and multiple users. The users include unmanned reconnaissance aircraft users, unmanned reconnaissance vehicle users and single-soldier handheld terminals. The central server and users can communicate normally, but users cannot communicate directly with each other and need to be forwarded through the server. In order to realize collaborative intelligent reconnaissance among users, the above-mentioned federated learning privacy protection method is used to train the comprehensive reconnaissance system, realize intelligent reconnaissance among users, and ensure the security of each user's data.

[0171] Example 2

[0172] See also Figure 1 Based on the above-mentioned federated learning privacy protection method, this application also proposes a federated learning privacy protection system, which may include:

[0173] The first module is used for a user to adopt a multi-secret reusable secret sharing method to generate multiple secret shares from a secret, and send the multiple secret shares to other users respectively.

[0174] The second module is used for each user to use the secret as a mask to encrypt their respective local model parameters, and send the local model parameter ciphertext to the cloud server. The cloud server aggregates the local model parameter ciphertext to obtain the model parameter aggregated ciphertext.

[0175] The third module is used by the user to determine the secret share combination of the current online user and send the secret share combination to the cloud server. The cloud server uses the Lagrange interpolation method to reconstruct all the received secret share combinations and obtains the global model parameters by combining the model parameter aggregation ciphertext.

[0176] The fourth module is used for the cloud server to send the global model parameters to the current online user and train and update the user's local model.

[0177] The fifth module is used to repeatedly execute the second module to the fourth module, iteratively training the local model of each user until the federated learning model of each user converges.

[0178] It should be noted that in the several embodiments provided in the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of each module is only a logical function division. There may be other division methods in actual implementation. For example, multiple modules can be combined or integrated into another device, or some features can be ignored or not executed. The module described as a separate component may or may not be physically separated. The component displayed as a module may be a physical unit or multiple physical units, that is, it may be located in one place, or it may be distributed in multiple different places. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment.

[0179] In addition, each module in each embodiment of the present invention may be integrated into a processing unit, each module may exist physically separately, or two or more modules may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of a software functional unit.

[0180] An electronic device provided in an embodiment of the present application includes a memory and a processor, wherein a computer program is stored in the memory, and when the processor executes the computer program, the steps of the federated learning privacy protection method described in any of the above embodiments are implemented.

[0181] Another electronic device provided in the embodiment of the present application may also include: an input port connected to the processor, used to transmit multimodal data collected by an external acquisition device to the processor; and a display unit connected to the processor, used to display the processing results of the processor to the outside world; a communication module connected to the processor, used to realize communication between the electronic device and the outside world. The display unit can be a display panel, a laser scanning display, etc.; the communication mode adopted by the communication module includes but is not limited to mobile high-definition link technology (HML), universal serial bus (USB), high-definition multimedia interface (HDMI), wireless connection (including wireless fidelity technology (WiFi), Bluetooth communication technology, low-power Bluetooth communication technology, and communication technology based on IEEE802.11s).

[0182] An embodiment of the present application provides a computer-readable storage medium, in which a computer program is stored. When the computer program is executed by a processor, the steps of the federated learning privacy protection method described in any of the above embodiments are implemented.

[0183] The computer-readable storage medium involved in the present application includes random access memory (RAM), internal memory, read-only memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, registers, hard disks, removable disks, CD-ROMs, or any other form of storage medium known in the technical field.

[0184] For the description of the relevant parts of the federated learning privacy protection method system, electronic device, and computer-readable storage medium provided in the embodiments of this application, please refer to the detailed description of the corresponding parts of the federated learning privacy protection method provided in the embodiments of this application, which will not be repeated here. In addition, the parts of the above technical solutions provided in the embodiments of this application that are consistent with the implementation principles of the corresponding technical solutions in the prior art are not described in detail to avoid excessive elaboration.

[0185] The above contents are only for explaining the technical idea of ​​the present invention and cannot be used to limit the protection scope of the present invention. Any changes made on the basis of the technical solution in accordance with the technical idea proposed by the present invention shall fall within the protection scope of the claims of the present invention.

Claims

1. A method for protecting privacy in federated learning, characterized in that: The following steps are involved: The user adopts a multi-secret reusable secret sharing method to divide the secret into multiple secret shares and send the multiple secret shares to the corresponding users respectively; The users use their own secrets as masks to encrypt their own local model parameters, and send the local model parameter ciphertexts to the server. The server aggregates the received local model parameter ciphertexts to obtain the model parameter aggregate ciphertext. The user establishes a secret share combination based on the secret share of the current online user and sends the secret share combination to the server. The server reconstructs all the received secret share combinations to obtain the secret combination, and uses it to decrypt the model parameter aggregation ciphertext to obtain the global model parameters, and sends the global model parameters to the user; the user uses the global model parameters for local training to obtain updated local model parameters, and mask-encrypts the local model parameters again, and iteratively updates the global model until the global model converges.

2. A method for protecting privacy in federated learning according to claim 1, characterized in that: The method of using a multi-secret reusable secret sharing method to divide a secret into multiple secret shares comprises: Determine multiple secret shares of the secret according to the secret share distribution function, and send the multiple secret shares to the corresponding multiple users respectively; The secret share distribution function is as follows: in, For users The secret, For users Send to user The secret share.

3. A method for protecting privacy in federated learning according to claim 1, characterized in that: The cloud server aggregates the received local model parameter ciphertext to obtain the model parameter aggregate ciphertext including: The server determines the number of online users based on the received local model parameter ciphertext. When the number of online users is greater than the set threshold, the server aggregates the local model parameter ciphertexts of all online users. The method for ciphertext aggregation of local model parameters is as follows: in, C Aggregate ciphertext for model parameters, Indicates user The model parameter ciphertext, Indicates the online user list consisting of users who have successfully sent the ciphertext of local model parameters. H represents a hash function, Represents a time interval.

4. A method for protecting privacy in federated learning according to claim 1, characterized in that: The user uses the global model parameters for local training to obtain updated local model parameters, and mask and encrypt the local model parameters again, and iteratively update the global model until the global model converges, including the following process: The server sends the global model parameters to the current online user, and the user uses the global model parameters for local training to obtain updated local model parameters; After the local model is updated, the updated local model parameters are masked and encrypted and sent to the server. The server aggregates and decrypts to obtain the global model. The above process is repeated until the global model converges.

5. A method for protecting privacy in federated learning according to claim 1, characterized in that: The server reconstructs all received secret share combinations to obtain the secret combination as follows: The user determines the online users according to the online user list, calculates the secret share combination of all online users' secret shares and sends it to the server; The server determines the current online user based on the received secret share combination. When the number of online users is greater than a threshold, the server uses the Lagrange interpolation method combined with the secret reconstruction function to aggregate and reconstruct all secret share combinations to obtain a secret combination.

6. A method for protecting privacy in federated learning according to claim 5, characterized in that: The expression of the secret share combination is as follows: The aggregation reconstruction method of the secret combination is as follows: The expression of the secret reconstruction function is as follows: in, is the combination of secret shares received by the user from the current user list, For users The secret, H represents a hash function, Represents a time interval.

7. A method for protecting privacy in federated learning according to claim 1, characterized in that: The method for determining the global model parameters is as follows: The exponential form of the global model parameters is obtained by aggregating the ciphertext according to the secret combination and the model parameters, and the discrete logarithm of the exponential form is calculated to obtain the global model parameters.

8. A method for protecting privacy in federated learning according to claim 7, characterized in that: The global model parameters The expression is as follows: in, Indicates the online user list consisting of users who have successfully sent the ciphertext of local model parameters. Indicates user The model parameter ciphertext, Represents a cyclic group The generator on .

9. A federated learning privacy protection system, characterized in that: The following steps are involved: A secret share module, used for users to adopt a multi-secret reusable secret sharing method, divide the secret into multiple secret shares, and send the multiple secret shares to corresponding users respectively; A model parameter aggregation module is used for users to use their own secrets as masks to encrypt their own local model parameters, and send the local model parameter ciphertext to the server. The server aggregates the received local model parameter ciphertext to obtain the model parameter aggregate ciphertext; The global model training module is used for users to establish secret share combinations based on the secret shares of current online users, and send the secret share combinations to the server. The server reconstructs all received secret share combinations to obtain secret combinations, and uses them to decrypt the model parameter aggregation ciphertext to obtain global model parameters, and sends the global model parameters to the user; the user uses the global model parameters for local training to obtain updated local model parameters, and mask-encrypts the local model parameters again, and iteratively updates the global model until the global model converges.

10. An electronic device, characterized in that: include: Memory for storing computer programs; A processor, configured to implement the steps of the federated learning privacy protection method as described in any one of claims 1 to 8 when executing the computer program.

Images