Efficient and verifiable federal learning method for enhancing privacy protection

Abstract

The invention discloses an efficient and verifiable federal learning method for enhancing privacy protection. The method comprises the steps that global model parameters are initialized, a user side generates a dynamic password, the dynamic password is sent to a manager, and the manager selects a random value; training a local model by using the local data to obtain a model gradient; acquiring a gradient ciphertext according to the model gradient, the dynamic password and the random value; sending the gradient ciphertext to a server side, aggregating the gradient ciphertext, and sending an aggregation password of a corresponding online user by a manager; the server side carries out verification based on the aggregated gradient ciphertext and the aggregated password, judges whether information transmitted by the double servers is accurate or not, and obtains a first judgment result; the user side carries out verification based on the aggregation result returned by the server side, judges whether the gradient of server aggregation is accurate or not, and obtains a second judgment result; and updating global model parameters based on the first judgment result and the second judgment result.

Description

Technical Field

[0001] The present invention belongs to the technical field of federated learning, and in particular relates to an efficient and verifiable federated learning method for enhancing privacy protection. Background Art

[0002] In today's big data era, the contribution of data has become a key factor in the rapid development of various industries. Effectively centralizing relevant data is an important step in leveraging data advantages. Federated learning, as a distributed machine learning method, enables data collaboration across institutions while effectively protecting data privacy during transmission. Specifically, users do not need to aggregate raw data with other nodes, but use locally trained model parameters. Once these parameters are uploaded to the central server, the server aggregates and updates the data to obtain global model parameters, which are then returned to the collaborative users, completing an iteration of federated learning. This process ensures the privacy and security of user data while enabling all users to benefit from collaborative training. At present, federated learning has been widely used in many IoT scenarios, such as smart homes and smart transportation. In addition, federated learning performs well in areas such as loan status prediction, health assessment, and next word prediction. However, studies have shown that central servers can infer private data from the parameters uploaded by users, resulting in privacy leakage, and even analyze when and how to recover the original data through recursive gradient attack methods.

[0003] In order to solve the key problem of privacy leakage in federated learning, researchers have proposed a variety of methods. The secure aggregation scheme based on pairwise additive masking proposed by Google at CCS'17 adopts a clever mask offset strategy. Participants need to negotiate pairwise additive masks to protect their own data. In the data aggregation stage, these masks can be offset by pairwise addition without affecting the accuracy of the model, and the resulting computational overhead is small. However, the biggest problem with this method is the disconnection of participants. Once a user is offline, the final data aggregation result cannot offset the mask pairwise, resulting in deviation in data accuracy. In order to recover the data of the offline user, the Shamir secret sharing technology is usually introduced, but the additional communication overhead becomes a key issue. In order to avoid the data deviation and overhead problems caused by pairwise additive masking, researchers have innovated masking strategies in subsequent schemes, and more and more methods such as single masking, double masking and polynomial masking are adopted. The ESVFL scheme adopts the encryption method of polynomial masking and uses the Lagrange interpolation method to achieve low computational overhead encryption on the user side. In addition, the encryption computation and communication overhead are independent of the number of users, so even if there are offline users, online users will not incur additional computation and communication overhead.

[0004] While ensuring data privacy and security, users also hope to obtain accurate results through federated learning. The main factors that lead to inaccurate results include: first, the fraudulent behavior of malicious servers, not following the protocol requirements of the aggregation operation, and arbitrarily falsifying results; second, the lazy behavior of malicious servers, performing aggregation operations normally but failing to aggregate all user data; third, the disconnection of some users, affecting the recovery of the final correct results. In order to avoid the above situation, verifiability has become an important part of improving the effectiveness of federated learning. Figure 1 As shown in the figure, the verification process of the server aggregation result is mainly used to confirm whether the data belongs to itself, and no additional data is introduced. The ESVFL scheme is a privacy-preserving federated learning framework based on a dual-server architecture, which verifies the aggregation results returned by the server through cross-validation and comparative verification. However, this verification method assumes that there is no collusion between the two servers. Although it is efficient, it cannot prevent the two servers from jointly shirking responsibility or jointly forging aggregation results. Therefore, there is an urgent need for an efficient and verifiable federated learning method that enhances privacy protection. Summary of the invention

[0005] To solve the above technical problems, the present invention proposes an efficient and verifiable federated learning method with enhanced privacy protection, which can solve the problems of privacy leakage and forged aggregation results caused by collusion between two servers.

[0006] The present invention provides an efficient and verifiable federated learning method for enhancing privacy protection, comprising:

[0007] Initialize global model parameters, the user terminal generates a dynamic password, sends the dynamic password to the administrator, and the administrator selects a random value;

[0008] Use local data to train the local model and obtain the model gradient;

[0009] Obtaining gradient ciphertext according to the model gradient, dynamic password and random value;

[0010] The gradient ciphertext is sent to the server, and the gradient ciphertext is aggregated, and the administrator sends the aggregated password of the corresponding online user;

[0011] The server performs verification based on the aggregated gradient ciphertext and the aggregated password to determine whether the information transmitted between the two servers is accurate, and obtains a first determination result;

[0012] The user end verifies based on the aggregation result returned by the server end, determines whether the gradient aggregated by the server is accurate, and obtains a second determination result;

[0013] Based on the first judgment result and the second judgment result, the global model parameters are updated.

[0014] Optionally, perform model training based on local data to obtain model gradients; including:

[0015] The private data is trained on the model without leaving the local machine, and the initial model gradient is obtained.

[0016] A series of random numbers are concatenated to the initial model gradient to obtain the model gradient.

[0017] Optionally, the method for obtaining the gradient ciphertext is:

[0018] f 1i (b1) = p i b1 2 +(g' i -δ i )b1+PRG(m i )

[0019] f 2i (b2) = p i b2 2 +(g' i -δ i )b2+PRG(n i )

[0020] f 3i (b2) = -p i b2 2 -(g' i +δ i )b2+PRG(m i )

[0021] f 4i (b1) = -p i b1 2 -(g' i +δ i )b1+PRG(n i )

[0022] Among them, f 1i (b1), f 2i (b2), f 3i (b2) and f 4i (b1) is a set of ciphertexts of model gradients, p i is a dynamic password, b1 and b2 are random values ​​selected by the administrator, and g' i is the model gradient after blinding, δ i is the perturbation of the gradient, m i and n i The seed of the pseudo-random number generator PRG;

[0023] Name f respectively 1i (b1), f2i (b2), f 3i (b2) and f 4i (b1) is F 1i 、F 2i 、F 3i and F 4i .

[0024] Optionally, the gradient ciphertext is sent to a server, and the gradient ciphertext is aggregated, and the administrator sends an aggregated password of a corresponding online user; including:

[0025] F 1i and F 2i Send to server S1, and F 3i and F 4i Send to server S2;

[0026] Server S1 aggregates the received F 1i and F 2i , server S2 aggregates the received F 3i and F 4i , obtain the aggregated ciphertext;

[0027] The administrator broadcasts the online user set of the user end, and the online user confirms whether to participate in it through the online user set. If an error occurs, an objection is raised and the server end is corrected according to the objection. Otherwise, the administrator sends the aggregate password to the server end and the user end.

[0028] Optionally, the method for obtaining the aggregated ciphertext is:

[0029] F1=∑ i∈U' F 1i =Pb1 2 +(G'-δ)b1+Ran(m i )

[0030] F2=∑ i∈U' F 2i =Pb2 2 +(G'-δ)b2+Ran(n i )

[0031] F3=∑ i∈U' F 3i =-Pb2 2 - (G '+δ)b2+Ran(m i )

[0032] F4=∑ i∈U' F 4i =-Pb1 2 -(G'+δ)b1+Ran(n i)

[0033] Among them, F1 and F2 are the aggregated gradient ciphertexts calculated by server S1, F3 and F4 are the aggregated gradient ciphertexts calculated by server S2, i is a certain user, U' is the set of online users, G' is the aggregated gradient, Ran(m i )、Ran(n i ) are seeds respectively m i and n i The pseudo-random number aggregation of all users.

[0034] Optionally, the server performs verification based on the aggregated gradient ciphertext and the aggregated password to determine whether the information transmitted between the two servers is accurate, and obtains a first determination result;

[0035] Server S1 sends F2 to server S2, and server S2 sends F3 to server S1. At this point, S1 has {F1, F2, F3}, and S2 has {F2, F3, F4}.

[0036] After server S1 sends F2 to server S2, server S1 calculates:

[0037] F 12 =F1+F2+2b1b2P=P(b1+b2) 2 +(G'-δ)(b1+b2)+Ran(m i )+Ran(n i )

[0038] F 13 =F1-F3+2b1b2P=P(b1+b2) 2 +G'(b1+b2)-δ(b1-b2)

[0039] And F 12 Send to server S2;

[0040] After server S2 sends F3 to server S1, server S2 calculates:

[0041] F 24 =F2-F4+2b1b2P=P(b1+b2) 2 +G'(b1+b2)+δ(b1-b2)

[0042] F 34 =-f3-f4+2b1b2P=P(b1+b2) 2 +(G'+δ)(b1+b2)-Ran(m i )-Ran(n i )

[0043] And F 24Send to server S1;

[0044] It is determined whether the sum calculated by server S1 is equal to the sum calculated by server S2. If they are equal, the verification is passed. Otherwise, an objection is raised. Based on the objection, the two servers correct their own erroneous behaviors.

[0045] Optionally, the user terminal verifies based on the aggregation result returned by the server terminal to determine whether the gradient aggregated by the server is accurate, and obtaining the second determination result includes:

[0046] Server S1 calculates F1 and F3:

[0047] H1=F1-F3=P(b1 2 +b2 2 )+G'(b1+b2)+δ(b2-b1)

[0048] Server S2 calculates F2 and F4:

[0049] H2=F4-F2=-P(b1 2 +b2 2 )-G'(b1+b2)+δ(b2-b1)

[0050] Among them, H1 and H2 are the intermediate values ​​of the decryption process;

[0051] The calculation result is sent to the user, and all users determine whether the server aggregation result is accurate through modulus operation to obtain a second judgment result.

[0052] Optionally, the calculation result is sent to the user, and all users are judged whether the server aggregation result is accurate through modulus operation. Obtaining the second judgment result includes:

[0053] The user determines whether H1+H2 divides b2-b1 and determines Whether b1+b2 is divisible. If it is divisible, it proves that the result of server aggregation is correct. If it is not divisible, the user raises an objection, and the two servers correct their own erroneous behavior based on the objection.

[0054] Optionally, based on the first judgment result and the second judgment result, updating global model parameters;

[0055] Based on the first judgment result and the second judgment result, the user terminal locally calculates the aggregation gradient;

[0056] Eliminate the cascade random numbers in the aggregated gradient to obtain the real model gradient;

[0057] Based on the true model gradient, the global model parameters are updated in combination with the online user set and the local data learning rate.

[0058] Compared with the prior art, the present invention has the following advantages and technical effects:

[0059] The present invention proposes an efficient and verifiable federated learning method, which is based on a dual-server architecture and enhances the privacy protection function, which is a significant improvement over the ESVFL scheme. The method can solve the privacy leakage problem caused by the communication between the two servers in ESVFL at a low cost, and increase the difficulty of malicious entities to carry out exhaustive attacks on privacy to a certain extent. The method also enhances the verifiability function and can effectively detect collusion between the two servers. In addition, the present invention analyzes the privacy security, verifiability and fault tolerance of the method through six proofs. When using non-independent and identically distributed data, the present invention can maintain the accuracy of the test, enhance privacy security without increasing the cost of privacy protection, improve the verifiability function without affecting the overall efficiency, and as the data dimension increases, the efficiency advantage becomes more significant. BRIEF DESCRIPTION OF THE DRAWINGS

[0060] The drawings constituting a part of the present application are used to provide a further understanding of the present application. The illustrative embodiments and descriptions of the present application are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings:

[0061] Figure 1 is a flow chart of a verification process of server aggregation results according to an embodiment of the present invention;

[0062] Figure 2 It is an efficient and verifiable federated learning method for enhancing privacy protection according to an embodiment of the present invention;

[0063] Figure 3 is a schematic diagram of the model training loss under the MNIST data set of an embodiment of the present invention, wherein: Figure 3 (a) is a schematic diagram of the model training loss of CNN under the MNIST dataset. Figure 3 (b) is a schematic diagram of the model training loss of MLP under the MNIST dataset;

[0064] Figure 4 is a schematic diagram of the model training loss under the CIFAR-10 dataset of an embodiment of the present invention, Figure 4 (a) is a schematic diagram of the model training loss of CNN under the CIFAR-10 dataset. Figure 4 (b) is a schematic diagram of the model training loss of MLP under the CIFAR-10 dataset;

[0065] Figure 5 is a schematic diagram of comparing the accuracy of FedAvg and EVEPFL on non-IID data sets according to an embodiment of the present invention;

[0066] Figure 6 is a schematic diagram comparing the computational overhead under different numbers of users in an embodiment of the present invention;

[0067] Figure 7 is a schematic diagram comparing partial computing overhead and total overhead of an embodiment of the present invention, Figure 7 (a) is a schematic diagram of the time ratio of the masking process. Figure 7 (b) Schematic diagram of the time ratio of the verification process;

[0068] Figure 8 It is a schematic diagram comparing the computational overhead of gradients of different dimensions in an embodiment of the present invention;

[0069] Fig. 9 is a schematic diagram comparing the computational overhead and total overhead of adding mask, verification and aggregation in an embodiment of the present invention, Fig. 9 (a) is a schematic diagram of the time consumption change of the masking process. Fig. 9 (b) is a schematic diagram of the time consumption change of the verification process. Fig. 9 (c) is a schematic diagram showing the change in time consumption during the polymerization process;

[0070] Fig.10 Schematic diagram of the communication overhead between the user and the dual servers in different gradient dimensions according to an embodiment of the present invention, wherein: Fig.10 (a) is the communication overhead of the user under different gradient dimensions, Fig.10 (b) Communication overhead of dual services under different gradient dimensions;

[0071] Fig.11 is a comparison of the computational overhead of adding mask and verification under different dimensional gradients in the embodiment of the present invention, wherein, Fig.11 (a) shows the computational overhead of various schemes in data protection. Fig.11 (b) shows a schematic diagram of the computational cost of the verification process for each scheme. DETAILED DESCRIPTION

[0072] It should be noted that, in the absence of conflict, the embodiments and features in the embodiments of the present application can be combined with each other. The present application will be described in detail below with reference to the accompanying drawings and in combination with the embodiments.

[0073] It should be noted that the steps shown in the flowcharts of the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and that, although a logical order is shown in the flowcharts, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0074] The professional terms involved in this embodiment are described in detail below:

[0075] Verifiable Federated Learning

[0076] According to different architectures of federated learning, verifiable federated learning can be divided into two types. One is centralized, involving two verifications by the server and users; the other is decentralized, involving two verifications by miners and trainers. Currently, most federated learning solutions are based on the client-server architecture, mainly aiming at the aggregation behavior of the central server that may forge or omit data, and seeking a supervision or verification mechanism to prevent such problems from occurring.

[0077] Privacy-Preserving Federated Learning

[0078] Different from traditional distributed machine learning, federated learning pays more attention to the privacy protection of collaborative users' data. Currently, there are mainly three privacy protection methods: homomorphic encryption, differential privacy, and data masking. Among them, data masking is a lightweight method that can achieve privacy protection without affecting the data accuracy.

[0079] For the mask-based privacy protection scheme, its core lies in that the mask is mostly represented as a data perturbation achieved by addition based on the original data, providing privacy protection in the propagation stage and eliminating the perturbation through a clever algorithm in the aggregation stage. It is a lightweight technical means. At present, masks are mainly divided into single masks and double masks. Assuming there are N users in federated learning, i ∈ {1, 2, …, N}, the original data x i becomes y after mask protection i :

[0080] y i = x i + M

[0081] Here, M mostly uses a pseudorandom number generator PRG to generate a result indistinguishable from an element randomly and uniformly selected in the output space. The input seed is a randomly and uniformly valued of fixed length, ensuring the unpredictability between users.

[0082] To eliminate the mask M, two methods can be adopted. One is that in the absence of a trusted third party, users i and j reach an agreement on a certain vector m i,j , where i < j and only the two parties know this vector. When adding the mask, user i adds the vector m i,j , while user j subtracts the vector m i,j . When pairwise users comply with the requirements of this protocol, the y of user i i can be expressed as:

[0083]

[0084] When the server performs data aggregation, the calculation obtains:

[0085]

[0086] Another way is to introduce a trusted third party to collect the masks M of all users, and then send the sum of the users' masks to the server during the aggregation phase, thereby eliminating the masks and obtaining the correct aggregation result.

[0087] An attack method against privacy in ESVFL:

[0088] ESVFL is a federated learning scheme that uses masked polynomials for privacy protection. The system adopts a dual-server architecture, where users split their secret information into two parts and upload them to two servers respectively. The threat model assumes that users are honest but curious, while the dual servers are malicious. Therefore, it can be inferred that users may collude with servers to obtain other people's private information. In addition, this assumption also allows the dual servers to collude to access the privacy of other users, but prohibits them from colluding to forge aggregated results. Therefore, this assumption is seized to launch an attack.

[0089] When the user uploads the secret information to the dual servers S1 and S2, S1 gets and S2 gets and If a user colludes with either server, the server can reconstruct the polynomial mask. By combining the information obtained by the two servers, a set of equations containing four unknowns can be formed, where b1, b2, b3, and b4 are known.

[0090]

[0091] It can be seen that the dual servers can deduce the exact gradient value of the target user based on the known mask polynomial structure, thereby achieving privacy theft.

[0092] In summary, it is a challenging task to ensure the security of private data while providing efficient and verifiable functions. Table 1 compares the differences in various attributes between this embodiment and the existing solution. The results show that the solution of this embodiment can protect user privacy to the maximum extent and achieve verifiable functions at a lower cost and higher rigor.

[0093] Table 1

[0094]

[0095] P1: Whether user privacy is protected;

[0096] P2: Privacy protection methods;

[0097] P3: Whether it is a dual-server architecture;

[0098] P4: Is the validation of the aggregation gradient provided?

[0099] P5: When some users are offline, no additional cost is required to restore the offline user information;

[0100] P6: Whether privacy and verifiability accept collusion between two servers;

[0101] System Model and Threat Model:

[0102] Before introducing the overall system, this embodiment lists the main symbols involved in the system design, as shown in Table 2.

[0103] Table 2

[0104]

[0105] This embodiment is described in detail below: This embodiment proposes an efficient and verifiable federated learning method (EVEPFL) with enhanced privacy protection, such as Figure 2 As shown, the specific steps include:

[0106] Initialize global model parameters, the user generates a dynamic password, sends the dynamic password to the administrator, and the administrator selects a random value;

[0107] Use local data to train the local model and obtain the model gradient;

[0108] Obtain gradient ciphertext according to model gradient, dynamic password and random value;

[0109] The gradient ciphertext is sent to the server, and the gradient ciphertext is aggregated, and the administrator sends the aggregated password of the corresponding online user;

[0110] The server performs verification based on the aggregated gradient ciphertext and the aggregated password to determine whether the information transmitted between the two servers is accurate, and obtains a first determination result;

[0111] The user end verifies based on the aggregation result returned by the server end, determines whether the gradient aggregated by the server is accurate, and obtains a second determination result;

[0112] Based on the first judgment result and the second judgment result, the global model parameters are updated.

[0113] Specifically, the system consists of three entities: an honest manager, two servers S1 and S2, and N users U i , i∈{1,2,…,N}. The functions of each entity are as follows:

[0114] Honest manager: responsible for the system initialization and auxiliary work in the aggregation phase. Before the first round of training, two points b1 and b2 are set for the user as the polynomial substitution points when generating the mask. And provide the aggregate password P of the online user when the user calculates the global gradient.

[0115] Server: Responsible for aggregating user gradients and mutual verification. The two obtain different information about user gradients for aggregation and verify each other by exchanging partial aggregation results. After verification, the intermediate information H1 and H2 are calculated and sent to the user and the online user set U' is broadcast.

[0116] User: Responsible for generating mask gradients and verifying the calculation of global gradients. Train the model locally to obtain the local model gradient g i , generate dynamic password p i And send it to the manager, add a mask to the gradient to generate four mask gradients, divide the information into two groups, send them to two servers respectively, and confirm whether they have successfully participated through the online user set broadcast by the server. Finally, receive the intermediate information H1 and H2 sent by the two servers and the aggregate password P sent by the manager, first verify the intermediate information, calculate the aggregate gradient after verification, and use the global model gradient for the next round of training.

[0117] More specifically, there are N users participating in the training in the system, U = {U1, U2, U3…U N}, the honest manager selects two random values ​​b1 and b2 and broadcasts them to all users and servers, and initializes the global model parameter W. The user locally sets three random numbers m i 、n i and δ i , where m i and n i As the seed of the pseudo-random number generator PRG, δ i As a perturbation of the gradient. In addition, each user generates a one-time password p i , and send it to the manager before uploading the gradient. All users negotiate four polynomials f1(x), f2(x), f3(x) and f4(x):

[0118] f1(x)=p i x 2 +(g' i -δ i )x+PRG(m i )

[0119] f2(x)=p i x 2 +(g' i -δ i )x+PRG(n i )

[0120] f3(x)=-p i x 2 -(g' i +δ i )x+PRG(m i )

[0121] f4(x)=-p i x 2 -(g' i +δ i )x+PRG(n i )

[0122] where g' i is the local model gradient after the cascaded random numbers.

[0123] Model training based on local data and obtaining model gradients include:

[0124] The private data is trained on the model without leaving the local machine, and the initial model gradient is obtained.

[0125] A series of random numbers are concatenated to the initial model gradient to obtain the model gradient, wherein the model gradient is the blinded model gradient.

[0126] Specifically, user U i Train the local data to minimize the loss function and obtain the model gradient g i , and give the gradient cascade a string of random numbers to get g' i , the length of the random number is fixed and consistent among all users. i Take b1 and b2 as input x and substitute them into the four agreed polynomials respectively:

[0127] f 1i (b1) = p i b1 2 +(g' i -δ i )b1+PRG(m i )

[0128] f 2i (v2) = p i b2 2 +(g' i -δ i )b2+PRG(n i )

[0129] f 3i (b2) = -p i b2 2 -(g' i +δi )b2+PRG(m i )

[0130] f 4i (b1) = -p i b1 2 -(g' i +δ i )b1+PRG(n i )

[0131] Among them, f 1i (b1), f 2i (b2), f 3i (b2) and f 4i (b1) is a set of ciphertexts of model gradients, p i is a dynamic password, b1 and b2 are random values ​​selected by the administrator, and g' i is the model gradient after blinding, δ i is the perturbation of the gradient, m i and n i The seed of the pseudo-random number generator PRG;

[0132] Name f respectively 1i (b1), f 2i (b2), f 3i (b2) and f 4i (b1) is F 1i 、F 2i 、F 3i and F 4i .

[0133] Furthermore, the gradient ciphertext is sent to the server, and the gradient ciphertext is aggregated. The administrator sends the aggregated password of the corresponding online user, including:

[0134] F 1i and F 2i Send to server S1, and F 3i and F 4i Send to server S2;

[0135] Server S1 aggregates the received F 1i and F 2i , server S2 aggregates the received F 3i and F 4i , obtain the aggregated ciphertext;

[0136] The administrator broadcasts the online user set of the user side. The online user confirms whether to participate in it through the online user set. If an error occurs, an objection is raised and the server side is corrected according to the objection. Otherwise, the administrator sends an aggregate password to the server and the user side.

[0137] Furthermore, the method for obtaining the aggregated ciphertext is:

[0138] F1=∑ i∈U' F 1i =Pb1 2 +(G'-δ)b1+Ran(m i )

[0139] F2=∑ i∈U' F 2i =Pb2 2 +(G'-δ)b2+Ran(n i )

[0140] F3=∑ i∈U' F 3i =-Pb2 2 -(G'+δ)b2+Ran(m i )

[0141] F4=∑ i∈U' F 4i =-Pb1 2 -(G'+δ)b1+Ran(n i )

[0142] Among them, F1 and F2 are the aggregated gradient ciphertexts calculated by server S1, F3 and F4 are the aggregated gradient ciphertexts calculated by server S2, i is a certain user, U' is the set of online users, G' is the aggregated gradient, Ran(m i )、Ran(n i ) are seeds respectively m i and n i The pseudo-random number aggregation of all users.

[0143] Further, the server performs verification based on the aggregated gradient ciphertext and the aggregated password to determine whether the information transmitted between the two servers is accurate, and obtaining the first determination result includes:

[0144] Server S1 sends F2 to server S2, and server S2 sends F3 to server S1. At this point, S1 has {F1, F2, F3}, and S2 has {F2, F3, F4}.

[0145] After server S1 sends F2 to server S2, server S1 calculates:

[0146] F 12 =F1+F2+2b1b2P=P(b1+b2) 2 +(G'-δ)(b1+b2)+Ran(m i )+Ran(n i )

[0147] F13 =F1-F3+2b1b2P=P(b1+b2) 2 +G'(b1+b2)-δ(b1-b2)

[0148] And F 12 Send to server S2;

[0149] After server S2 sends F3 to server S1, server S2 calculates:

[0150] F 24 =F2-F4+2b1b2P=P(b1+b2) 2 +G'(b1+b2)+δ(b1-b2)

[0151] F 34 =-F3-F4+2b1b2P=P(b1+b2) 2 +(G'+δ)(b1+b2)-Ran(m i )-Ran(n i )

[0152] And F 24 Send to server S1;

[0153] Determine whether the sum calculated by server S1 is equal to the sum calculated by server S2. If they are equal, the verification passes. Otherwise, an objection is raised. Based on the objection, the two servers correct their own erroneous behaviors.

[0154] Furthermore, the user end verifies based on the aggregation result returned by the server end to determine whether the gradient aggregated by the server is accurate, and obtaining the second judgment result includes:

[0155] Server S1 calculates F1 and F3:

[0156] H1=F1-F3=P(b1 2 +b2 2 )+G'(b1+b2)+δ(b2-b1)

[0157] Server S2 calculates F2 and F4:

[0158] H2=F4-F2=-P(b1 2 +b2 2 )-G'(b1+b2)+δ(b2-b1)

[0159] Among them, H1 and H2 are the intermediate values ​​of the decryption process;

[0160] The calculation result is sent to the user, and all users determine whether the server aggregation result is accurate through modulus operation to obtain a second judgment result.

[0161] Further, the calculation result is sent to the user, and it is determined whether the server mask aggregation is accurate. Obtaining the second determination result includes:

[0162] The user determines whether H1+H2 divides b2-b1 and determines Whether b1+b2 is divisible. If it is divisible, it proves that the result of server aggregation is correct. If it is not divisible, the user raises an objection, and the two servers correct their own erroneous behavior based on the objection.

[0163] Further, based on the first judgment result and the second judgment result, updating the global model parameters;

[0164] Based on the first judgment result and the second judgment result, the user terminal locally calculates the aggregation gradient;

[0165] Remove the cascade random numbers in the aggregated gradient to obtain the true model gradient;

[0166] Based on the true model gradient, the global model parameters are updated by combining the online user set and local data learning rate.

[0167] The following is a detailed analysis of this embodiment in terms of privacy security, verifiability, and fault tolerance:

[0168] 1. Privacy and security

[0169] Definition 1 (Perfect Confidentiality). Let ε = (E, D) be the C) Under the definition of Shannon ciphertext, consider a probability experiment in which the random variable κ is uniformly distributed on K. If for all And for all c∈C, this embodiment has:

[0170] Pr[E(κ,m0)=c]=Pr[E(κ,m1)=c],

[0171] Therefore, in this embodiment, ε can be said to be a perfectly confidential Shannon ciphertext.

[0172] Theorem 1. A malicious server cannot obtain privacy through uploaded mask gradients.

[0173] Proof: Its security is based on Shannon's perfect confidentiality. In the mask structure, the one-time password p i , PRG(seed) and perturbation δ i are all random and uniform, Pr[E(p i ,PRG(seed),δ i ,g' i )=c]=Pr[E(p i ,PRG(seed),δ i,a)=c]. a is the same as g' i Random vectors with the same dimension. From the server's perspective, the received mask gradients are four sets of random vectors, which do not pose a threat to privacy when no other information is obtained.

[0174] Although in theory this embodiment increases the difficulty for attackers to obtain privacy, it is still relatively easy to crack and obtain private data through an exhaustive attack for random numbers with simple definitions and small value ranges. Suppose a curious user wants to obtain the gradients of other users, and by colluding with the server, he obtains four masked gradients after adding masks. Because the curious user knows the structure of the masked gradients, under the condition that there are five unknowns, he only needs to crack any one of the unknowns to get the gradient. It is worth noting that in this scheme, in addition to the gradient of the user's local training that cannot be determined, the other four random number variables must be changed in each round of iteration, so the attacker cannot get any help from the information of two rounds of iterations. When the attacker wants to crack the one-time password p i , in p i When exhaustively enumerating within the value range, it is necessary to calculate whether the quaternary linear equation system has a solution and whether the obtained solution is within the value range. Once both are satisfied, it proves that the cracking is successful. In order to prevent attackers from cracking easily, this embodiment allows the user to cascade a string of random numbers to the gradient to cover up the true value of the gradient, thereby resisting exhaustive attacks to a certain extent.

[0175] Theorem 2. When curious users collude with malicious servers, they cannot obtain the privacy of other users.

[0176] Proof: If an attacker wants to obtain privacy through a brute force attack, he must first collude with both servers at the same time, and then crack any random number variable. Assuming that the gradient is a w-dimensional vector, this embodiment analyzes four random number variables respectively:

[0177] One-time password i : The attacker needs to exhaustively enumerate w-dimensional SHA-1-based OTPs. Each item of its output is 160 bits long, with a large value range, and the cost of cracking is high.

[0178] Perturbation δ i : The attacker needs to exhaustively enumerate the w dimension Random perturbations within the range are difficult to enumerate because R is taken from a large integer.

[0179] Pseudo-random number PRG (seed): Since the seed is hidden from the attacker and the output value of PRG is indistinguishable from the value randomly sampled in the output space, The output space is difficult to enumerate.

[0180] Cascade gradient g' i : Depending on the data intersection between users, federated learning is divided into horizontal federated learning, vertical federated learning, and federated transfer learning. Regardless of the type, the gradient generated when training the model is unpredictable. Even if curious users narrow the exhaustive range by referring to the local gradient range, the uncertainty of the cascaded random number can still ensure the security of the gradient.

[0181] Consider another extreme case. When N-2 users collude and obtain the mask gradients of two attacked users from the dual servers, after receiving the aggregate password P of the online user set broadcast by the honest administrator, even if the aggregate password of N-2 users is subtracted, it is still impossible to easily obtain the privacy of any user. Therefore, this scheme can effectively resist the collusion between N-2 users and the server.

[0182] 2. Verifiability

[0183] This scheme includes two layers of verification process. One layer is the cross-verification between the two servers, which is used to discover the malicious behavior of a single server, such as when the two servers exchange information, the malicious server deliberately forges information for transmission. The other layer is the user's verification of the two servers, which is used to discover the lazy behavior of a single server or the collusion of the two servers, such as the malicious server may not aggregate the mask gradients of all online users in the aggregation stage, or the two servers jointly forge intermediate information and aggregation results.

[0184] Theorem 3. When a single server forges F 12 and F 13 (F 24 and F 34 ) or one of them, the cross-validation process on both servers can be discovered.

[0185] Proof: Before cross-validation, S1 sends F2 to S2, S2 sends F3 to S1, and S1 calculates F locally. 12 and F 13 , and F 12 Sent to S2, S2 calculates F locally 24 and F 34 , and F 24 Suppose S1 forges F 12 for Sent to S2, S1 calculates F normally 13 +F 24 , and S2 at this time is calculated as When comparing the two:

[0186] F 13 +F 24 =P(b1+b2)2 +G'(b1+b2)-δ(b1-b2)+P(b1+b2) 2 +G'(b1+b2)+δ(b1-b2)=2P(b1+b2) 2 +2G'(b1+b2)

[0187]

[0188] The results are inconsistent and therefore cannot be verified. Suppose S1 forges F 12 and F 13 for and Since I don't know F 24 and F 34 , so it cannot be easily constructed The results are verified.

[0189] Theorem 4. When a single server is lazy or forges H1 and H2, it can be discovered during the user verification phase.

[0190] Proof: Server S1 receives F from user 1i and F 2i , server S2 receives F from user 3i and F 4i , assuming that S1 is lazy, 1i The aggregation result is The rest of the aggregation process is correct. To prevent this behavior from being discovered, S1 broadcasts the correct set of online users. In the dual-server cross-validation phase, S2 sends F3 to S1, and S1 performs the following calculations locally:

[0191]

[0192] and will S1 sends F2 to S2, and S2 calculates locally:

[0193] F 24 =F2-F4+2b1b2P=P(b1+b2) 2 +G'(b1+b2)+δ(b1-b2)

[0194] F 34 =-F3-F4+2b1b2P=P(b1+b2) 2 +(G'+δ)(b1+b2)-Ran(m i )-Ran(n i )

[0195] And F 24 Sent to S1. When S1 calculates:

[0196]

[0197] S2 calculation:

[0198]

[0199] Since both have parts that are changed, Laziness cannot be detected during this verification phase, but during the user verification phase, S1 calculates:

[0200]

[0201] S2 calculation:

[0202]

[0203]

[0204] When the user receives the After H2, the following calculations are performed:

[0205]

[0206] This can prove There are incorrect aggregation results in H2 or malicious servers forged and H2.

[0207] Theorem 5. When two servers negotiate to be lazy or jointly forge H1 and H2, it can be discovered during the user verification phase.

[0208] Proof: Assume that the two servers negotiate the aggregation set U * The mask gradient of the inner user, U' is the set of online users. At this time, S1 has 1i and F 2i The aggregation result is and S2 vs F 3i and F 4i The aggregation result is and The online user set broadcast by both servers is U'. Due to the collusion between the two servers, the cross-validation phase is passed by default. In the user verification phase, S1 calculates:

[0209]

[0210] S2 calculation:

[0211]

[0212] When the user receives the and Then, perform the following calculation:

[0213]

[0214]

[0215] So far, the user has not been able to detect the lazy behavior, so he continues to verify. therefore:

[0216]

[0217] Thus, it is concluded and The number of mask gradients aggregated in the online user set is different from the number of users in the online user set. The two servers negotiated lazily or forged and

[0218] 3. Fault Tolerance

[0219] In federated learning, fault tolerance mainly involves the impact of user disconnection on the overall training process. In the past, many privacy protection schemes in federated learning adopted the Shamir secret sharing method to reconstruct the decrypted information of disconnected users, thereby avoiding the impact on the final privacy data recovery. Although this method can tolerate the disconnection behavior of any user, the additional communication and computing overhead is obvious. The method of this embodiment allows any user to join and exit. During the training process, the exit of users at any stage will not affect the generation of the final global parameters. When new users want to join the training, they only need to follow the protocol requirements and provide a one-time password to the honest manager before the start of a new round of iterations.

[0220] Theorem 6. User disconnection at any stage will not affect the final recovery process of private information.

[0221] Proof: This embodiment analyzes the timing of user disconnection from three perspectives:

[0222] When a user is offline after a round of iteration ends and before a new round of iteration begins, the honest administrator is not provided with a new round of one-time password p i , nor does it upload mask gradients to the server, so it does not involve the recovery of private information.

[0223] When a user goes offline before uploading the mask gradient to the server, since the user's mask gradient does not participate in the aggregation process, the online user set U' broadcast by the server does not include the user, so the honest manager will not aggregate the user's one-time password p i, and will not affect the recovery of other users' private information.

[0224] When a user is offline after uploading the mask gradient to the server, since the user has provided the honest administrator with the one-time password p for this iteration, i , the dual servers aggregate the mask gradient of the user, and the broadcasted online user set U' also includes the user, so the aggregate password P contains the user's information. In the subsequent stages, users have no other online requirements except for verifying the dual servers to calculate the correct aggregate gradient. Even if some users are offline, as long as the online users know b1, b2, the aggregate password P and the number of broadcast U', it will not affect the acquisition of the global gradient of this round.

[0225] The present embodiment is evaluated by experiments as follows:

[0226] 1. Set up the dataset:

[0227] In terms of obtaining training loss and test accuracy, this embodiment selects two data sets, MNIST and CIFAR-10, and in order to adapt to the actual scenario of federated learning, the data are selected to be non-independent and identically distributed.

[0228] MNIST is a dataset of handwritten digits from 0 to 9. The dataset contains 60,000 training samples and 10,000 test samples. Each sample is a grayscale image of 28×28 pixels.

[0229] CIFAR-10 is a color image dataset in RGB format containing 100 objects. The dataset contains 50,000 training samples and 10,000 test samples, and the size of each sample is 32×32 pixels.

[0230] Model and parameter settings:

[0231] This embodiment selects two models: multi-layer perceptron (MLP) and convolutional neural network (CNN). The dimension of the MLP hidden layer is set to 200, and the ReLU activation function is used. Before training with the CNN model, different data sets will undergo different data preprocessing processes. The default number of users is 100, the local training round is 5, the local batch size is 10, the test batch size is 128, the learning rate is 0.01, and the stochastic gradient descent (SGD) momentum is 0.5.

[0232] Experimental environment:

[0233] This experiment was conducted on a 64-bit Windows 11 Professional operating system, using a 12th-generation Intel(R) Core(TM) i5-12600KF 3.70GHz processor, 32GB of memory, and an NVIDIA GeForce RTX3060Ti graphics card. The code was run in Python 3.11 and developed using PyCharm 2022.1.1.

[0234] 2. Model training loss and test accuracy

[0235] Since the method of this embodiment is based on the FedAvg algorithm framework, this embodiment compares the algorithm with EVEPFL in terms of training loss and test accuracy.

[0236] First, the classic FedAvg algorithm does not involve the data encryption process, but focuses on data aggregation. The algorithm of this embodiment adds protection to the data, which is equivalent to aggregating the encrypted data. From the perspective of training loss, in the presence of fluctuations, the difference in loss values ​​between the two is not significant. This shows that EVEPFL's data protection operation does not increase the loss value, nor does it affect the efficiency of model convergence. In addition, since the user's data has the characteristic of non-independent and identically distributed, in theory the convergence speed of the model will be lower than that of independent and identically distributed data. Figure 3 (a)-(b) and Figure 4 As shown in (a)-(b), with the increase of training rounds, the training loss of the model gradually converges and stabilizes at a level of about 100. Finally, by comparing the experimental results of the two models, the training effect of CNN is better than that of MLP in terms of both fluctuation range and convergence speed.

[0237] In the training of the test set, this embodiment compares the accuracy in four different scenarios. Figure 5 As shown in the figure, due to the use of non-IID data, the accuracy has decreased to varying degrees. Among them, the grayscale images of the MNIST dataset performed better in the test than the color RGB images of the CIFAR-10 dataset. In addition, by comparing with FedAvg, this embodiment found that the test accuracy of EVEPFL did not change significantly. In other words, EVEPFL maintains good test accuracy while enhancing data security.

[0238] 3. Complexity Analysis

[0239] This embodiment analyzes the computational complexity and communication complexity of the entities in the system, and lists the complexity of users and servers in EVEPFL and the previous three schemes in Table 3. In the following analysis, this embodiment assumes that there are N users and they are all online, the gradient size is V, and in the scheme with auxiliary nodes, the number of auxiliary nodes is set to M, assuming that the cost in the initialization process can be ignored.

[0240] From a macro perspective, all four schemes adopt a mask privacy protection method and realize the correctness verification of the aggregation results. Except for the classic Verifynet, no third-party entity is involved in the aggregation process. The other three schemes all introduce third-party entities, among which RVFL uses auxiliary nodes. In addition, the scheme of this embodiment and RVFL are both dual-server architectures. RVFL sets up an aggregation server and a verification server respectively, while the scheme of this embodiment performs aggregation and verification functions on each server at the same time. From the perspective of complexity comparison with RVFL, this does not increase the complexity of the scheme of this embodiment. Whether on the user side or the server side, EVEPFL has the lowest complexity. In terms of the user's computational and communication complexity, EVEPFL is not limited by the number of third-party entities and users, but is only determined by the gradient size, which undoubtedly helps to improve the user's trust and participation. For the server, the computational and communication complexity of EVEPFL is also not limited by third-party entities, but only by the number of users. Compared with the other three schemes, its complexity has not increased, and it has advantages in communication. In view of the characteristics of high communication costs in federated learning, EVEPFL can better adapt to efficient needs. Next, this embodiment will introduce the specific complexity of the solution.

[0241] Table 3(a) Users

[0242]

[0243] (b) Server

[0244]

[0245] 3.1 User Performance Analysis

[0246] Computational overhead: The user needs to calculate the one-time password p for this round i , and bring b1 and b2 into the four mask polynomials to obtain the mask gradient, where the size of the gradient is V. In the verification phase, the correctness of the aggregation result is verified by calculating three discriminants. Therefore, the computational complexity of the user is O(V).

[0247] Communication overhead: The user's communication overhead includes two types. One is to transmit the one-time password of size V in this round to the administrator, and the other is to upload four mask gradients of size V to the server. Therefore, the user's communication complexity is O(V).

[0248] 3.2 Server Performance Analysis

[0249] Computational overhead: Both servers in the scheme are aggregation servers. First, they need to aggregate the mask gradients of all online users, with an overhead complexity of Ο(NV). In the cross-validation phase, they verify the accuracy of each other's information through calculations, with an overhead complexity of Ο(V). Finally, they calculate H1 and H2 for user verification, with an overhead complexity of Ο(V). Based on the above analysis, the computational complexity of the server is Ο(NV+V).

[0250] Communication overhead: The communication volume of the two servers includes three aspects. After completing the aggregation, the two servers need to broadcast the online user set U', and the complexity of the overhead is O(N). In addition, to achieve cross-validation, both parties need to exchange one of their own aggregation results, and the complexity of the overhead is O(V). Finally, verification information H1 and H2 are sent to the user respectively, and the complexity of the overhead is O(V). Therefore, the communication complexity of the server is O(N+V).

[0251] 3.3 Manager Performance Analysis

[0252] Computational overhead: The administrator only needs to aggregate the one-time passwords sent by this part of users after receiving the online user set broadcast by the two servers. Therefore, the computational complexity of the administrator is O(NV).

[0253] Communication overhead: The only communication of the manager is the broadcast of P used to recover the final aggregate gradient, so the communication complexity of the manager is O(V).

[0254] 3.4 Performance Analysis

[0255] In this embodiment, the running time of the masking, verification and aggregation processes in the proposed scheme is compared and analyzed. The user's gradient dimension is set to 1000. Figure 6 As shown in Figure 2, due to the lightweight nature of the masking process, its time consumption is very low. The verification process mainly involves communication and calculation between two servers, as well as local calculation on the user side, so the time consumption is not affected by the number of users. However, the aggregation process is mainly affected by the number of users and the gradient dimension, so as the number of users increases, the time consumption will also increase accordingly. Overall, the time overhead of adding a mask for a single user accounts for approximately 3.1% to 23.5% of the average aggregation time overhead.

[0256] Figure 7(a)-(b) show the proportion of time taken by the mask and verification process as the number of users changes in one round of iterative operation. The time cost required to add the mask accounts for about 8.3% to 10.9% of the total time, while the time cost required for verification accounts for about 15.4% to 17.3% of the total time. Therefore, in one round of iterative training, the reduction in the number of users due to user churn does not affect the implementation of privacy protection in federated learning and the verification of the correctness of the results, but instead promotes the improvement of overall operational efficiency.

[0257] In addition, this embodiment sets the number of users to 500, and observes the changes in the running time of each stage by changing the gradient dimension of each user. Figure 8 and Fig. 9 It can be concluded from (a)-(c) that the time consumption of adding mask, verification and aggregation process increases to varying degrees with the increase of gradient dimension. In contrast, the time change of aggregation process is the most significant, and it accounts for the largest proportion of the overall running time, while the time change of verification process is the most stable. Figure 6 and Figure 8 ,It can be seen that the changes in the number of users and ,gradient dimension have little impact on the verification process of the ,scheme, indicating that the performance of the verification function in ,adapting to different conditions is stable.

[0258] The user's communication overhead mainly includes providing a one-time password to the administrator and uploading gradients to the dual servers, while the dual server's communication overhead includes cross-verification information transmission, broadcasting the online user set, and returning the aggregated results to the user. Fig.10 As shown in (a)-(b), the communication overhead between the user and the dual servers is proportional to the dimension of the gradient, because their main communication overhead contains parameters with the same dimension as the gradient.

[0259] In order to better analyze the effectiveness of the solution, this embodiment chooses to compare RVFL, NIVP-DS and ESVFL, all of which use the same dual-server architecture, and compares them with the solution of this embodiment. This embodiment controls the number of users in all solutions to 500 and sets 50 auxiliary nodes in RVFL. Fig.11 (a) shows the computational overhead of various schemes in data protection. Compared with RVFL, the computational overhead of EVEPFL increased by 93.28% on average. Similarly, ESVFL using the mask method and the scheme of this embodiment are comparable in computational overhead and are the lowest among the four schemes, indicating that the cost of implementing data protection is minimal. In addition, the scheme of this embodiment adds more random numbers to solve the privacy leakage problem caused by dual-server communication in ESVFL, but does not incur significant computational costs, thereby achieving the expected experimental results. Fig.11(b) shows the computational cost of the verification process of each scheme. RVFL measures the interests of each entity from the perspective of game theory and reduces the possibility of collusion between two servers. However, the verification mechanism itself cannot resist the collusion between the two servers, and due to the participation of auxiliary nodes, the computational cost of the verification process is high. On this basis, EVEPFL achieved an average improvement of 92.98% compared with RVFL. NIVP-DS effectively resists the malicious behavior of a single server by using a digital envelope method, which is similar to the scheme of this embodiment in terms of computational cost. However, as the gradient dimension increases, the stability of EVEPFL in the verification process becomes more prominent. In addition, the verification mechanism of this embodiment not only resists the malicious behavior of a single server, but also prevents collusion between two servers. Therefore, with a slight increase in computational cost, the functionality of this embodiment is better than NIVP-DS and ESVFL.

[0260] The above are only preferred specific implementations of the present application, but the protection scope of the present application is not limited thereto. Any changes or substitutions that can be easily thought of by a person skilled in the art within the technical scope disclosed in the present application should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

Claims

1. An efficient and verifiable federated learning method with enhanced privacy protection, characterized in that: include: Initialize global model parameters, the user terminal generates a dynamic password, sends the dynamic password to the administrator, and the administrator selects a random value; Use local data to train the local model and obtain the model gradient; Obtaining gradient ciphertext according to the model gradient, dynamic password and random value; The gradient ciphertext is sent to the server, and the gradient ciphertext is aggregated, and the administrator sends the aggregated password of the corresponding online user; The server performs verification based on the aggregated gradient ciphertext and the aggregated password to determine whether the information transmitted between the two servers is accurate, and obtains a first determination result; The user end verifies based on the aggregation result returned by the server end, determines whether the gradient aggregated by the server is accurate, and obtains a second determination result; Based on the first judgment result and the second judgment result, the global model parameters are updated.

2. An efficient and verifiable federated learning method with enhanced privacy protection according to claim 1, characterized in that: Model training based on local data and obtaining model gradients include: The private data is trained on the model without leaving the local machine, and the initial model gradient is obtained. A series of random numbers are concatenated to the initial model gradient to obtain the model gradient.

3. An efficient and verifiable federated learning method with enhanced privacy protection according to claim 1, characterized in that: The method to obtain the gradient ciphertext is: f 1i (b1)=p i b1 2 +(g′ i -δ i )b1+PRG(m i ) f 2i (b2)=p i b2 2 +(g′ i -δ i )b2+PRG(n i ) f 3i (b2)=-p i b2 2 -(g′ i +δ i )b2+PRG(m i ) f 4i (b1)=-p i b1 2 -(g′ i +δ i )b1+PRG(n i ) Among them, f 1i (b1), f 2i (b2), f 3i (b2) and f 4i (b1) is a set of ciphertexts of model gradients, p i is a dynamic password, b1 and b2 are random values ​​selected by the administrator, and g′ i is the model gradient after blinding, δ i is the perturbation of the gradient, m i and n i The seed of the pseudo-random number generator PRG; Name f respectively 1i (b1), f 2i (b2), f 3i (b2) and f 4i (b1) is F 1i 、F 2i 、F 3i and F 4i .

4. An efficient and verifiable federated learning method with enhanced privacy protection according to claim 3, characterized in that: The gradient ciphertext is sent to the server, and the gradient ciphertext is aggregated. The administrator sends the aggregated password of the corresponding online user, including: F 1i and F 2i Send to server S1, and F 3i and F 4i Send to server S2; Server S1 aggregates the received F 1i and F 2i , server S2 aggregates the received F 3i and F 4i , obtain the aggregated ciphertext; The administrator broadcasts the online user set of the user end, and the online user confirms whether to participate in it through the online user set. If an error occurs, an objection is raised and the server end is corrected according to the objection. Otherwise, the administrator sends the aggregate password to the server end and the user end.

5. An efficient and verifiable federated learning method with enhanced privacy protection according to claim 4, characterized in that: The method to obtain the aggregated ciphertext is: F1=∑ i∈U′ F 1i =Pb1 2 +(G′-δ)b1+Ran(m i ) F2=∑ i∈U′ F 2i =Pb2 2 +(G′-δ)b2+Ran(n i ) F3=∑ i∈U′ F 3i =-Pb2 2 -(G′+δ)b2+Ran(m i ) F4=∑ i∈U′ F 4i =-Pb1 2 -(G′+δ)b1+Ran(n i ) Among them, F1 and F2 are the aggregated gradient ciphertexts calculated by server S1, F3 and F4 are the aggregated gradient ciphertexts calculated by server S2, i is a certain user, U′ is the set of online users, G′ is the aggregated gradient, Ran(m i )、Ran(n i ) are seeds respectively m i The pseudo-random number aggregation of all users of and.

6. An efficient and verifiable federated learning method with enhanced privacy protection according to claim 5, characterized in that: The server performs verification based on the aggregated gradient ciphertext and the aggregated password to determine whether the information transmitted between the two servers is accurate, and obtaining the first determination result includes: Server S1 sends F2 to server S2, and server S2 sends F3 to server S1. At this point, S1 has {F1, F2, F3}, and S2 has {F2, F3, F4}. After server S1 sends F2 to server S2, server S1 calculates: F 12 =F1+F2+2b1b2P=P(b1+b2) 2 +(G′-δ)(b1+b2)+Ran(m i )+Ran(n i ) <h2 style=";text-align:left;direction:ltr">F<h2 style=";text-align:left;direction:ltr"> 13 <h2 style=";text-align:left;direction:ltr"> (F1-F3+2b1b2P) = (P(b1+b2)<h2 style=";text-align:left;direction:ltr"> 2 <h2 style=";text-align:left;direction:ltr"> +G′(b1+b2)-δ(b1-b2) And F 12 Send to server S2; After server S2 sends F3 to server S1, server S2 calculates: F 24 =F2-F4+2b1b2P=P(b1+b2) 2 +G′(b1+b2)+δ(b1-b2) F 34 =-F3-F4+2b1b2P=P(b1+b2) 2 +(G′+δ)(b1+b2)-Ran(m i )-Ran(n i ) And F 24 Send to server S1; It is determined whether the sum calculated by server S1 is equal to the sum calculated by server S2. If they are equal, the verification is passed. Otherwise, an objection is raised. Based on the objection, the two servers correct their own erroneous behaviors.

7. The efficient and verifiable federated learning method with enhanced privacy protection according to claim 5, characterized in that: The user end verifies based on the aggregation result returned by the server end, and determines whether the gradient aggregated by the server is accurate. Obtaining the second judgment result includes: Server S1 calculates F1 and F3: H1=F1-F3=P(b1 2 +b2 2 )+G′(b1+b2)+δ(b2-b1) Server S2 calculates F2 and F4: H2=F4-F2=-P(b1 2 +b2 2 )-G′(b1+b2)+δ(b2-b1) Among them, H1 and H2 are the intermediate values ​​of the decryption process; The calculation result is sent to the user, and all users determine whether the server aggregation result is accurate through modulus operation to obtain a second judgment result.

8. An efficient and verifiable federated learning method with enhanced privacy protection according to claim 7, characterized in that: The calculation result is sent to the user, and all users are judged whether the server aggregation result is accurate through modulus operation. Obtaining the second judgment result includes: The user determines whether H1+H2 divides b2-b1 and determines Whether b1+b2 is divisible. If it is divisible, it proves that the result of server aggregation is correct. If it is not divisible, the user raises an objection, and the two servers correct their own erroneous behavior based on the objection.

9. The efficient and verifiable federated learning method with enhanced privacy protection according to claim 2, characterized in that: Based on the first judgment result and the second judgment result, updating the global model parameters; Based on the first judgment result and the second judgment result, the user terminal locally calculates the aggregation gradient; Eliminate the cascade random numbers in the aggregated gradient to obtain the real model gradient; Based on the true model gradient, the global model parameters are updated in combination with the online user set and the local data learning rate.

Images