A privacy-preserving, bidirectional, verifiable federated learning method

Abstract

This invention relates to a privacy-preserving, bidirectionally verifiable federated learning method. Based on an aggregation server and various clients, combined with a trusted third-party KGC, and incorporating stable communication client analysis, federated learning is performed on the global model of the target to be trained, resulting in a trained global model. A dual-mode approach is employed to ensure the security of intermediate models. By designing parameters to implement the encrypted state of the model aggregation results, attackers cannot infer the gradient information of the client's local model based on multiple rounds of aggregation results. This effectively protects client privacy and security while resolving the client disconnection issue. Furthermore, a bidirectional verifiable approach is implemented, with the encrypted encryption and decryption functions directly handled by the client and the aggregation server, avoiding the possibility of leaking encryption security parameters. In addition, the client's model training settings eliminate the need to consider weight leakage issues, and the design utilizes non-interactive zero-knowledge proofs to ensure the verifiability of uploaded encrypted text.

Description

Technical Field

[0001] This invention relates to a privacy-preserving, bidirectional, verifiable federated learning method, belonging to the field of federated learning technology. Background Technology

[0002] With the development of big data technology, the amount of data and information generated and collected in social activities has increased dramatically, and federated learning has gradually become a popular distributed machine learning paradigm. It allows institutions to participate in model training without sharing their own raw data, alleviating data privacy issues caused by collecting different datasets and effectively mitigating the problem of data silos. However, the shared model gradients in federated learning can also leak participants' private information, making the model vulnerable to attacks, such as model inversion attacks and member inference attacks.

[0003] To address the aforementioned issues, existing technical solutions mostly focus on attacks encountered during the uploading of local model information, such as gradient leakage attacks. They employ various encryption schemes to encrypt shared gradient information to protect user privacy, such as homomorphic encryption, function encryption, and secret sharing. However, most solutions fail to consider the leakage of intermediate models during training due to multiple iterations, which could allow attackers to obtain sensitive information about a user. Furthermore, existing research in this area also has shortcomings, such as the server's inability to check uploaded information, allowing attackers to masquerade as ordinary customers and upload incorrect information to disrupt the training process. Additionally, malicious servers could forge aggregation results to influence the training performance of the local model. Summary of the Invention

[0004] The technical problem to be solved by this invention is to provide a privacy-preserving, bidirectional, verifiable federated learning method that strengthens security design and application from multiple perspectives, thereby improving security and protection during the federated learning process.

[0005] To solve the above-mentioned technical problems, the present invention adopts the following technical solution: The present invention designs a privacy-preserving bidirectional verifiable federated learning method, based on an aggregation server and various clients, combined with a trusted third party KGC, and performs federated learning on the global model of the target to be trained according to the following steps A to B, to obtain the trained global model of the target;

[0006] Step A. Initialize the public parameter pm based on the trusted third party KGC and distribute it to the aggregation server and each client. The aggregation server generates a key, and each client generates a key. Then proceed to Step B.

[0007] Step B. Based on the preset maximum number of training rounds T, each client trains the global model of the target to be trained in each training round to obtain local model parameters. Combined with the stable communication list determined by the trusted third party KGC, the local model parameters of each stable communication client are updated, and then the trained target global model is obtained in the Tth training round.

[0008] As a preferred embodiment of the present invention: step B includes steps B1 to B5;

[0009] Step B1. Initialize the number of training rounds t = 1, and proceed to step B2;

[0010] Step B2. Each client trains the global model of the target to be trained in the t-th round of training to obtain local model parameters, builds an authentication and encryption label, uploads it to the aggregation server, and then proceeds to step B3.

[0011] Step B3. Based on the communication and interaction between the aggregation server and the trusted third party KGC regarding keys, determine the stable communication list, and then proceed to step B4;

[0012] Step B4. The aggregation server performs model parameter aggregation for the authentication and encryption tags uploaded by each client in the stable communication list. When t = T, the trained target global model is obtained. When t ≠ T, proceed to step B5.

[0013] Step B5. Increment the value of t by 1, then return to step B2.

[0014] As a preferred technical solution of the present invention: step B2 includes the following steps B2-1 to B2-2;

[0015] Step B2-1. Each client trains the global model of the target to be trained in the t-th round of training to obtain the local model parameters w. i,t And according to x i,t =w i,t ||δ·k i,t ||0, obtain the x corresponding to each client in the t-th round of training. i,t Then proceed to step B2-2; where k i,t ∈Z p Z p It is the residue class of a safe large prime number p. When t≠T, then δ=1, and when t=T, then δ=0. T represents the preset maximum number of training rounds.

[0016] Step B2-2. First, each client, according to... Calculate the corresponding C in the t-th training round. i,t T i,tWhere H0(t) and H1(t) represent the functions of H0 and H1 with respect to t, respectively;

[0017] Then, each client according to Calculate and obtain data v and v', and then... Calculate and obtain the verification key CVk for each client in the t-th round of training. i,t Furthermore, based on a non-interactive zero-knowledge proof mechanism, each client respectively uses C... i,t CVk i,t The function NIZK.Prove is executed to obtain the zero-knowledge proof π for each client in the t-th round of training. i,t ;

[0018] Finally, each client combines the corresponding authentication encryption tag (C) from the t-th round of training. i,t ,T i,t ,π i,t Then upload it to the aggregation server, and proceed to step B3.

[0019] As a preferred technical solution of the present invention: step B3 includes the following steps B3-1 to B3-4;

[0020] Step B3-1. The aggregation server, according to... Construct the public key of the aggregation server in the t-th round of training. And for each client corresponding to the received authentication encryption tag, a public key PK is sent respectively. A,t At the same time, broadcast the list of communication clients U1 constructed by each client, and then proceed to step B3-2;

[0021] Step B3-2. Each client in the communication client list U1 uses the received public key pk. A,t ,according to Calculate the auxiliary information corresponding to the t-th round of training. and combined constitute Upload to a trusted third-party KGC, then proceed to step B3-3; where, This indicates the number of clients in the communication client list U1. Indicates the first in the communication client list U1 The key for each client, Indicates the first in the communication client list U1 Auxiliary information corresponding to each client in the t-th round of training. Indicates the first in the communication client list U1 The client verifies the key during the t-th round of training;

[0022] Step B3-3. Trusted third-party KGC performs various checks on the received data. Given a communication client list U2 consisting of corresponding clients, the intersection of communication client lists U1 and U2 is obtained, which is the stable communication list U3. A trusted third party, KGC, performs overall verification of the auxiliary information uploaded by each client in the stable communication list U3. If the verification is successful, the auxiliary information uploaded by each client in the stable communication list U3 is aggregated to obtain (Au t ,SVk t ), and will provide auxiliary information Au t The stable communication list U3 is sent to the aggregation server, along with the auxiliary information SVk. t Send to each client in the stable communication list U3, and then proceed to step B4; if the verification fails, discard the auxiliary information uploaded by each client in the stable communication list U3, and proceed to step B3-4;

[0023] Step B3-4. The trusted third party KGC requests each client in the stable communication list U3 to re-upload auxiliary information to the trusted third party KGC according to step B3-2, and then return to step B3-3.

[0024] As a preferred technical solution of the present invention: in step B3-3, the trusted third party KGC performs the following operation to perform overall verification of the auxiliary information uploaded by each client in the stable communication list U3;

[0025] First press Calculate and obtain auxiliary information Au t Then, based on the Perdersen commitment system, according to Calculate v and v'; then verify and judge Au. t Does it satisfy the equation? as well as If so, the verification is successful. Calculate and obtain auxiliary information SVk t And will the auxiliary information Au t The stable communication list U3 is sent to the aggregation server, along with the auxiliary information SVk. t Send the information to each client in the stable communication list U3, and then proceed to step B4; otherwise, verification fails, discard the auxiliary information uploaded by each client in the stable communication list U3, and proceed to step B3-4; among which, This indicates the number of clients in the stable communication list U3. Indicates the number of stable communication entries in list U3 Auxiliary information corresponding to each client in the t-th round of training. Indicates the number of stable communication entries in list U3 The client verifies the key during the t-th round of training.

[0026] As a preferred technical solution of the present invention: step B4 includes the following steps B4-1 to B4-4;

[0027] Step B4-1. The aggregation server, based on a non-interactive zero-knowledge proof mechanism, executes the verification function NIZK.Verify to verify the data uploaded by each client in the stable communication list U3. The reliability of each If all are reliable, proceed to step B4-3; if any are unreliable... Then proceed to step B4-2;

[0028] Step B4-2. The aggregation server discards unreliable data. And request that each unreliable The client then re-uploads the authentication encryption tag to the aggregation server according to step B2, and returns to step B4-1;

[0029] Step B4-3. Aggregator server according to Calculate C t , and according to Calculate X t If t = T, then X t This refers to the parameters of the target global model after T rounds of training, thus obtaining the trained target global model; if t≠T, then further... Calculate T t and combined with X t Send to each client in the stable communication list U3, and then proceed to step B4-4;

[0030] Step B4-4. Verify each client in the stable communication list U3. Whether it is true or not, if yes, it is obtained through interaction calculation between clients in the stable communication list U3. Each client in the stable communication list U3 is configured as follows: Obtain model aggregation parameters And update the local model parameters Then proceed to step B5; otherwise, proceed directly to step B5.

[0031] As a preferred embodiment of the present invention, the communication link between each client and the aggregation server is a secure communication link.

[0032] The privacy-preserving bidirectional verifiable federated learning method described in this invention, compared with existing technologies, has the following technical advantages:

[0033] (1) The privacy-preserving bidirectional verifiable federated learning method designed in this invention is based on an aggregation server and various clients, combined with a trusted third party KGC, and introduces stable communication client analysis. Federated learning is performed on the global model of the target to be trained to obtain the trained global model. The design adopts a dual mode to ensure the security of the intermediate model. During training, the encrypted state of the model aggregation result is realized through parameter design, so that attackers cannot infer the gradient information of the client's local model based on the multi-round aggregation result, and non-data owners cannot obtain its private data. While effectively protecting the privacy and security of the client, it also effectively solves the client disconnection problem. Furthermore, the design realizes bidirectional verifiability. The encrypted encryption and decryption function is directly handed over to the client and the aggregation server without relying on a trusted third party, avoiding the possibility of leaking encrypted security parameters. In addition, the model training settings of the client do not need to consider the weight leakage problem, and the design uses non-interactive zero-knowledge proof to ensure the verifiability of the uploaded encrypted text. Attached Figure Description

[0034] Figure 1 An application system architecture diagram for the privacy-preserving bidirectional verifiable federated learning method of this invention is shown.

[0035] Figure 2 A flowchart illustrating the privacy-preserving, bidirectional, verifiable federated learning method designed for this invention is provided. Detailed Implementation

[0036] The specific embodiments of the present invention will be further described in detail below with reference to the accompanying drawings.

[0037] The present invention proposes a privacy-preserving, bidirectional, verifiable federated learning method, the idea of ​​which is as follows:

[0038] (1) Security protection is provided for the local model and the aggregated model during the training process. The gradients of the uploaded local model are encrypted and aggregated. During training, the parameters are designed to configure the aggregation server to obtain the encrypted state of the global model, so that even if a malicious server obtains the model results during the training process, it will not be able to infer any sensitive information.

[0039] (2) Considering that servers or malicious users may forge information and deliberately disrupt the training process and results of the model, a two-way verification method is used to perform security verification on the information transmitted between the client and the server, so as to realize the verifiability of secure aggregation.

[0040] Based on the above design concept, this invention designs a privacy-preserving, bidirectional, verifiable federated learning method, according to... Figure 1 As shown, based on the aggregation server and various clients, combined with the trusted third-party KGC, the communication links between each client and the aggregation server are designated as secure communication links, and each client is assumed to be honest and trustworthy. Figure 2 As shown, the specific design executes the following steps A to B: perform federated learning on the global model of the target to be trained to obtain the trained global model of the target.

[0041] Step A. Initialize the public parameter pm based on the trusted third party KGC and distribute it to the aggregation server and each client. The aggregation server generates a key, and each client generates a key. Then proceed to step B1.

[0042] Step B1. Initialize the number of training rounds t = 1, and proceed to step B2-1.

[0043] Step B2-1. Each client trains the global model of the target to be trained in the t-th round of training to obtain the local model parameters w. i,t And according to x i,t =w i,t ||δ·k i,t ||0, obtain the x corresponding to each client in the t-th round of training. i,t Then proceed to step B2-2; where k i,t ∈Z p Z p It is the residue class of a safe large prime number p. When t≠T, then δ=1, and when t=T, then δ=0. T represents the preset maximum number of training rounds.

[0044] Step B2-2. First, each client, according to... Calculate the corresponding C in the t-th training round. i,t T i,t , where H0(t) and H1(t) represent the functions of H0 and H1 with respect to t, respectively.

[0045] Then, each client according to Calculate and obtain data v and v', and then... Calculate and obtain the verification key CVk for each client in the t-th round of training. i,t Furthermore, based on a non-interactive zero-knowledge proof mechanism, each client respectively uses C... i,t CVk i,t The function NIZK.Prove is executed to obtain the zero-knowledge proof πi for each client in the t-th round of training. ,t .

[0046] Finally, each client combines the corresponding authentication encryption tag (Ci) from the t-th round of training.,t Ti ,t ,πi , t), and upload it to the aggregation server, then proceed to step B3-1.

[0047] Step B3-1. The aggregation server, according to... Construct the public key of the aggregation server in the t-th round of training. And for each client corresponding to the received authentication encryption tag, a public key PK is sent respectively. A,t At the same time, broadcast the list of communication clients U1 constructed by each client, and then proceed to step B3-2;

[0048] Step B3-2. Each client in the communication client list U1 uses the received public key pk. A,t ,according to Calculate the auxiliary information corresponding to the t-th round of training. and combined constitute Upload to a trusted third-party KGC, then proceed to step B3-3; where, This indicates the number of clients in the communication client list U1. Indicates the first in the communication client list U1 The key for each client, Indicates the first in the communication client list U1 Auxiliary information corresponding to each client in the t-th round of training. Indicates the first in the communication client list U1 The client verifies the key during the t-th round of training;

[0049] Step B3-3. Trusted third-party KGC performs various checks on the received data. Given a communication client list U2 consisting of corresponding clients, the intersection of communication client lists U1 and U2 is obtained, which is the stable communication list U3. A trusted third party, KGC, performs overall verification of the auxiliary information uploaded by each client in the stable communication list U3. If the verification is successful, the auxiliary information uploaded by each client in the stable communication list U3 is aggregated to obtain (Au t ,SVk t ), and will provide auxiliary information Au t Send the stable communication list U3 to the aggregation server, send the auxiliary information SVkt to each client in the stable communication list U3, and then proceed to step B4-1; if the verification fails, discard the auxiliary information uploaded by each client in the stable communication list U3, and proceed to step B3-4.

[0050] In practical applications, in step B3-3 above, the trusted third party KGC performs the following operations to verify the auxiliary information uploaded by each client in the stable communication list U3 as a whole.

[0051] First press Calculate and obtain auxiliary information Au t Then, based on the Perdersen commitment system, according to Calculate v and v'; then verify and judge Au. t Does it satisfy the equation? as well as If so, the verification is successful. Calculate and obtain auxiliary information SVK t And will the auxiliary information Au t The stable communication list U3 is sent to the aggregation server, along with the auxiliary information SVk. t Send the information to each client in the stable communication list U3, then proceed to step B4-1; otherwise, verification fails, discard the auxiliary information uploaded by each client in the stable communication list U3, and proceed to step B3-4; where, This indicates the number of clients in the stable communication list U3. Indicates the number of stable communication entries in list U3 Auxiliary information corresponding to each client in the t-th round of training. Indicates the number of stable communication entries in list U3 The client verifies the key during the t-th round of training.

[0052] Step B3-4. The trusted third party KGC requests each client in the stable communication list U3 to re-upload auxiliary information to the trusted third party KGC according to step B3-2, and then return to step B3-3.

[0053] Step B4-1. The aggregation server, based on a non-interactive zero-knowledge proof mechanism, executes the verification function NIZK.Verify to verify the data uploaded by each client in the stable communication list U3. The reliability of each If all are reliable, proceed to step B4-3; if any are unreliable... Then proceed to step B4-2;

[0054] Step B4-2. The aggregation server discards unreliable data. And request that each unreliable The client then re-uploads the authentication encryption tag to the aggregation server according to step B2, and returns to step B4-1;

[0055] Step B4-3. Aggregator server according to Calculate C t , and according to Calculate X t If t = T, then X t This refers to the parameters of the target global model after T rounds of training, thus obtaining the trained target global model; if t≠T, then further... Calculate T t and combined with X t Send to each client in the stable communication list U3, and then proceed to step B4-4;

[0056] Step B4-4. Verify each client in the stable communication list U3. Whether it is true or not, if yes, it is obtained through interaction calculation between clients in the stable communication list U3. Each client in the stable communication list U3 is configured as follows: Obtain model aggregation parameters And update the local model parameters Then proceed to step B5; otherwise, proceed directly to step B5.

[0057] Step B5. Increment the value of t by 1, then return to step B2.

[0058] The aforementioned technical solution employs a privacy-preserving, bidirectionally verifiable federated learning method. Based on an aggregation server and various clients, and combined with a trusted third-party KGC, it introduces stable communication client analysis. Federated learning is performed on the global model of the target to be trained, yielding the trained global model. A dual-mode design ensures the security of intermediate models. During training, parameter design implements an encrypted state for the model aggregation results, preventing attackers from inferring the gradient information of the client's local model based on multiple rounds of aggregation results. Furthermore, non-data owners cannot access the client's private data, effectively protecting client privacy and security while resolving client disconnection issues. The design also implements bidirectional verifiability, with encryption and decryption functions directly handled by the client and aggregation server, eliminating reliance on a trusted third-party institution and preventing the leakage of encryption security parameters. Additionally, the client's model training settings eliminate concerns about weight leakage, and the design utilizes non-interactive zero-knowledge proofs to guarantee the verifiability of uploaded encrypted data.

[0059] In practical applications, whether the server is trusted or malicious, it can only obtain obfuscated aggregated model gradients during training, and the final aggregated result is only obtained at the end of training. During this process, the aggregation server cannot obtain any private information from the clients, effectively avoiding security issues caused by intermediate model leakage. Furthermore, bidirectional verification communication is implemented between the aggregation server and each client to verify the aggregation results from the server and the information uploaded by each client. This effectively prevents malicious servers and attackers from impersonating participants to forge results, disrupt the training process, and impair training effectiveness.

[0060] The embodiments of the present invention have been described in detail above with reference to the accompanying drawings. However, the present invention is not limited to the above embodiments. Within the scope of knowledge possessed by those skilled in the art, various changes can be made without departing from the spirit of the present invention.

Claims

1. A privacy-preserving, bidirectional, verifiable federated learning method, characterized in that: Based on the aggregation server and various clients, combined with trusted third parties Follow steps A to B below to perform federated learning on the global model of the target to be trained, and obtain the trained global model of the target. Step A. Based on a trusted third party Initialize common parameters The keys are distributed to the aggregation server and each client. The aggregation server generates a key, and each client generates a key. Then, proceed to step B. Step B. Based on the preset maximum number of training rounds Based on the local model parameters obtained by each client training the global model of the training target in each round of training, combined with those obtained by a trusted third party... Based on the established list of stable communications, the local model parameters of each stable communication client are updated, and then in the... The target global model is obtained after training through multiple rounds of training; Step B above includes steps B1 to B5; Step B1. Initialize the number of training rounds Then proceed to step B2; Step B2. Each client respectively in the... In the training round, the global model of the target to be trained is trained to obtain the local model parameters, and the authentication and encryption tags are constructed, uploaded to the aggregation server, and then proceed to step B3; Step B3. Based on the aggregation server and trusted third party The communication and interaction between them regarding the key determines a stable communication list, and then proceeds to step B4; Step B4. The aggregation server performs model parameter aggregation for the authentication and encryption tags uploaded by each client in the stable communication list. When, the trained global model of the target is obtained, when If so, proceed to step B5; Step B5. Targeting The value is incremented by 1 to update the value, and then the process returns to step B2; Step B2 above includes steps B2-1 to B2-2 as follows; Step B2-1. Each client respectively in the... In round training, the local model parameters are obtained by training the global model of the target object. and according to To obtain the information of each client in the 1st month... The corresponding round of training Then proceed to step B2-2; where, , It is a safe large prime number. The remaining classes, when At that time, ,when At that time, , This indicates the preset maximum number of training rounds; Step B2-2. First, each client, according to... , Calculate to obtain the first The corresponding round of training , ,in, , They represent and In response to Function operations with the independent variable; Then, each client according to , Calculate and obtain data , , and according to The calculations obtained for each client at the [number]th [month] Validation key under round training Furthermore, based on a non-interactive zero-knowledge proof mechanism, each client respectively... , Execute function To obtain the information of each client in the 1st month... Zero-knowledge proof under round training ; Finally, each client combines the first... The corresponding authentication encryption label under the training round Then upload it to the aggregation server, and proceed to step B3.

2. The privacy-preserving bidirectional verifiable federated learning method according to claim 1, characterized in that: Step B3 includes the following steps B3-1 to B3-4; Step B3-1. The aggregation server, based on... , Build the aggregation server in the first Public key training in rounds It also sends public keys to the clients corresponding to each received authentication encryption label. At the same time, broadcast the list of communication clients constructed by each client. Then proceed to step B3-2; Step B3-2. List of Communication Clients Each client in the process uses the received public key. ,according to , Calculate to obtain the first Auxiliary information corresponding to each training round and combined ,constitute Upload to a trusted third party Then proceed to step B3-3; where, , Represents the list of communication clients The number of Chinese clients Represents the list of communication clients The Middle The key for each client, Represents the list of communication clients The Middle The client in the first The corresponding auxiliary information during the training round. Represents the list of communication clients The Middle The client in the first Verification key under round training; Step B3-3. Trusted Third Party For each received The corresponding client constitutes a list of communication clients. Obtain the list of communication clients List of communication clients The intersection between them, i.e., the stable communication list By a trusted third party For stable communication list The auxiliary information uploaded by each client is verified as a whole. If the verification is successful, the stable communication list is aggregated. Each client uploads auxiliary information and obtains... and auxiliary information With stable communication list Send to the aggregation server to provide auxiliary information Send to stable communication list For each client, proceed to step B4; if verification fails, discard the stable communication list. The auxiliary information uploaded by each client is then processed, and the process proceeds to steps B3-4. Step B3-4. Trusted Third Party Request a stable communication list Each client should re-follow step B3-2 and submit the request to the trusted third party. Upload the auxiliary information and return to step B3-3.

3. The privacy-preserving bidirectional verifiable federated learning method according to claim 2, characterized in that: In step B3-3, a trusted third party... Follow these steps for the stable communication list. The auxiliary information uploaded by each client is verified as a whole; First press Calculate and obtain auxiliary information ; then based on Commitment system, according to , Calculation obtained , Then verify and judge. Does it satisfy the equation? ,as well as If so, the verification is successful. Calculate and obtain auxiliary information and auxiliary information With stable communication list Send to the aggregation server to provide auxiliary information Send to stable communication list If all clients are verified, proceed to step B4; otherwise, verification fails and the stable communication list is discarded. The auxiliary information uploaded by each client is then processed, and the process proceeds to steps B3-4; among which, Represents a stable communication list The number of Chinese clients Represents a stable communication list The Middle The client in the first The corresponding auxiliary information during the training round. Represents a stable communication list The Middle The client in the first Validation key under round training .

4. The privacy-preserving bidirectional verifiable federated learning method according to claim 2, characterized in that: Step B4 includes the following steps B4-1 to B4-4; Step B4-1. The aggregation server executes the verification function based on a non-interactive zero-knowledge proof mechanism. Verify the stable communication list respectively Uploaded by various clients The reliability of each If all are reliable, proceed to step B4-3; if any are unreliable... Then proceed to step B4-2; Step B4-2. The aggregation server discards unreliable data. And request that each unreliable The client then re-uploads the authentication encryption tag to the aggregation server according to step B2, and returns to step B4-1; Step B4-3. Aggregator server according to Calculation obtained , and according to Calculation obtained ,like At that time, That is The parameters of the target global model after training are obtained, thus yielding the trained target global model; if Then, further according to Calculation obtained and combined Send to stable communication list Then proceed to step B4-4; Step B4-4. Stable Communication List Each client in China verifies separately. If true, then a stable communication list is established. Interactive computation between various clients Stable communication list Each client in China presses the following: Obtain model aggregation parameters And update the local model parameters Then proceed to step B5; Otherwise, proceed directly to step B5.

5. A privacy-preserving bidirectional verifiable federated learning method according to any one of claims 1 to 4, characterized in that: The communication links between each client and the aggregation server are secure communication links.

Images