Verifiable Federated Learning Method Based on BLS Signature and Gradient Sparsity

Abstract

This invention discloses a verifiable federated learning method based on BLS signatures and gradient sparsity, relating to the interdisciplinary fields of artificial intelligence and information security. The method includes: a client compressing its trained local model using a gradient sparsity-based model compression algorithm, and then encrypting the compressed local model parameters using a homomorphic encryption scheme. A trusted third-party organization verifies the integrity of all model parameters uploaded to the cloud server by the client using a BLS signature-based protocol. Verified local model parameters participate in global model aggregation. After aggregation, a similar protocol is used to verify the correctness of the aggregation result on the cloud server. Upon successful verification, the cloud server sends the updated global model to the client; the client decrypts the received global model and loads it into its local model for the next iteration of training.

Description

Technical Field

[0001] This invention relates to the interdisciplinary fields of information security and artificial intelligence, specifically to a verifiable federated learning aggregation protocol based on BLS signatures and gradient sparsity. Background Technology

[0002] Federated learning, as a novel collaborative learning method, allows multiple clients to collaboratively develop deep learning models. Clients upload model parameters to a cloud server, which then aggregates the uploaded model parameters to update the global model without accessing the client's local data.

[0003] However, in practical applications, federated learning has revealed some security issues. Firstly, the correctness of the aggregation results from cloud servers in a federated learning system cannot be guaranteed. Driven by illicit interests, malicious cloud servers may perform incorrect aggregations and return incorrect results to clients. In this case, the performance of the global model will be affected. Malicious cloud servers may intentionally send pre-designed results to specific clients in order to analyze the characteristics of that client's data. Secondly, unreliable clients may exist in the federated system. If model parameters uploaded by unreliable clients participate in model aggregation, the global model may be poisoned, and the gradient aggregation protocol may be compromised. Summary of the Invention

[0004] The purpose of this invention is to provide a verifiable federated learning method based on BLS signatures and gradient sparsity, which aims to solve the problems of malicious aggregation of cloud servers and poisoning by unreliable clients in federated learning systems.

[0005] The technical solution of the present invention is as follows:

[0006] A verifiable federated learning method based on BLS signature and gradient sparsity is applied to a federated learning system comprising a trusted third-party institution, a cloud server, and K clients, where K > 1 and K is an integer. The method performs the following operations in the i-th iteration of the federated learning training:

[0007] First, the cloud server sends the global model to the client, and the model type includes neural network model, convolutional neural network model, LSTM neural network model, GAN neural network model or deep belief neural network model;

[0008] The second and kth clients receive the global model Enc(w) sent by the cloud server. glo After that, for Enc(w) glo Decryption yields the global model parameters w. glo ;

[0009] The 3rd and kth clients will transfer the global model parameters w. glo Load the data locally and train it using an optimization algorithm (including gradient descent, momentum gradient descent, SGD, Adam, Adagrad, or RMSProp) on the local dataset to obtain the local model parameters w. a,k Specifically, it includes:

[0010] 3.1 The k-th client randomly selects a batch of data from its local dataset. k The training samples, where batch k Greater than 0 and less than or equal to the total number of data samples for that client;

[0011] 3.2 The k-th client performs local model training and obtains the gradient parameters ▽w a,k Then, an optimization algorithm is used to calculate the weight parameters w of the local model. a,k ;

[0012] The 4th and kth clients use a gradient sparsity-based model compression algorithm to compress their local models to obtain Sparse(w) a,k Specifically, this includes:

[0013] 4.1 The k-th client downloads the sparsity ratio ρ from the cloud server, where 0 ≤ ρ < 1;

[0014] 4.2 The kth client calculates the model gradient threshold θ based on the sparsity rate ρ, where θ is greater than the minimum absolute value of the model gradient and less than the maximum absolute value of the model gradient.

[0015] 4.3. The k-th client selects model parameters whose absolute gradient value exceeds θ to obtain Sparse(w) a,k The index Idx of the sparse model parameters. k ;

[0016] The 5th and kth clients use a homomorphic encryption algorithm to encrypt the compressed model parameters to obtain Enc(Sparse(w a,k The homomorphic encryption algorithms specifically include: CKKS algorithm, Paillier algorithm, or RSA algorithm;

[0017] The 6th and kth clients generate auxiliary verification information based on the BLS signature verification protocol, and then use Enc(Sparse(w) to generate auxiliary verification information. a,k )) and auxiliary verification information are sent to the cloud server; specifically including:

[0018] 6.1 The k-th client divides the encrypted sparse model parameters into n data blocks, and each block is further divided into s data segments.

[0019] 6.2, The k-th client calculates the tag and signature The calculation formula is Where H is a one-way hash function. These are the parameters of the processed sparse model; spσ (k) The calculation formula is in It is the generated tag, sk k It is the private key distributed to the k-th client by a trusted third-party institution, and g is the generator of the cyclic group G and G1. These are the parameters of the processed sparse model, λ. i It is the public parameter of the i-th data block;

[0020] 6.3, the k-th client will use Enc(Sparse(w) a,k )), mark and signature Send to the cloud server;

[0021] 7. The aforementioned trusted third-party organization, in conjunction with the cloud server, verifies the integrity of the parameters uploaded by each client; specifically including:

[0022] 7.1 A trusted third-party organization uses a random number generator to generate a sequence of challenge information {β1, β2, ..., β} consisting of c integers. c}, and send the challenge information sequence to the cloud server, where 1 <c<n;

[0023] 7.2 The cloud server generates proof information based on the challenge information sequence.

[0024] Section 7.3, the aforementioned trusted third-party organization performs integrity verification and calculates... Whether the equation holds true or not, e is a bilinear pairing mapping. If the equation holds true, the integrity verification is passed; if it does not hold true, the integrity verification is failed, and the cloud server refuses to use the model uploaded by the client for aggregation.

[0025] 8. The cloud server aggregates the verified local models and returns the aggregation results to each client.

[0026] 9. The aforementioned trusted third-party organization, in conjunction with each of the aforementioned clients, verifies the correctness of the aggregation result; specifically including:

[0027] 9.1 The k-th client divides the encrypted local model parameters into n data blocks, and each block is further divided into s data segments. Where n>0, and n is an integer; s>0, and s is an integer.

[0028] 9.2, the k-th client calculates the tag and signature The calculation formula is: Where H is a one-way hash function. These are the processed local model parameters; Calculation formula in It is the generated tag, sk k It is the private key distributed to the k-th client by a trusted third-party institution, and g is the generator of the cyclic group G and G1. These are the processed local model parameters, λ i It is the public parameter of the i-th data block;

[0029] 9.3, the kth client will mark and signature Send to the cloud server;

[0030] 9.4. A trusted third-party organization uses a random number generator to generate a sequence of c integers {β} as the challenge information. 1, β2,…,β c}, and send the challenge information to each of the aforementioned clients, where 1 <c<n;

[0031] 9.5 Each client generates proof information based on the challenge information. Send to the aforementioned trusted third-party organization; where c is the number of challenge messages, and g is the generator of the cyclic group G and G1. These are the processed global model parameters, λ i It is the public parameter of the i-th data block

[0032] Section 9.6, the calculation by the aforementioned trusted third-party institution If the equation is true, the aggregation result is correct; if the equation is false, the cloud server performed incorrect aggregation.

[0033] 10. Each client receives the global model Enc(w) glo Decrypt the global model w. glo Load it locally for the next training session.

[0034] This invention also provides a federated learning system, including a trusted third-party institution, a cloud server, and K clients, where K > 1 and K is an integer;

[0035] The aforementioned trusted third-party organization verifies the integrity of the local model parameters uploaded by the client before the local model participates in the aggregation; and verifies the correctness of the aggregation result on the cloud server after the global model is aggregated on the cloud server.

[0036] The cloud server, based on the Enc(Sparse(w) uploaded by each client a,k Aggregate the model parameters to obtain Enc(w) glo ), and Enc(w glo Distribute to each client;

[0037] The k-th client, for Enc(w) glo Decrypt to obtain w glo , will w glo Load the local model, train the local model using the optimization algorithm, and obtain the updated parameters w of the local model. a,k The local model parameters are sparsified using a gradient-based sparsification-based model compression algorithm to obtain Sparse(w) a,k Then, the encryption yields Enc(Sparse(w)). a,k Then upload it to the parameter server.

[0038] The present invention also provides a computer device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the method described above.

[0039] Since the BLS signature-based verification protocol does not affect the model's performance, meaning the verification process does not impact the model's performance, this allows for the verification of the correctness of cloud server aggregation results and the completeness of client-uploaded parameters while ensuring model accuracy.

[0040] Because gradient sparsity-based model compression algorithms are adaptive, meaning they can adjust the sparsity rate according to the model's training state, they can achieve a good balance between ensuring model performance and reducing resource overhead.

[0041] Advantages and beneficial effects of the present invention:

[0042] (i) Compared with the traditional federated algorithm, the algorithm of this invention introduces an aggregation protocol based on BLS signature to verify the correctness of the cloud server aggregation results and the integrity of the parameters uploaded by the client, protect the rights and interests of the client, and prevent the model from being attacked by malicious cloud servers and untrusted clients.

[0043] (ii): This invention uses the feedback adjustment method in the field of control to design a model compression algorithm based on gradient sparsity. By compressing the local model parameters, resource overhead is further reduced and the performance of the model is guaranteed.

[0044] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and do not limit the embodiments of this specification.

[0045] Furthermore, none of the embodiments described in this specification need to achieve all of the above effects. Attached Figure Description

[0046] Figure 1 This is a flowchart illustrating the federated learning method based on BLS signature and gradient sparsity provided in an embodiment of the present invention.

[0047] Figure 2 This is a schematic diagram illustrating the principle of the federated learning method based on BLS signature and gradient sparsity provided in the embodiments of the present invention;

[0048] Figure 3 This is a schematic diagram of the structure of a federated learning cloud server based on BLS signature and gradient sparsity provided in an embodiment of the present invention;

[0049] Figure 4 This is a schematic diagram of a client for federated learning based on BLS signatures and gradient sparsity provided in an embodiment of the present invention;

[0050] Figure 5 This is a schematic diagram of the structure of a device used to configure the method of the embodiments of the present invention. Detailed Implementation

[0051] The embodiments of the present invention will now be clearly and completely described with reference to the accompanying drawings. Obviously, the described embodiments are only a part of, and not all, of the embodiments of the present invention. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without creative effort are within the scope of protection of the present invention.

[0052] Please see Figure 1 , Figure 1 This is a flowchart illustrating the federated learning method based on BLS signature and gradient sparsity provided in this embodiment of the invention, including steps S101-S106:

[0053] When training a deep learning model, the model parameters are usually adjusted through iterative training. S101-S106 is one iteration in the model training process. The training iteration can be stopped when the training stopping condition is met. Specifically, the training stopping condition can be set as follows: the number of times steps S100 to S106 are executed reaches a specified number of epochs, or the loss function value of the verification dataset is less than a specified value.

[0054] For ease of description, in this embodiment of the invention, S101 to S106 are considered as the steps executed in the t-th iteration of the federated learning system. If the training termination condition is that the number of iterations reaches a specified number of epochs, then 0 ≤ t ≤ epoch.

[0055] Figure 1 The method shown is applied to a federated learning system, which includes one trusted third-party institution, one cloud server, and K clients.

[0056] In this embodiment of the invention, for ease of description, the homomorphic encryption algorithm is denoted as Enc, the data encrypted by the homomorphic encryption algorithm is denoted as Enc(*), * represents plaintext data, the model compression algorithm is denoted as Sparse, the model compressed by the model compression algorithm is denoted as Sparse(·), · represents model parameters, and the weight parameters of the global model are denoted as w. glo The weight parameters of the k-th client model are denoted as w. a,k .

[0057] S101: The cloud server sends global model parameters to K clients.

[0058] The application scenarios of this invention mainly include cloud server to B (cloud server and at least two organizations collaborating on training) scenarios and cloud server to C (cloud server and at least two individual users collaborating on training) scenarios. In these scenarios, the cloud server, organizations, and individual users are honest and curious entities; that is, they are capable of executing the agreement as agreed, but may infer from the obtained data to obtain the privacy information of other entities.

[0059] In cloud server-to-B scenarios, due to the limited number of clients, the global model can be sent to each client in each training iteration.

[0060] In cloud server-to-consumer scenarios, a large number of individual users typically participate in training. In each iteration, to prevent excessive processing pressure on the cloud server, some clients can be selected to upload model parameters to participate in model aggregation.

[0061] It should be noted that the number K of clients selected by the server in the t-th iteration is... t They can be different, where 1 <K t ≤K.

[0062] S102: The k-th client performs model training locally, compresses the local model parameters using a model compression algorithm based on gradient sparsification to obtain Sparse(w a,k ), encrypts the sparsified model parameters using a homomorphic encryption algorithm to obtain Enc(Sparse(w a,k )), generates auxiliary verification information based on the integrity verification protocol of BLS signature, and sends Enc(Sparse(w a,k )) and the auxiliary verification information to the cloud server;

[0063] It should be noted that Sparse(w a,k ) is the local model parameters compressed by the k-th client using the model compression algorithm based on gradient sparsification, and w a,k represents the local model weight parameters obtained after the k-th client completes local model training.

[0064] It should also be noted that the model compression algorithm based on gradient sparsification refers to adding an adaptive sparsity rate adjustment algorithm to the traditional gradient sparsification algorithm, specifically adjusting the sparsity rate adaptively through the change of model accuracy, reducing the system resource overhead while ensuring the model performance.

[0065] In the embodiments of this specification, various neural network models can be used for federated learning. Here, the neural network algorithm is taken as an example for illustration.

[0066] At the beginning of the t-th iteration, the k-th client calculates the local model accuracy acc

[0067] , and then sends it to the cloud server; k , and then sends it to the cloud server;

[0068] The cloud server calculates the average accuracy μ k represents the data distribution rate of the k-th client, and is calculated by the formula μ k =|D k | / |D all |, where |D k | represents the number of data samples of the k-th client, and |D all | represents the number of data samples of all clients;

[0069] When the average accuracy acc glo is lower than the highest average accuracy ψ times, where 0 < ψ < epoch and ψ is an integer, the cloud server adjusts the sparsity rate ρ, and the adjustment formula is ρ = max(ρ - d, 0), where d represents the attenuation rate, and 0 < d < 1;

[0070] The cloud server will use the sparsity ρ and the global model parameters Enc(w) glo Send to the client;

[0071] Assume the neural network model has parameters w, and the samples are labeled (x, y), where x is the data sample and y is the data label;

[0072] The k-th client randomly selects a batch of data from its local dataset. k The training samples (x, y), where batch k Greater than 0 and less than or equal to the total number of data samples for that client;

[0073] The kth client performs forward propagation calculations based on the extracted samples to obtain the neural network model output value out = f(w,x);

[0074] The k-th client calculates the empirical loss value L of the neural network model based on the model output value and the data label value;

[0075] The k-th client calculates the gradient parameter ▽w based on the empirical loss value of the neural network model, the parameters of the neural network model, and the sample. ,k ;

[0076] The k-th client uses the gradient descent optimization algorithm to calculate the weight parameters w of the local model. k ;

[0077] The kth client will set the gradient parameter ▽w ,k The sequence G is obtained by sorting the absolute values ​​from smallest to largest.

[0078] The kth client calculates the threshold θ = G[ρ·L] based on the sparsity rate, where L represents the number of gradient parameters in sequence G;

[0079] If the gradient parameter is greater than the threshold, it means that the weight parameter has changed sufficiently in this iteration and is representative enough, that is, it represents the model weight parameter trained locally by the corresponding client in this round. Therefore, the model parameter needs to be uploaded to the cloud server.

[0080] The k-th client selects weight parameters whose gradient parameters are greater than the threshold to obtain Sparse(w) a,k ), will Sparse(w a,k The index sequence of weight parameters is sent to the cloud server.

[0081] The cloud server receives the Sparse(w) uploaded by the k-th client. a,k After the local model is restored using the index sequence of weight parameters, the model parameters that were not uploaded are the global model parameters aggregated in the (t-1)th iteration.

[0082] S103: A trusted third-party organization verifies the integrity of the parameters uploaded by each client.

[0083] In this embodiment of the invention, a BLS signature-based verification protocol is used to verify the integrity of the model parameters uploaded by the client. Specifically:

[0084] During the system initialization phase, the trusted third-party institution selects cyclic groups G and G1 with prime order q and generator g, bilinear pairing mapping e: G×G=G1, and one-way hash function H.

[0085] The trusted third-party institution randomly selects a private key sk for each client and uses it to calculate the client's public key pk = g. sk ;

[0086] The trusted third-party organization publishes the parameters {g,q,e,G,G1,sk} to other members of the system.

[0087] The k-th client first processes the encrypted compression model parameters, dividing them into n data blocks, and each data block is further divided into s data segments, resulting in... Then, calculate the tag of the nth block. and signature Finally, mark and signature Send to the cloud server;

[0088] A trusted third-party organization uses a random number generator to generate a sequence of challenge information {β1,β2,…,β} consisting of c integers. c}, sent to the cloud server, of which 1 <c<n;

[0089] The cloud server generates proof information based on the challenge information sequence. Used to assist trusted third-party organizations in performing integrity verification;

[0090] S104: The cloud server performs weighted aggregation on the verified local models and returns the aggregation results to each client.

[0091] After receiving the proof information sent by the cloud server, the trusted third-party institution calculates the equation. If the equation is true, the integrity verification passes; otherwise, the integrity verification fails, and the model parameters uploaded by the client are deemed unreliable. The cloud server then refuses to use the model uploaded by the client for aggregation.

[0092] S105: The trusted third party verifies the correctness of the aggregation result.

[0093] In this embodiment of the invention, a verifiable protocol based on BLS signatures is used to verify the correctness of the cloud server aggregation results. Specifically:

[0094] The k-th client divides the encrypted local model parameters into n data blocks, and each block is further divided into s data segments. Then calculate the mark of the nth block. and signature and mark and signature Send to the cloud server;

[0095] A trusted third-party organization uses a random number generator to generate a sequence of challenge information {β1,β2,…,β} consisting of c integers. c}Sent to each of the aforementioned clients, of which 1 <c<n;

[0096] Each client generates proof information based on the challenge information. Send to the aforementioned trusted third-party organization;

[0097] The third-party trusted institution calculates whether the equation holds true. If the equation is true, the aggregation result is correct; if the equation is false, the cloud server performed incorrect aggregation.

[0098] S106: Each client receives the global model Enc(W) glo Decrypt the global model W. glo Load it locally for the next training session.

[0099] Figure 2 This is a schematic diagram illustrating the principle of the federated learning method based on BLS signature and gradient sparsity provided in this embodiment of the invention.

[0100] exist Figure 2The diagram illustrates a federated learning system comprising one trusted third-party institution, one cloud server, and three clients. First, the cloud server distributes global model parameters to each client. Then, each client decrypts the global model and trains it locally. After local model training, the local model is compressed using a gradient sparsity-based model compression algorithm, and the compressed model is encrypted using a homomorphic encryption algorithm. Auxiliary verification information is generated according to a BLS signature-based verification protocol, and the encrypted compressed model parameters and auxiliary verification information are sent to the cloud server. The trusted third-party institution, in conjunction with the cloud server, verifies the integrity of the model parameters uploaded by each client and returns the verification result to the cloud server. The cloud server aggregates the verified client models and distributes the models to each client. The trusted third-party institution, in conjunction with the clients, verifies the correctness of the aggregation result. After verifying the aggregation result is correct, each client decrypts the global model and loads it locally for the next iteration of training. This entire process is repeated until the system's training termination conditions are met.

[0101] It should be noted that the input data of the model in the embodiments of the present invention may include: images, text, and voice.

[0102] It should also be noted that the models used for image processing in the embodiments of the present invention may include: image classification models, image segmentation models, object detection models, etc.; the models used for text processing in the present invention may include natural language processing models, intelligent recommendation models, etc.; and the models used for speech processing in the present invention may include voice assistant models, speech recognition models, etc.

[0103] Figure 3 This is a schematic diagram of the structure of a federated learning cloud server based on BLS signatures and gradient sparsity provided in an embodiment of the present invention. The cloud server includes:

[0104] The local model parameter receiving module 301 receives the encrypted and compressed local model.

[0105] The sparsity rate adjustment module 302 calculates the average accuracy and adjusts the sparsity rate according to the set attenuation rate.

[0106] The parameter integrity proof information generation module 303 calculates the parameter integrity proof information of each client based on the challenge information sequence sent by the third-party trusted institution, and sends it to the third-party trusted institution.

[0107] The global model aggregation module 304 restores the local model that has passed integrity verification and performs global model aggregation;

[0108] The global model distribution module 305 distributes the aggregated global model to each client.

[0109] Figure 4 This is a schematic diagram of the structure of a federated learning client based on BLS signatures and gradient sparsity provided in an embodiment of the present invention. The federated system includes K clients, and the k-th client includes:

[0110] The global model parameter receiving module 401 receives global model parameters, decrypts them, and loads them locally.

[0111] The local model training module 402 trains the model on the local dataset.

[0112] The local model compression module 403 uses a gradient sparsity-based model compression algorithm to compress the local model parameters;

[0113] The local model parameter encryption module 404 uses a homomorphic encryption algorithm to encrypt the model parameters.

[0114] The auxiliary verification information generation module 405 generates auxiliary verification information according to the BLS signature-based verification protocol;

[0115] The local model parameters and auxiliary verification information upload module 406 sends the encrypted and compressed model parameters and auxiliary verification information to the cloud server.

[0116] Figure 5 A schematic diagram of a hardware operating environment structure provided by an embodiment of the present invention is shown.

[0117] The methods, modules, or systems described in the embodiments of this invention can be implemented by a computer chip or other entities. The specific form of the computer can be a portable computer, a smartphone, a smartwatch, a tablet computer, or any combination of these devices.

[0118] Please see Figure 5 The hardware device environment includes: network interface 501, communication interface 502, input / output interface 503, processor 504, memory 505, and bus 506.

[0119] Network interface 501 is used for network communication and may include standard wired interfaces (such as network cables) and wireless interfaces (such as mobile networks and WIFI).

[0120] The communication interface 502 is mainly used to realize communication and data transmission between this device and other devices, and may include wired interfaces (such as USB interfaces) and wireless interfaces (such as Bluetooth, NFC, etc.).

[0121] Input / output interface 503 is used for inputting and outputting data information. Input devices may include microphones, keyboards, mice, cameras, sensors, etc., and output devices may include monitors, speakers, etc. It should be noted that input and output devices can be configured as components in computer equipment or connected externally to other devices.

[0122] The processor 504 is used to call and execute the program stored in the memory to implement the technical solution in the embodiment of the present invention. It can be a CPU (Central Processing Unit), microprocessor, etc.

[0123] The memory 505 is used to store the operating system and computer program code, and may include RAM memory, disk storage, and storage devices independent of the processor.

[0124] Bus 506 is used to transfer information between various device components.

[0125] It should be noted that the embodiments of the present invention only illustrate network interface 501, communication interface 502, input / output interface 503, processor 504, memory 505, and bus 506. However, in actual implementation, the hardware environment may also include other components necessary for normal operation, or it may only include the components necessary in the embodiments of the present invention, instead of those shown. Figure 5 All components.

[0126] As can be seen from the above description, those skilled in the art can clearly understand that the embodiments of the present invention can be implemented by means of software plus necessary general-purpose hardware platforms. Based on this understanding, the technical solutions of the embodiments of the present invention, or the parts that contribute to the prior art, can be embodied in the form of software products. These software products can be stored in storage media, such as ROM / RAM, optical discs, etc., and include several instructions to cause a computer device (which may be a personal computer, a service device, or a network device, etc.) to execute the methods described in various embodiments or certain parts of the embodiments of the present invention.

[0127] The above description is merely a specific implementation of the embodiments of the present invention. It should be noted that for those skilled in the art, several improvements and modifications can be made without departing from the principle of the embodiments of the present invention, and these improvements and modifications should also be considered within the protection scope of the embodiments of the present invention.

Claims

1. A verifiable federated learning method based on BLS signature and gradient sparsification, applied to a federated learning system comprising a third-party trusted authority, a cloud server and K clients, wherein K>1, and K is an integer, characterized in that The method comprises the following steps: Step 1, a cloud server sends global model parameters to K clients; The second and the kth clients locally train a model, compress local model parameters using a gradient sparsification-based model compression algorithm to obtain Sparse(w a,k ), encrypt the sparsified model parameters using a homomorphic encryption algorithm to obtain Enc(Sparse(w a,k )), generate auxiliary verification information according to a BLS signature-based verification protocol, and send the Enc(Sparse(w a,k )) and the auxiliary verification information to the cloud server; Step 3, a third-party trusted agency verifies the integrity of the parameters uploaded by each client in combination with the cloud server; Step 4, the cloud server aggregates the local models that pass the verification, and returns the aggregation result to each client; Step 5, the third-party trusted agency verifies the correctness of the aggregation result in combination with each client; After the correctness of the aggregation result is verified, each client decrypts the received global model Enc(w glo ) and loads the global model w glo locally for the next training. In step 3, the third-party trusted agency verifies the integrity of the model uploaded by each client, specifically including: The third-party trusted authority uses a random number generator to generate a challenge information sequence composed of c integers 1, 2, …, c} and sends the challenge information sequence to the cloud server; wherein 1 < c < n; The cloud server generates the proof information according to the challenge information sequence , , ; The trusted third-party organization performs integrity verification and calculates... Whether the equation holds true or not, e is a bilinear pairing mapping. If the equation holds true, the integrity verification is passed; if it does not hold true, the integrity verification is failed, and the cloud server refuses to use the model uploaded by the client for aggregation. In step 5, the third-party trusted agency verifies the correctness of the aggregation result in combination with each client, specifically including: The kth client divides the encrypted local model parameters into n data blocks, and each block is divided into s data segments to obtain ; The kth client computing tag and signature , g is a generator of a cyclic group G, G1, H is a one-way hash function; wherein , and n is an integer; The kth client sends the label and signature to the cloud server; The third-party trusted authority uses a random number generator to generate a challenge information sequence composed of c integers 1, 2, …, c} and sends the challenge information to each of the clients; wherein 1 < c < n; Each client generates proof information based on challenge information , , Send to the third party trusted agency; The third-party trusted authority calculates If the equation is established, it means that the aggregation result is correct, and if the equation is not established, it means that the cloud server performs malicious aggregation.

2. The method of claim 1, wherein, The kth client locally trains a model, compresses the local model using a gradient sparsification-based model compression algorithm to obtain Sparse(w a,k ), and specifically includes: Step 2.1, the kth client randomly draws a batch of training samples from the local dataset, where batch k is the batch size, and batch k is greater than 0 and less than or equal to the number of data samples owned by the client. The 2.2, the kth client carries out local model training, and obtains gradient parameters ∇w ,k Then, using an optimization algorithm, the weight parameters w of the local model are calculated k ; 2.

3. The kth client downloads the sparsification rate from the cloud server ; 2.

4. the kth client according to the sparsification rate model gradient threshold value ; greater than the minimum value of the absolute value of the model gradient and less than the maximum value of the absolute value of the model gradient The absolute value of the model parameter of the 2.5th, kth client selection model exceeds Sparse(w a,k ) and the sparsified model parameter index Idx k .

3. The method of claim 2, wherein, The optimization algorithm specifically includes: gradient descent algorithm, momentum gradient descent algorithm, SGD algorithm, Adam algorithm, Adagrad algorithm or RMSProp.

4. The method of claim 1, wherein, The kth client uses a homomorphic encryption method to encrypt the sparse model to obtain Enc(Sparse(w a,k )) In the method, the homomorphic encryption algorithm specifically comprises a CKKS algorithm, a Paillier algorithm or an RSA algorithm.

5. The method of claim 1, wherein, The auxiliary verification information is generated according to a verification protocol based on the BLS signature, specifically including: In step 5.1, the kth client divides the encrypted sparse model parameters into n data blocks, and each block is divided into s data segments to obtain ; 5.2, kth client computing tag and signature g is a generator of the cyclic group G, G1, H is a one-way hash function; and n is an integer; The 5th.3, the kth client sends Enc(Sparse(w a,k )), , the mark and signature to the cloud server.

6. The method of claim 1, wherein, The type of the model includes: neural network model, convolutional neural network model, LSTM neural network model, GAN neural network model or deep belief neural network model.

7. The method according to any one of claims 1 to 6, wherein The input data corresponding to the model includes: text, picture, voice or video.

8. A computer device comprising a memory, a processor, and a computer program stored on the memory and executable on the processor, wherein, The processor implements the method of any one of claims 1-6 when executing the program.

Images