A Federated Learning Method and System Based on a Conditional Filtering Gradient Update Mechanism

Abstract

The present invention belongs to the field of network security technology and relates to a federated learning method and system based on a conditional filtering gradient update mechanism. The method includes that a trusted institution assigns conditional thresholds to participants, sends public key pairs and homomorphic hash functions to the participants, and assigns private key pairs and prime numbers of a homomorphic cryptosystem for non-interactive zero-knowledge proof to a cloud server. The cloud server issues an initial global model to the participants. The participants train on the global model and obtain local models by using the conditional filtering gradient update mechanism. The participants encrypt the parameters of the local models by using the public key pairs and upload them to the cloud server. The cloud server aggregates the uploaded local models according to a preset aggregation rule, decrypts the aggregation result by using the private key pairs to generate an average model, and issues the averaged global model to the participants. The present invention selects the most accurate and important gradient updates by using the conditional filtering gradient update mechanism, reduces the communication overhead, and improves the accuracy of gradient updates.

Description

Technical Field

[0001] The present invention belongs to the technical field of network security, and particularly relates to a federated learning method and system based on a conditional filtering gradient update mechanism. Background Art

[0002] In recent years, machine learning algorithms based on deep learning have been widely applied in many tasks, such as speech recognition, image recognition, and autonomous driving. To pursue higher performance, deep learning increasingly relies on a large amount of training data. However, with the increasing emphasis on data privacy by the public, the practice of collecting all participants' data to form training data has become more challenging. To address this challenge, a series of privacy-preserving deep learning frameworks have emerged. Among these frameworks, federated learning has become one of the mainstream systems, which provides a way to perform collaborative training without uploading local data. The traditional federated learning architecture adopts a centralized design, which includes a central server and multiple participants. The participants use their local data to train the model and upload the obtained model updates to the central server. After aggregating all participants' parameter updates, the central server will perform aggregation and return the global model to the participants. The participants then adjust their local models according to the global model returned by the central server. This process is iterated in multiple rounds until the global model reaches the expected performance.

[0003] Although federated learning provides the advantage of privacy protection for data by enabling participants to perform collaborative training without centralized data, it still faces some challenges. One of them is the security issue of the central server. If the central server fails or is maliciously attacked, the entire federated learning process may be interrupted. Existing research usually assumes that the central server is honest or semi-honest and does not verify the security of the global model. If an attacker obtains the global model, they may profit from it. More importantly, once the central server is controlled by an attacker, they may disrupt the training task. Even if the central server performs tasks normally, it may infer a large amount of sensitive information from the gradients of model updates through privacy inference attacks.

[0004] To solve the above problems, distributed federated learning is an effective solution. This method fundamentally solves the problems of single-point server failure and malicious behavior. However, this framework also faces challenges in privacy protection and model quality detection technologies. Therefore, more effective methods are needed to protect the data privacy and model reliability of the distributed federated learning framework to ensure the security and reliability of the federated learning system. Summary of the Invention

[0005] To overcome the deficiencies of the above-mentioned existing technologies, the purpose of the present invention is to provide a federated learning method and system based on a conditional filtering gradient update mechanism, which filters gradient updates based on three conditions: availability, sensitivity, and relevance, and protects the privacy of participants based on a non-interactive zero-knowledge proof homomorphic cryptosystem (NIZKP-HC), while significantly improving communication efficiency.

[0006] In the first aspect of the present invention, the present invention provides a federated learning method based on a conditional filtering gradient update mechanism, and the method includes:

[0007] The trusted institution assigns conditional thresholds to each participant;

[0008] The trusted institution selects two homomorphic hash functions for each participant, assigns the private key pair and the prime number of the non-interactive zero-knowledge proof homomorphic cryptosystem to the cloud server, and sends the public key pair, the two homomorphic hash functions, and the product of the prime numbers of the non-interactive zero-knowledge proof homomorphic cryptosystem to each participant;

[0009] The cloud server distributes the initialized global model to all participants;

[0010] Each participant trains on the received global model using its local dataset, performs local gradient updates using the distributed selective stochastic gradient descent optimization method, and obtains a local model using the conditional filtering gradient update mechanism;

[0011] Each participant encrypts the parameters of the local model based on the product of the prime numbers of the non-interactive zero-knowledge proof homomorphic cryptosystem using the public key pair and uploads them to the cloud server;

[0012] The cloud server decrypts the local models uploaded by each participant using the private key, aggregates the decrypted local models according to the preset aggregation rule, and distributes the aggregated global model to all participants.

[0013] In the second aspect of the present invention, the present invention also provides a federated learning system based on a conditional filtering gradient update mechanism, including a trusted institution, a cloud server, and each participant;

[0014] The trusted institution is used to assign conditional thresholds to each participant; and is used to select two homomorphic hash functions for each participant, assign the private key pair and the prime number of the non-interactive zero-knowledge proof homomorphic cryptosystem to the cloud server, and send the public key pair, the two homomorphic hash functions, and the product of the prime numbers of the non-interactive zero-knowledge proof homomorphic cryptosystem to each participant;

[0015] The cloud server is used to distribute the initialized global model to all participating parties; and is used to decrypt the local models uploaded by each participating party using the private key, aggregate the decrypted local models according to a preset aggregation rule, and distribute the aggregated global model to all participating parties.

[0016] Each of the participating parties is used to train on the global model received by it using a local dataset, perform local gradient updates using the distributed selective stochastic gradient descent optimization method, and obtain a local model using a conditional filtering gradient update mechanism; and encrypt the parameters of the local model using a public key and upload them to the cloud server after encrypting with a prime number of a homomorphic cryptosystem based on non-interactive zero-knowledge proof.

[0017] Advantages of the present invention:

[0018] The present invention overcomes the defects of low system security and reliability and high communication overhead cost existing in the aforementioned prior art, adopts a new filtering mechanism for gradient update to select the most accurate and important gradient updates to reduce communication overhead and improve the accuracy of gradient update, uses a homomorphic cryptosystem based on non-interactive zero-knowledge proof (NIZKP-HC) to encrypt and protect the privacy of the participating parties, and finally realizes DSSGD optimization during the local model training process to minimize the computational cost. Description of the drawings

[0019] Figure 1 It is a schematic diagram of the federated learning framework provided by an embodiment of the present invention;

[0020] Figure 2 It is a flowchart of the federated learning method provided by an embodiment of the present invention;

[0021] Figure 3 It is a structural diagram of the federated learning system provided by an embodiment of the present invention. Detailed implementation manners

[0022] Next, the technical solutions in the embodiments of the present invention will be clearly and completely described in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of the present invention.

[0023] Figure 1 It is a schematic diagram of the federated learning framework provided by an embodiment of the present invention, as Figure 1As shown in the figure, the federated learning framework of the present invention includes several core processes: device initialization, local gradient update, filtering local gradient updates, encrypting gradient updates, model aggregation, model decryption, and training round judgment; when the number of training times has not reached the maximum number of iterations, it is necessary to return to perform local gradient updates until the iterative training of the model is completed.

[0024] Figure 2 is a flowchart of a federated learning method based on a conditional filtering gradient update mechanism provided by an embodiment of the present invention. As Figure 2 shown, the method includes:

[0025] 101. The trusted authority assigns conditional thresholds to each participating party;

[0026] In the embodiment of the present invention, as the system initialization stage, the trusted authority needs to assign corresponding conditional thresholds to each participating party. The conditional thresholds are used to assist gradient updates in subsequent processes. The conditional thresholds include an availability conditional threshold, a sensitivity conditional threshold, and a correlation conditional threshold. For the availability conditional threshold, the availability conditions include resources such as battery power, memory, bandwidth, and processing power. For the sensitivity conditional threshold, the sensitivity conditions include location or emergency data. If the gradient contains sensitive data, the update will not wait any longer and will be uploaded immediately after encryption. For the correlation conditional threshold, the correlation conditions include the correlation of application-based data. If the gradient is relevant, the participant uploads the gradient update after encryption; otherwise, the participant discards the gradient update and waits for the next round.

[0027] 102. The trusted authority selects two homomorphic hash functions for each participating party, assigns the private key pair and the prime number of the homomorphic cryptosystem for non-interactive zero-knowledge proof to the cloud server, and sends the public key pair, the two homomorphic hash functions, and the product of the prime numbers of the homomorphic cryptosystem for non-interactive zero-knowledge proof to each participating party;

[0028] In the embodiment of the present invention, the server initializes the homomorphic encryption scheme and generates two pairs of keys: (pk 1 , sk 1 ) and (pk 2 , sk 2 ), where pk 1 and pk 2 are public keys, and sk 1 and sk 2 are the corresponding private keys; the public key pair (pk 1 , pk 2 ) is sent to each participating party, and then the homomorphic hash function H and y, z are sent to each participating party; the master key mk = (F(sk 1 ), F(sk 2), y, z) is assigned to the cloud server, where y and z are two large prime numbers of the homomorphic cryptosystem for non-interactive zero-knowledge proof (NIZKP-HC);

[0029] Among them, for the initialization process of the homomorphic cryptosystem for non-interactive zero-knowledge proof (NIZKP-HC), first, the number of digits of the prime numbers needs to be determined. To ensure the security of the RSA algorithm, the number of digits of the prime numbers y and z must be large enough. The RSA algorithm is an asymmetric cryptographic algorithm that uses a pair of keys: a public key and a private key. The public key is used to encrypt data, and the private key is used to decrypt data. The security of the RSA algorithm is based on the difficulty of large number factorization and prime number detection. In the RSA algorithm, two large prime numbers y and z are selected, n = y × z is calculated, and φ(n) = (y - 1)(z - 1). Then an integer e is selected, 1 < e < φ(n), and e is relatively prime to φ(n). e is used as part of the public key. Then the modular multiplicative inverse d of e with respect to φ(n) is calculated, and d is used as the private key. The encryption process is C = M^e mod n, where M is the plaintext and C is the ciphertext. The decryption process is M = C^d mod n. The advantages of the RSA algorithm include its wide range of application scenarios, such as public key encryption, digital signatures, etc., and the algorithm is public, which is convenient for wide application and development.

[0030] In the embodiment of the present invention, the number of digits of each prime number is approximately 1024 bits. Secondly, candidate prime numbers need to be randomly generated; a high-quality random number generator is used to generate large integers as candidate prime numbers. The generator needs to be able to generate random numbers large enough to meet the security requirements. Then prime number tests are required. The prime numbers generated by the generator are subjected to the Miller-Rabin primality test and the Lucas-Lehmer prime number test to verify the primality of the generated prime numbers. Finally, the above steps are repeated until prime numbers that meet the requirements are found.

[0031] 103. The cloud server distributes the initialized global model to all participating parties;

[0032] In the embodiment of the present invention, the cloud server distributes the initialized global model to all participating parties; in the subsequent update and iteration process, the cloud server can then distribute a new round of the global model to all participating parties until the iterative training ends.

[0033] 104. Each participating party trains on the global model received by it using the local dataset, performs local gradient updates using the distributed selective stochastic gradient descent optimization method, and obtains a local model using the conditional filtering gradient update mechanism;

[0034] In the embodiment of the present invention, first, the large dataset needs to be divided into multiple small datasets, and the corresponding small datasets are distributed to the participating parties. Each participating party executes the stochastic gradient descent optimization method on the local small dataset, calculates the gradient, and updates the local model parameters.

[0035] In the embodiments of the present invention, using the distributed selective stochastic gradient descent optimization method for local gradient update includes dividing the gradient vector and the weight vector into n parts respectively; using the distributed selective stochastic gradient descent optimization method to perform n updates of the weight vector of the local model, and obtaining the updated weight vector after each update respectively; calculating the updated gradient vector according to the updated weight vector after each update and the preset learning rate; determining the parameters of the local model according to the n updates performed in this round, and uploading the local parameters determined by the n updates performed in this round to the cloud server for weighted averaging to obtain the parameters of the updated global model, where n represents the preset number of iterations in each round.

[0036] In the embodiments of the present invention, each participant uses the local dataset to perform machine learning model training on the received global model to obtain a local model; the local gradient update is obtained through distributed selective stochastic gradient descent (DSSGD) optimization, which aims to find the parameters and minimize the overall loss function. Then, in order to select the most accurate and important gradient updates of the participants, a filtering local gradient update mechanism is introduced to screen and update the gradients.

[0037] Specifically, using the distributed selective stochastic gradient descent (DSSGD) optimization method for local gradient update, specifically, the weight vector and the gradient vector are respectively divided into n parts, such as and can respectively represent the weights and models in machine learning. Each part of the weight vector and each part of the gradient vector can participate in the gradient update of the local model. Since different algorithms have different iteration methods, the iteration method of each vector in this embodiment is not specifically limited; the gradient update rule of the local model parameters is expressed as:

[0038]

[0039] where μ r+1 represents the local model updated by the participant in the (r + 1)-th communication round, and μ r represents the local model updated by the participant in the r-th communication round, and η represents the learning rate.

[0040] In the embodiments of the present invention, the conditional filtering gradient update mechanism includes:

[0041] If the available resources of the participant are less than the availability condition threshold, the gradient of this update is discarded and does not participate in future gradient updates. If the available resources of the participant are equal to or greater than the availability condition threshold, the participant checks the sensitivity of the gradient data;

[0042] For availability detection, the participating party checks its own availability, such as resources like battery power, memory, bandwidth, and processing capacity. The participating party measures the available resources of the device. If the available resources are less than the corresponding available resource condition threshold, it indicates that the available resources are insufficient. Then the participating party immediately discards the gradient of this update and does not participate in future gradient updates.

[0043] If the sensitivity of the gradient data checked by the participating party exceeds the sensitivity condition threshold, the participating party immediately encrypts the gradient update and uploads it to the cloud server. If the sensitivity of the gradient data checked by the participating party does not exceed the sensitivity condition threshold, the participating party checks the correlation of the gradient data;

[0044] For sensitivity detection, the participating party checks whether the gradient data contains sensitive information. If the gradient contains sensitive data, the update will not wait any longer and will be uploaded immediately after encryption.

[0045] In the embodiments of the present invention, different from the conventional threshold, the sensitivity condition threshold of the present invention is a binary sequence. If there is a client with pre-set specific location information or emergency data, it is 1 and the update is directly uploaded. Otherwise, it is 0.

[0046] If the correlation between the current gradient data checked by the participating party and the previous gradient data exceeds the correlation condition threshold, the participating party immediately encrypts the gradient update and uploads it to the cloud server. If the correlation between the current gradient data checked by the participating party and the previous gradient data does not exceed the correlation condition threshold, the participating party discards this gradient update and waits for the next round of gradient data.

[0047] For correlation detection, the participating party checks the correlation between the current gradient data and the previous gradient data. If this correlation exceeds the correlation condition threshold, it indicates that the gradient data may be unqualified, so this data needs to be discarded.

[0048] The participating party compares the calculated gradient with the threshold assigned by the trusted authority (TA). To measure the difference between these two gradient updates, we calculate the gradient update difference in the (r + 1)-th round:

[0049]

[0050] where, |μ r+1 (new)| It represents the gradient update difference of the (r + 1)-th round of the local model. At this time, the gradient update difference is compared with the correlation condition threshold. If the difference does not exceed the correlation condition threshold, it indicates that the gradient data corresponding to the (r + 1)-th round of the local model may be unqualified, so this data needs to be discarded. Based on the above conditional filtering gradient update mechanism, some unnecessary gradient updates can be filtered out, and participants can update according to the most accurate and important gradients, improving the accuracy of gradient updates, enhancing the timeliness of model iteration, and reducing communication overhead.

[0051] When the participating party has made a gradient update decision, the participating party uses the public key pk and executes the encryption of NIZKP-HC to generate the ciphertext for its gradient vector Δ g The public key here can be pk 1 It can also be pk 2 , and the ciphertext is calculated as follows:

[0052]

[0053] where represents the ciphertext generated by the local gradient vector of the j-th participating party in the (r + 1)-th communication round; represents the plaintext of the local gradient vector of the j-th participating party in the (r + 1)-th communication round, ν r represents the random number used for homomorphic encryption in the r-th communication round, and n represents the modulus in the homomorphic encryption system, that is, the product of two large prime numbers x and y of the homomorphic cryptosystem of non-interactive zero-knowledge proof.

[0054] 105. Each participating party encrypts the parameters of the local model using the public key based on the prime product of the homomorphic cryptosystem of non-interactive zero-knowledge proof and uploads them to the cloud server;

[0055] It can be understood that since the encryption process is not the focus of the present invention, the present invention does not introduce it in detail here. Those skilled in the art can implement the above encryption process using existing technologies. 106. The cloud server decrypts the local models uploaded by each participating party using the private key, aggregates the decrypted local models according to the preset aggregation rules, and distributes the aggregated global model to all participating parties.

[0056] In the embodiment of the present invention, the participating party j sends the ciphertext to the cloud server for secure aggregation; once all participating parties upload all their encrypted gradient updates ξ r+1 to the cloud server in the (r + 1)-th communication round, the cloud server first aggregates the encrypted gradients, and the aggregation formula is expressed as:

[0057]

[0058] In the embodiment of the present invention, the cloud server uses the NIZKP-HC key sk = {(F(sk 1 ), F(sk 2 )} to decrypt the aggregation result, which is expressed as:

[0059]

[0060] where w r+1 represents the globally aggregated decrypted model parameters of the cloud server in the (r + 1)-th communication round; ξ r+1 represents the number of participants in the (r + 1)-th communication round, and μ r+1 represents the locally updated model parameters of the participants in the (r + 1)-th communication round; F(sk) represents the decryption function in NIZKP-HC. If the public key used by the participant for encryption using NIZKP-HC is pk 1 , then the corresponding private key sk 1 is used. If the public key used by the participant for encryption using NIZKP-HC is pk 2 , then the corresponding private key sk 2 is used; represents the aggregated ciphertext in the (r + 1)-th communication round, represents the ciphertext generated by the local gradient vector of the j-th participant in the (r + 1)-th communication round; represents the aggregated plaintext gradient in the (r + 1)-th communication round, represents the plaintext of the local gradient vector of the j-th participant in the (r + 1)-th communication round; ν r represents the random number used for homomorphic encryption in the r-th communication round, and n represents the modulus in the homomorphic encryption system, that is, the product of two large prime numbers x and y in the homomorphic cryptosystem of non-interactive zero-knowledge proof.

[0061] Through the above operations, the globally aggregated model can be obtained, and the globally aggregated model can be distributed to each participant for each participant to continue local training.

[0062] Figure 3 is the structural diagram of the federated learning system provided by the embodiment of the present invention. As Figure 3 shown, the federated learning system includes a trusted institution, a cloud server, and each participant;

[0063] The trusted institution is used to allocate conditional thresholds to each participant; and is used to select two homomorphic hash functions for each participant, allocate the private key pair and the prime numbers of the homomorphic cryptosystem of non-interactive zero-knowledge proof to the cloud server, and send the public key pair, the two homomorphic hash functions, and the product of the prime numbers of the homomorphic cryptosystem of non-interactive zero-knowledge proof to each participant;

[0064] The cloud server is configured to issue an initial global model to all participating parties; and to decrypt the local models uploaded by each participating party using a private key, aggregate the decrypted local models according to a preset aggregation rule, and issue the aggregated global model to all participating parties.

[0065] Each of the participating parties is configured to train on the received global model using a local dataset, perform local gradient updates using a distributed selective stochastic gradient descent optimization method, and obtain a local model using a conditional filtering gradient update mechanism; and to encrypt the parameters of the local model using a public key with a prime number of a homomorphic cryptosystem based on non-interactive zero-knowledge proof and upload it to the cloud server.

[0066] Although embodiments of the present invention have been shown and described, it will be understood by those of ordinary skill in the art that various changes, modifications, substitutions, and variations can be made to these embodiments without departing from the principles and spirit of the present invention, and the scope of the present invention is defined by the appended claims and their equivalents.

Claims

1. A federated learning method based on conditional filtering gradient update mechanism, characterized in that: The method comprises: The trusted institution assigns condition thresholds to each participant; the condition thresholds include availability condition thresholds, sensitivity condition thresholds and relevance condition thresholds; The trusted institution selects two homomorphic hash functions for each participant, distributes the private key pair and the prime number of the homomorphic cryptographic system of the non-interactive zero-knowledge proof to the cloud server, and sends the product of the public key pair, the two homomorphic hash functions and the prime number of the homomorphic cryptographic system of the non-interactive zero-knowledge proof to each participant; The cloud server sends the initialized global model to all participants; Each participant uses a local data set to train on the global model it receives, uses a distributed selective stochastic gradient descent optimization method to perform local gradient updates, and uses a conditional filtering gradient update mechanism to obtain a local model; the conditional filtering gradient update mechanism includes: If the available resources of the participant are less than the availability condition threshold, the participant discards the gradient of this update and does not participate in future gradient updates. If the available resources of the participant are equal to or greater than the availability condition threshold, the participant checks the sensitivity of the gradient data; If the sensitivity of the gradient data checked by the participant exceeds the sensitivity condition threshold, the participant immediately encrypts the gradient update and uploads it to the cloud server. If the sensitivity of the gradient data checked by the participant does not exceed the sensitivity condition threshold, the participant checks the relevance of the gradient data; If the correlation between the current gradient data checked by the participant and the previous gradient data exceeds the correlation condition threshold, the participant will immediately encrypt the gradient update and upload it to the cloud server. If the correlation between the current gradient data checked by the participant and the previous gradient data does not exceed the correlation condition threshold, the participant will discard the updated gradient and continue to detect the sensitivity of the gradient data. Each participant uses the public key to encrypt the prime product of the parameters of the local model based on the homomorphic cryptographic system of non-interactive zero-knowledge proof and uploads it to the cloud server; The cloud server uses the private key to decrypt the local models uploaded by each participant, aggregates the decrypted local models according to the preset aggregation rules, and sends the aggregated global model to all participants.

2. A federated learning method based on a conditional filtering gradient update mechanism according to claim 1, characterized in that: The method of using the distributed selective stochastic gradient descent optimization method to perform local gradient updating includes dividing the gradient vector and the weight vector into n' parts respectively; performing n' weight vector updates of the local model using the distributed selective stochastic gradient descent optimization method to obtain the weight vectors after each update respectively; and calculating the updated gradient vector according to the weight vector after each update and a preset learning rate; The parameters of the local model are determined according to the n' updates performed in this round, and the local parameters determined by the n' updates performed in this round are uploaded to the cloud server for weighted averaging to obtain the updated parameters of the global model, where n' represents the preset number of iterations in each round.

3. A federated learning method based on a conditional filtering gradient update mechanism according to claim 1, characterized in that: The sensitivity condition threshold is a binary sequence, and the binary sequence indicates a participant of specific location information or emergency data.

4. A federated learning method based on a conditional filtering gradient update mechanism according to claim 1, characterized in that: Each participant uses a public key to encrypt the prime product of the parameters of the local model based on the homomorphic cryptographic system of non-interactive zero-knowledge proof and then uploads it to the cloud server, including the participant using the public key to perform encryption based on the homomorphic cryptographic system of non-interactive zero-knowledge proof and generate a ciphertext of the gradient vector, and the ciphertext is calculated as follows: in, represents the ciphertext generated by the local gradient vector of the jth participant in the r+1th communication round; represents the plaintext of the local gradient vector of the jth participant in the r+1th communication round, ν r represents the random number used for homomorphic encryption in the rth communication round, and n represents the modulus in the homomorphic encryption system, that is, the product of two large prime numbers x and y in the homomorphic cryptographic system of non-interactive zero-knowledge proof.

5. The federated learning method based on conditional filtering gradient update mechanism according to claim 1, characterized in that: The calculation method used to send the averaged global model to all participants includes: Among them, w r+1 represents the global model parameters after cloud server aggregate decryption in the r+1th communication round; ξ r+1 represents the number of participants in the r+1th communication round, μ r+1 represents the local model parameters updated by the participants in the r+1th communication round; F(sk) represents the decryption function in NIZKP-HC; represents the ciphertext after aggregation in the r+1th communication round, represents the ciphertext generated by the local gradient vector of the jth participant in the r+1th communication round; represents the aggregated gradient plaintext in the r+1th communication round, represents the plaintext of the local gradient vector of the jth participant in the r+1th communication round; ν r represents the random number used for homomorphic encryption in the rth communication round, and n represents the modulus in the homomorphic encryption system, that is, the product of two large prime numbers x and y in the homomorphic cryptographic system of non-interactive zero-knowledge proof.

6. A federated learning system based on a conditional filtering gradient update mechanism, characterized in that: Including trusted institutions, cloud servers and various participants; The trusted institution is used to assign conditional thresholds to each participant; and for selecting two homomorphic hash functions for each participant, allocating the prime numbers of the homomorphic cryptographic system of the private key pair and the non-interactive zero-knowledge proof to the cloud server, and sending the product of the public key pair, the two homomorphic hash functions and the prime numbers of the homomorphic cryptographic system of the non-interactive zero-knowledge proof to each participant; the condition thresholds include an availability condition threshold, a sensitivity condition threshold and a correlation condition threshold; The cloud server is used to send an initialized global model to all participants; and to decrypt the local models uploaded by each participant using a private key, aggregate the decrypted local models according to a preset aggregation rule, and send the aggregated global model to all participants; Each participant is used to use a local data set to train on the global model it receives, use a distributed selective stochastic gradient descent optimization method to perform local gradient updates, and use a condition-based filtering gradient update mechanism to obtain a local model; and use a public key to encrypt the parameters of the local model based on the prime number of a homomorphic cryptographic system of a non-interactive zero-knowledge proof and upload them to a cloud server; The conditional filtering gradient update mechanism includes: If the available resources of the participant are less than the availability condition threshold, the participant discards the gradient of this update and does not participate in future gradient updates. If the available resources of the participant are equal to or greater than the availability condition threshold, the participant checks the sensitivity of the gradient data; If the sensitivity of the gradient data checked by the participant exceeds the sensitivity condition threshold, the participant immediately encrypts the gradient update and uploads it to the cloud server. If the sensitivity of the gradient data checked by the participant does not exceed the sensitivity condition threshold, the participant checks the relevance of the gradient data; If the correlation between the current gradient data checked by the participant and the previous gradient data exceeds the correlation condition threshold, the participant will immediately encrypt the gradient update and upload it to the cloud server. If the correlation between the current gradient data checked by the participant and the previous gradient data does not exceed the correlation condition threshold, the participant will discard the updated gradient and continue to detect the sensitivity of the gradient data.

Images