Distributed federated learning method based on homomorphic encryption and air computing

Abstract

The invention belongs to the technical field of federated learning, and particularly relates to a distributed federated learning method based on homomorphic encryption and air computing, which is characterized in that a client carries out homomorphic encryption on locally trained model parameters, carries out wireless communication preprocessing on ciphertexts and then uploads the ciphertexts to a parameter server; the model parameters of all the clients are aggregated through air calculation, and the clients locally process the aggregation result and then decrypt the aggregation result to obtain an updated global model. According to the method, homomorphic mapping processing is constructed for encryption parameters, and the processed parameter ciphertext is calculated and transmitted in the air, so that aggregation and communication time delay of federal learning starting homomorphic encryption is greatly reduced, and efficient and privacy-protected federal learning is realized.

Description

Technical Field

[0001] The present invention belongs to the technical field of federated learning, and in particular relates to a distributed federated learning method based on homomorphic encryption and over-the-air computing. Background Art

[0002] Federated learning is a distributed machine learning method that allows training data to remain local to the client, thereby ensuring data privacy. Federated learning has become increasingly popular in many applications such as healthcare, smart transportation, and smart cities. However, the large amount of data exchange between clients and parameter servers poses a significant threat to data privacy and security. Previous studies have shown that the local data of distributed clients may be leaked to attackers or untrusted parameter servers through model updates.

[0003] In response to the above problems, the use of homomorphic encryption for privacy protection in federated learning has become a promising research. Homomorphic encryption is a cryptographic technique that can perform calculations on encrypted data to ensure the privacy and security of the data. There are two main challenges in enabling homomorphic encryption technology in current federated learning: one is that the computational overhead of encryption, decryption, and homomorphic operations is too high, and the other is that the communication overhead is too high. And as the number of clients participating in federated learning increases, these performance bottlenecks become more serious, resulting in a decrease in system efficiency. Summary of the invention

[0004] The purpose of the present invention is to propose a distributed federated learning method based on homomorphic encryption and over-the-air computing, which can simultaneously reduce the computational and communication overheads of enabling homomorphic encryption.

[0005] The present invention is achieved through the following technical solutions:

[0006] A distributed federated learning method based on homomorphic encryption and air computing, characterized by:

[0007] Step S1: Each client uses a local data set to perform model training to obtain model parameters for this round of training, and locally encrypts the model parameters using the CKKS homomorphic encryption algorithm. Each client performs ciphertext-oriented wireless communication preprocessing on the encrypted model parameters locally, and uploads the preprocessed encrypted model parameters to the parameter server.

[0008] Step S2: Utilize the channel superposition of the multi-access channel to perform air calculation on the pre-processed encryption model parameters of each client, and the parameter server performs beamforming on the results of the air calculation to obtain an aggregated result. The data remains encrypted during the entire process.

[0009] Step S3: The parameter server broadcasts the aggregated results to all clients. The clients perform post-processing of the aggregated results via wireless communication locally to obtain the ciphertext form of the aggregated global model. The ciphertext is decrypted using the CKKS homomorphic encryption algorithm to obtain the global model parameters in the decrypted state for the next round of training.

[0010] Step S4: In order to accelerate the convergence speed of the federated learning of air computing with homomorphic encryption enabled, a problem of minimizing the optimization gap of federated learning based on decryption approximation error is constructed. Under the conditions of known channel state and transmit power constraint, the transmit and receive beamforming matrices are optimized so that the optimization gap of federated learning based on decryption error is minimized. A semi-closed form solution can be obtained by alternately performing block coordinate descent on each column of the transmit and receive beamforming matrices;

[0011] Step S5: Repeat steps S1 to S4 several times until the global model converges.

[0012] Furthermore, the step S1 specifically comprises the following steps:

[0013] The CKKS homomorphic encryption algorithm is used to encrypt the model parameters of the local training. For the model parameters θ obtained by client k in this round of training, k , divide it into several vectors of length N Where N is the polynomial degree of the CKKS algorithm, M is the number of partitioned vectors, and these partitioned vectors are encrypted to obtain the corresponding ciphertext in Represents the circle domain Module q L The remaining ring of It can be expressed as a combination of polynomials on the remainder ring of two cyclotomic domains

[0014] Before each client transmits the ciphertext, wireless communication preprocessing for the CKKS ciphertext is performed. The preprocessing method is: construct an inverse homomorphic mapping for in is composed of N unit root vectors The Vandermonde matrix is ​​composed of Right now Represents an integer that is relatively prime to 2N, and the ciphertext to be transmitted of Respectively by the inverse homomorphism mapping ρ -1 Map to Client k will encrypt the preprocessed model parameters and m∈{1,…,M} is uploaded to the parameter server.

[0015] Furthermore, in step S2, the pre-processed encryption model parameters are transmitted respectively by air calculation. and Consider a multiple-input multiple-output system and assume that all clients achieve signal-level synchronization and all clients send signals at the same time. The signal is obtained by superimposing the wireless channel in the air Among them, H k is the channel state information between client k and parameter server, which obeys independent and identically distributed complex Gaussian distribution, that is, B k is the beamforming matrix at the transmitting end, n 0 is additive white Gaussian noise, and the parameter server is used to calculate the superimposed signal Perform beamforming to obtain an estimate of the aggregation result Where A is the beamforming matrix of the parameter server, and The same method is used to perform air calculations to obtain superimposed signals. and the estimated value of the aggregation result Where D k and C represent the beamforming matrices of the transmitter and parameter server respectively, n 1 is additive white Gaussian noise, and the beamforming matrix B at the transmitter is k and D k Satisfy the transmit power constraint, that is, Where P 0 Indicates the maximum transmit power.

[0016] Furthermore, in step S3, the parameter server aggregates the estimated value of the result Broadcast to all clients, and the clients estimate the local value of the aggregation result Post-processing is performed by constructing a homomorphic mapping. for The estimated value of the aggregation result Homomorphic mapping to Where M is the number of partition model parameters, and the results of these partitions are combined to obtain the estimated value of the ciphertext of the global model Using CKKS homomorphic encryption algorithm Decrypt and get the estimated value of the global model for the next round of training.

[0017] Furthermore, in step S4, an optimization problem of minimizing the optimal gap of federated learning based on decryption approximation error is constructed. Specifically, the decryption approximation error δ is defined as in The private key used by the CKKS algorithm. is the estimated value of the global model ciphertext, represents the true value of the ciphertext of the global model, 〈·,·〉 is the inner product operation, and the decryption approximation error δ in the worst case is expressed by the canonical embedding norm To measure, is the Vandermonde matrix of the homomorphic mapping, The cumulative distribution function of is:

[0018]

[0019] according to The cumulative distribution function of δ obtains a high probability upper bound B in the worst case. δ for The optimal gap for federated learning is defined as Where F(·) is the global loss function, F ★ is the optimal value of the global loss function. According to the upper bound of the optimal gap, The relevant inequalities, construct the minimization The optimization problem is:

[0020]

[0021] in where · represents polynomial multiplication, and A and B are optimized alternately using block coordinate descent. k , C, D k , we obtain the semi-closed form solutions of the beamforming matrices at the transmitter and receiver.

[0022] The present invention has the following beneficial effects:

[0023] Compared with the existing federated learning with homomorphic encryption enabled, the CKKS homomorphic encryption algorithm makes the encryption and decryption process of federated learning more efficient, and the use of air computing technology significantly reduces the latency of the aggregation and communication process, and makes the latency of the aggregation and communication process independent of the number of participating clients. Compared with the existing federated learning with air computing, the homomorphic encryption technology is used to provide data privacy and security for federated learning. By constructing an optimization problem that minimizes the optimal gap of federated learning based on decryption approximation error, the learning accuracy and convergence speed of air computing federated learning with homomorphic encryption enabled are accelerated.

Claims

1. A distributed federated learning method based on homomorphic encryption and air computing, characterized by: include: Step S1: Each client uses a local data set to perform model training to obtain model parameters for this round of training, and locally encrypts the model parameters using the CKKS homomorphic encryption algorithm. Each client performs ciphertext-oriented wireless communication preprocessing on the encrypted model parameters locally, and uploads the preprocessed encrypted model parameters to the parameter server. Step S2: Utilize the channel superposition of the multi-access channel to perform air calculation on the pre-processed encryption model parameters of each client, and the parameter server performs beamforming on the results of the air calculation to obtain an aggregated result. The data remains encrypted during the entire process. Step S3: The parameter server broadcasts the aggregated results to all clients. The clients perform post-processing of the aggregated results via wireless communication locally to obtain the ciphertext form of the aggregated global model. The ciphertext is decrypted using the CKKS homomorphic encryption algorithm to obtain the global model parameters in the decrypted state for the next round of training. Step S4: In order to accelerate the convergence speed of the federated learning of air computing with homomorphic encryption enabled, a problem of minimizing the optimization gap of federated learning based on decryption approximation error is constructed. Under the conditions of known channel state and transmit power constraint, the transmit and receive beamforming matrices are optimized so that the optimization gap of federated learning based on decryption error is minimized. A semi-closed form solution can be obtained by alternately performing block coordinate descent on each column of the transmit and receive beamforming matrices; Step S5: Repeat steps S1 to S4 several times until the global model converges.

2. A distributed federated learning method based on homomorphic encryption and air computing according to claim 1, characterized in that: In step S1, the model parameters of local training are encrypted using the CKKS homomorphic encryption algorithm. For the model parameters θ obtained by client k in this round of training, k , divide it into several vectors of length N Where N is the polynomial degree of the CKKS algorithm, M is the number of partitioned vectors, and these partitioned vectors are encrypted to obtain the corresponding ciphertext in Represents the circle domain Module q L The remaining ring of It can be expressed as a combination of polynomials on the remainder ring of two cyclotomic domains 3. A distributed federated learning method based on homomorphic encryption and air computing according to claim 2, characterized in that: In step S1, before each client transmits the ciphertext, wireless communication preprocessing for the CKKS ciphertext is performed. The preprocessing method is: constructing an inverse homomorphic mapping for ρ -1 (c) = Wc, where is composed of N unit root vectors The Vandermonde matrix is ​​composed of Right now Represents an integer that is relatively prime to 2N, and the ciphertext to be transmitted of Respectively, by the inverse homomorphism mapping ρ -1 Map to Client k will encrypt the preprocessed model parameters and m∈{1,…,M} is uploaded to the parameter server.

4. A distributed federated learning method based on homomorphic encryption and air computing according to claim 3, characterized in that: In step S2, the encrypted model parameters after preprocessing are transmitted respectively by air calculation. and Consider a multiple-input multiple-output system and assume that all clients achieve signal-level synchronization and all clients send signals at the same time. The signal is obtained by superimposing the wireless channel in the air Among them, H k is the channel state information between client k and parameter server, which obeys independent and identically distributed complex Gaussian distribution, that is, B k is the beamforming matrix of the transmitter, n0 is the additive white Gaussian noise, and the parameter server is the superimposed signal y0 (m) Perform beamforming to obtain an estimate of the aggregation result Where A is the beamforming matrix of the parameter server, and The same method is used to perform air calculations to obtain superimposed signals. and the estimated value of the aggregation result Where D k and C represent the beamforming matrices of the transmitter and parameter server respectively, n1 is additive white Gaussian noise, and the beamforming matrix B of the transmitter is k and D k Satisfy the transmit power constraint, that is, Where P0 represents the maximum transmit power.

5. A distributed federated learning method based on homomorphic encryption and air computing according to claim 4, characterized in that: In step S3, the parameter server aggregates the estimated value of the result Broadcast to all clients, and the clients estimate the local value of the aggregation result Post-processing is performed by constructing a homomorphic mapping. for Where mod represents the modulus operation, which takes the estimated value of the aggregation result Homomorphic mapping to m∈{1,…,M}, where M is the number of partition model parameters, and the results of these partitions are combined to obtain the estimated value of the ciphertext of the global model Using CKKS homomorphic encryption algorithm Decrypt and get the estimated value of the global model for the next round of training.

6. A distributed federated learning method based on homomorphic encryption and air computing according to claim 5, characterized in that: In step S4, an optimization problem of minimizing the optimal gap of federated learning based on decryption approximation error is constructed. Specifically, the decryption approximation error δ is defined as in The private key used by the CKKS algorithm. is the estimated value of the global model ciphertext, represents the true value of the ciphertext of the global model, 〈·,·〉 is the inner product operation, and the decryption approximation error δ in the worst case is expressed by the canonical embedding norm To measure, is the Vandermonde matrix for homomorphic mapping, The cumulative distribution function of is: according to The cumulative distribution function of δ obtains a high probability upper bound B in the worst case. δ for The optimal gap for federated learning is defined as Where F(·) is the global loss function, F ★ is the optimal value of the global loss function. According to the upper bound of the optimal gap, The relevant inequalities, construct the minimization The optimization problem is: in where · represents polynomial multiplication, and A and B are optimized alternately using block coordinate descent. k , C, D k , we obtain the semi-closed form solutions of the beamforming matrices at the transmitter and receiver.

Images