A distributed federated learning method based on homomorphic encryption and air computing

Abstract

The application belongs to the technical field of federated learning, and particularly relates to a distributed federated learning method based on homomorphic encryption and air computing, characterized in that: a client homomorphically encrypts model parameters of local training, uploads the ciphertext to a parameter server after preprocessing for wireless communication, aggregates model parameters of all clients by using air computing, and decrypts the updated global model after post-processing of the aggregation result at the client. The method constructs homomorphic mapping processing for encrypted parameters, and transmits the processed parameter ciphertext by using air computing, thereby greatly reducing the aggregation and communication time delay of federated learning with homomorphic encryption, and realizing efficient and privacy-protected federated learning.

Description

Technical Field

[0001] The present invention belongs to the technical field of federated learning, and specifically relates to a distributed federated learning method based on homomorphic encryption and over-the-air computing. Background Art

[0002] Federated learning is a distributed machine learning method that allows training data to remain local to the client, ensuring data privacy. Federated learning is becoming increasingly popular in numerous applications, including healthcare, intelligent transportation, and smart cities. However, the large amount of data exchanged between clients and parameter servers poses a significant threat to data privacy and security. Previous studies have shown that local data on distributed clients can be leaked to attackers or untrusted parameter servers through model updates.

[0003] To address these issues, the use of homomorphic encryption for privacy protection in federated learning has become a promising area of ​​research. Homomorphic encryption is a cryptographic technique that allows computations to be performed on encrypted data, thereby ensuring data privacy and security. Currently, implementing homomorphic encryption in federated learning faces two major challenges: high computational overhead in encryption, decryption, and homomorphic operations, and high communication overhead. Furthermore, these performance bottlenecks become more severe as the number of participating clients in federated learning increases, leading to reduced system efficiency. Summary of the Invention

[0004] The purpose of this invention is to propose a distributed federated learning method based on homomorphic encryption and over-the-air computing, which can simultaneously reduce the computational and communication overhead of enabling homomorphic encryption.

[0005] The present invention is achieved through the following technical solutions:

[0006] A distributed federated learning method based on homomorphic encryption and over-the-air computing, characterized by comprising:

[0007] Step S1: Each client uses a local dataset to perform model training to obtain the model parameters for this round of training. The model parameters are encrypted locally using the CKKS homomorphic encryption algorithm. Each client performs ciphertext-oriented wireless communication preprocessing on the encrypted model parameters locally and uploads the preprocessed encrypted model parameters to the parameter server.

[0008] Step S2: Utilizing the channel superposition of the multi-access channel, the pre-processed encryption model parameters of each client are calculated over the air. The parameter server performs beamforming on the results of the over-the-air calculation to obtain an aggregated result. The data remains encrypted throughout the entire process.

[0009] Step S3: The parameter server broadcasts the aggregated results to all clients. The clients perform wireless post-processing on the aggregated results locally to obtain the ciphertext form of the aggregated global model. The ciphertext is decrypted using the CKKS homomorphic encryption algorithm to obtain the global model parameters in the decrypted state for the next round of training.

[0010] Step S4: To accelerate the convergence of over-the-air federated learning with homomorphic encryption enabled, a problem for minimizing the optimization gap of federated learning based on decryption approximation error is constructed. Under the conditions of known channel state and transmit power constraints, the transmit and receive beamforming matrices are optimized to minimize the optimization gap of federated learning based on decryption error. A semi-closed form solution can be obtained by alternately performing block coordinate descent on each column of the transmit and receive beamforming matrices.

[0011] Step S5: Repeat steps S1 to S4 several times until the global model converges.

[0012] Furthermore, the step S1 specifically includes the following steps:

[0013] The CKKS homomorphic encryption algorithm is used to encrypt the model parameters of the local training. For the model parameters θ obtained by client k in this round of training, k , divide it into several vectors of length N Where N is the polynomial degree of the CKKS algorithm, M is the number of partitioned vectors, and these partitioned vectors are encrypted to obtain the corresponding ciphertext in Represents the cyclotomic domain Module q L The remaining ring of It can be expressed as a combination of polynomials on the remainder ring of two cyclotomic fields

[0014] Before each client transmits the ciphertext, wireless communication preprocessing for the CKKS ciphertext is performed. The preprocessing method is: constructing an inverse homomorphic mapping for in is composed of N unit root vectors The Vandermonde matrix is ​​composed of Right now Represents an integer that is relatively prime to 2N, and the ciphertext to be transmitted of Respectively by the inverse homomorphism mapping ρ -1 Map to Client k will be the pre-processed encrypted model parameters and m∈{1,…,M} is uploaded to the parameter server.

[0015] Furthermore, in step S2, the pre-processed encryption model parameters are transmitted respectively by air calculation. and Consider a MIMO system and assume that all clients achieve signal-level synchronization and all clients send signals at the same time. The signal is obtained by superimposing wireless channels in the air Among them H k is the channel state information between client k and parameter server, which obeys independent and identically distributed complex Gaussian distribution, that is, B k is the beamforming matrix of the transmitter, n0 is the additive white Gaussian noise, and the parameter server is the superimposed signal Perform beamforming to obtain an estimated value of the aggregation result Where A is the beamforming matrix of the parameter server, The same method is used to perform air calculations to obtain superimposed signals. and the estimated value of the aggregation result Among them D k and C represent the beamforming matrices of the transmitter and parameter server respectively, n1 is additive white Gaussian noise, and the beamforming matrix B of the transmitter is k and D k Satisfy the transmit power constraint, that is, Where P0 represents the maximum transmit power.

[0016] Furthermore, in step S3, the parameter server aggregates the estimated value of the result Broadcast to all clients, and the clients estimate the value of the aggregation result locally Perform post-processing, the post-processing method is: construct homomorphic mapping for The estimated value of the aggregation result Homomorphic mapping to Where M is the number of partition model parameters, and the results of these partitions are combined to obtain the estimated value of the ciphertext of the global model Using CKKS homomorphic encryption algorithm Decrypt and get the estimated value of the global model Used for the next round of training.

[0017] Furthermore, in step S4, an optimization problem is constructed to minimize the optimal gap of federated learning based on decryption approximation error. Specifically, the decryption approximation error δ is defined as in The private key used by the CKKS algorithm, is the estimated value of the global model ciphertext, represents the true value of the ciphertext of the global model, 〈·,·〉 is the inner product operation, and the decryption approximation error δ in the worst case is represented by the canonical embedding norm To measure, is the Vandermonde matrix of the homomorphic mapping, The cumulative distribution function of is:

[0018]

[0019] according to The cumulative distribution function of δ obtains a high probability upper bound B in the worst case δ for The optimal gap for federated learning is defined as Where F(·) is the global loss function, F ★ is the optimal value of the global loss function, and the upper bound of the optimal gap is Related inequalities, constructing the minimization The optimization problem is:

[0020]

[0021] in Where · represents polynomial multiplication, and A and B are optimized alternately using block coordinate descent. k , C, D k , we obtain the semi-closed form solutions of the beamforming matrices at the transmitter and receiver.

[0022] The present invention has the following beneficial effects:

[0023] Compared to existing federated learning systems using homomorphic encryption, the CKKS homomorphic encryption algorithm makes the encryption and decryption process more efficient. Furthermore, by leveraging over-the-air computing, the latency of aggregation and communication is significantly reduced, making the latency independent of the number of participating clients. Compared to existing federated learning systems using over-the-air computing, the use of homomorphic encryption ensures data privacy and security. By constructing an optimization problem to minimize the optimal gap in federated learning based on decryption approximation error, the learning accuracy and convergence speed of over-the-air federated learning with homomorphic encryption are accelerated.

Claims

1. A distributed federated learning method based on homomorphic encryption and over-the-air computing, characterized by: include: Step S1: Each client uses a local dataset to perform model training to obtain the model parameters for this round of training. The model parameters are encrypted locally using the CKKS homomorphic encryption algorithm. Each client performs ciphertext-oriented wireless communication preprocessing on the encrypted model parameters locally and uploads the preprocessed encrypted model parameters to the parameter server. Step S2: Utilizing the channel superposition of the multi-access channel, the pre-processed encryption model parameters of each client are calculated over the air. The parameter server performs beamforming on the results of the over-the-air calculation to obtain an aggregated result. The data remains encrypted throughout the entire process. Step S3: The parameter server broadcasts the aggregated results to all clients. The clients perform wireless post-processing on the aggregated results locally to obtain the ciphertext form of the aggregated global model. The ciphertext is decrypted using the CKKS homomorphic encryption algorithm to obtain the global model parameters in the decrypted state for the next round of training. Step S4: To accelerate the convergence of over-the-air federated learning with homomorphic encryption enabled, a problem for minimizing the optimization gap of federated learning based on decryption approximation error is constructed. Under the conditions of known channel state and transmit power constraints, the transmit and receive beamforming matrices are optimized to minimize the optimization gap of federated learning based on decryption error. A semi-closed form solution can be obtained by alternately performing block coordinate descent on each column of the transmit and receive beamforming matrices. Step S5: Repeat steps S1 to S4 several times until the global model converges.

2. The distributed federated learning method based on homomorphic encryption and over-the-air computing according to claim 1, characterized in that: In step S1, the model parameters of the local training are encrypted using the CKKS homomorphic encryption algorithm. For the model parameters θ obtained by the client k in this round of training, k , divide it into several vectors of length N Where N is the polynomial degree of the CKKS algorithm, M is the number of partitioned vectors, and these partitioned vectors are encrypted to obtain the corresponding ciphertext in Represents the cyclotomic domain Module q L The remaining ring of It can be expressed as a combination of polynomials on the remainder ring of two cyclotomic fields 3. The distributed federated learning method based on homomorphic encryption and over-the-air computing according to claim 2, characterized in that: In step S1, before each client transmits the ciphertext, wireless communication preprocessing for the CKKS ciphertext is performed. The preprocessing method is: constructing an inverse homomorphic mapping for ρ -1 (c) = Wc, where is composed of N unit root vectors The Vandermonde matrix is ​​composed of Right now Represents an integer that is relatively prime to 2N, and the ciphertext to be transmitted of Respectively by the inverse homomorphism mapping ρ -1 Map to Client k will be the pre-processed encrypted model parameters and m∈{1,…,M} is uploaded to the parameter server.

4. The distributed federated learning method based on homomorphic encryption and over-the-air computing according to claim 3, characterized in that: In step S2, the encrypted model parameters after preprocessing are transmitted respectively by air calculation. and Consider a MIMO system and assume that all clients achieve signal-level synchronization and all clients send signals at the same time. The signal is obtained by superimposing wireless channels in the air Among them H k is the channel state information between client k and parameter server, which obeys independent and identically distributed complex Gaussian distribution, that is, B k is the beamforming matrix of the transmitter, n0 is the additive white Gaussian noise, and the parameter server is the superimposed signal y0 (m) Perform beamforming to obtain an estimated value of the aggregation result Where A is the beamforming matrix of the parameter server, The same method is used to perform air calculations to obtain superimposed signals. and the estimated value of the aggregation result Among them D k and C represent the beamforming matrices of the transmitter and parameter server respectively, n1 is additive white Gaussian noise, and the beamforming matrix B of the transmitter is k and D k Satisfy the transmit power constraint, that is, Where P0 represents the maximum transmit power.

5. The distributed federated learning method based on homomorphic encryption and over-the-air computing according to claim 4, characterized in that: In step S3, the parameter server aggregates the estimated value of the result Broadcast to all clients, and the clients estimate the value of the aggregation result locally Perform post-processing, the post-processing method is: construct homomorphic mapping for Where mod represents the modulo operation, which aggregates the estimated value of the result Homomorphic mapping to m∈{1,…,M}, where M is the number of partition model parameters, and the results of these partitions are combined to obtain the estimated value of the ciphertext of the global model Using CKKS homomorphic encryption algorithm Decrypt and get the estimated value of the global model Used for the next round of training.

6. The distributed federated learning method based on homomorphic encryption and over-the-air computing according to claim 5, characterized in that: In step S4, an optimization problem of minimizing the optimal gap of federated learning based on decryption approximation error is constructed. Specifically, the decryption approximation error δ is defined as in The private key used by the CKKS algorithm, is the estimated value of the global model ciphertext, represents the true value of the ciphertext of the global model, 〈·,·〉 is the inner product operation, and the decryption approximation error δ in the worst case is represented by the canonical embedding norm To measure, is the Vandermonde matrix for homomorphic mapping, The cumulative distribution function of is: according to The cumulative distribution function of δ obtains a high probability upper bound B in the worst case δ for The optimal gap for federated learning is defined as Where F(·) is the global loss function, F ★ is the optimal value of the global loss function, and the upper bound of the optimal gap is Related inequalities, constructing the minimization The optimization problem is: in Where · represents polynomial multiplication, and A and B are optimized alternately using block coordinate descent. k , C, D k , we obtain the semi-closed form solutions of the beamforming matrices at the transmitter and receiver.

Images