Federal learning privacy protection method and system based on homomorphic encryption and incremental updating

Abstract

The invention provides a federated learning privacy protection method based on homomorphic encryption and incremental updating, and relates to the technical field of privacy computing. The federated learning privacy protection method based on homomorphic encryption and incremental updating is realized by a federated learning privacy protection system based on homomorphic encryption and incremental updating, and comprises the following steps: based on a federated learning framework, each participant adopts local training data to train a local initial model, and local initial model parameters are generated; an incremental updating mechanism is adopted, the difference value between the local model and the global model is calculated, the updated difference value of the model is obtained, the difference value of model parameters is subjected to fixed-point quantization, a quantized value is obtained, and the quantized value is encrypted by adopting an improved Paillier homomorphic encryption algorithm; and the aggregation server performs aggregation processing on the encrypted model parameters to obtain an updated encrypted global model. By adopting the method, the problems of privacy leakage risk and calculation bottleneck in the traditional method can be solved.

Description

Technical Field

[0001] The present invention relates to the technical field of privacy computing, and in particular to a privacy protection method and system for federated learning based on homomorphic encryption and incremental update. Background Art

[0002] With the rapid development of big data technology and artificial intelligence, data privacy and security issues have attracted wide attention. The traditional centralized data processing method requires data to be centralized on the server for processing, which is prone to leakage of sensitive information. In the field of privacy computing, how to effectively utilize distributed data for collaborative analysis and model training while ensuring data privacy is the current research focus.

[0003] Federated learning is a distributed machine learning framework. It allows each participating party to perform data training locally and upload the trained model parameters to the central server for aggregation, thus avoiding the transmission and storage of sensitive data. The advantage of federated learning is that data always remains local and does not require centralized storage and calculation, thereby reducing the risk of data privacy leakage to a certain extent. However, although federated learning can avoid exposing data to the central server, there are still some privacy risks, especially during the model aggregation and update process. Since the model parameters of the participating parties may contain sensitive information, how to protect these model parameters and how to effectively aggregate the model updates of different participating parties without leaking privacy are one of the main challenges faced by federated learning.

[0004] Homomorphic encryption is an encryption technology that allows operations to be performed on encrypted data without prior decryption. That is to say, the operation result of the ciphertext is the same as that of the plaintext under the same operation. Through homomorphic encryption, the participating party can encrypt the locally trained model parameters and upload them without exposing the plaintext data, and the central server can also perform aggregation calculations on the encrypted model parameters, avoiding the risk of sensitive information leakage. In federated learning, the use of homomorphic encryption can effectively protect the privacy of model parameters.

[0005] Although the existing privacy protection methods use homomorphic encryption technology to ensure the security of data and models to a certain extent, in large-scale federated learning, there are problems of large computational burden and low privacy protection security when optimizing the efficiency of encrypted computing, especially during incremental updates. Summary of the Invention

[0006] In order to solve the technical problems of large computational amount and reduced privacy protection security during incremental updates in large-scale federated learning existing in the prior art, the embodiments of the present invention provide a privacy protection method and system for federated learning based on homomorphic encryption and incremental update. The technical solutions are as follows:

[0007] On the one hand, a privacy protection method for federated learning based on homomorphic encryption and incremental update is provided. This method is implemented by a privacy protection device for federated learning based on homomorphic encryption and incremental update, and the method includes:

[0008] S1. Based on the federated learning framework, each participating party trains the local initial model using local training data to generate local initial model parameters;

[0009] S2. According to the initial model parameters, an incremental update mechanism is used to calculate the difference value of the initial model parameters; the initial model parameters are updated according to the difference value of the initial model parameters to obtain updated model parameters;

[0010] S3. An improved Paillier homomorphic encryption algorithm is used to encrypt the updated model parameters to obtain encrypted model parameters;

[0011] S4. Each participating party transmits the encrypted model parameters to the aggregation server, and the aggregation server performs aggregation processing on the encrypted model parameters to obtain aggregated encrypted model parameters; the aggregation server updates the global model according to the aggregated encrypted model parameters to obtain an updated encrypted global model.

[0012] Optionally, the process of calculating the difference value of the initial model parameters according to the initial model parameters by using an incremental update mechanism in S2 is represented by the following formula (1):

[0013] (1)

[0014] where represents the model parameters of the previous round; represents the current model parameters; represents the difference value of the initial model parameters.

[0015] Optionally, before the step of updating the initial model parameters according to the difference value of the initial model parameters in S2 to obtain updated model parameters, it further includes: using a fixed-point quantization technique to quantize the difference value of the initial model parameters to obtain a quantized difference value.

[0016] Optionally, the step of using an improved Paillier homomorphic encryption algorithm to encrypt the updated model parameters in S3 to obtain encrypted model parameters includes:

[0017] S31. Generate a key by dynamically pre-selecting a specified number of random numbers r;

[0018] S32. Calculate the pre-computed value h according to the random number r. The process of obtaining the pre-computed value h is represented by the following formula (2):

[0019] (2)

[0020] Among them, the pre-computed value h is saved together with the public key and the private key;

[0021] S33. According to the pre-computed value h, each participant uses the improved Paillier homomorphic encryption algorithm to encrypt the locally updated model parameters to obtain the encrypted model parameters. The encryption formula is represented by the following formula (3):

[0022] (3)

[0023] Among them, m represents the model parameters, r represents the pre-generated random number, h represents the pre-computed value, n represents a part of the encryption public key; c represents the encrypted model parameters.

[0024] Optionally, the aggregation server in S4 performs an aggregation process on the encrypted model parameters to obtain the aggregated encrypted model parameters, including:

[0025] The aggregation server sums up the encrypted model parameters to obtain the aggregated encrypted model parameters. The process of obtaining the aggregated encrypted model parameters is represented by the following formula (3):

[0026] (4)

[0027] Among them, represents the incremental update of the model of the i-th client; represents the incremental update of the global model after aggregation; i represents the i-th client.

[0028] Optionally, after the step that the aggregation server in S4 updates the global model according to the aggregated encrypted model parameters to obtain the updated encrypted global model, it further includes:

[0029] The aggregation server transmits the updated encrypted global model to each participant;

[0030] Each participant uses the private key and the improved Paillier homomorphic encryption algorithm to decrypt the encrypted parameters of the updated encrypted global model to obtain the decrypted global model; merges the local difference values according to the decrypted global model to update the local model; the participant uses the updated local model for the next round of training.

[0031] Optionally, each of the participating parties uses a private key and an improved Paillier homomorphic encryption algorithm to decrypt the encrypted parameters of the updated encrypted global model, obtaining the decrypted global model, including:

[0032] Decompose the modulus of the improved Paillier homomorphic encryption algorithm into multiple small moduli, perform decryption calculations on each small modulus, and obtain the decomposition corresponding to each small modulus;

[0033] Among them, the decryption process is represented by the following formula (5):

[0034] (5)

[0035] Among them, is an auxiliary function; is the first parameter of the private key; is the second parameter of the private key; is the modulus of the improved Paillier homomorphic encryption algorithm; represents the ciphertext; represents the decrypted plaintext;

[0036] Use the Chinese Remainder Theorem to combine the decompositions corresponding to each small modulus to obtain the final decryption result.

[0037] On the other hand, a federated learning privacy protection system based on homomorphic encryption and incremental update is provided. This system is applied to the federated learning privacy protection method based on homomorphic encryption and incremental update. This system includes:

[0038] The participating party is used to train the local initial model with local training data based on the federated learning framework, generating local initial model parameters; according to the initial model parameters, using an incremental update mechanism, calculating the difference value of the initial model parameters; according to the difference value of the initial model parameters, updating the initial model parameters to obtain updated model parameters; using an improved Paillier homomorphic encryption algorithm to encrypt the updated model parameters to obtain encrypted model parameters;

[0039] The aggregation server is used for each participating party to transmit the encrypted model parameters to the aggregation server. The aggregation server performs aggregation processing on the encrypted model parameters to obtain aggregated encrypted model parameters; the aggregation server updates the global model according to the aggregated encrypted model parameters to obtain an updated encrypted global model.

[0040] Optionally, the process of calculating the difference value of the initial model parameters using an incremental update mechanism according to the initial model parameters is represented by the following formula (1):

[0041] (1)

[0042] Among them, represents the model parameters of the previous round; represents the current model parameters; represents the difference value of the initial model parameters.

[0043] Optionally, before the step of updating the initial model parameters according to the difference value of the initial model parameters to obtain updated model parameters, it further includes: quantifying the difference value of the initial model parameters by using fixed-point quantization technology to obtain a quantified difference value.

[0044] Optionally, the step of encrypting the updated model parameters by using an improved Paillier homomorphic encryption algorithm to obtain encrypted model parameters includes:

[0045] Generating a key by dynamically pre-selecting a specified number of random numbers r;

[0046] Calculating a pre-computed value h according to the random number r; among them, the process of obtaining the pre-computed value h is represented by the following formula (2):

[0047] (2)

[0048] Among them, saving the pre-computed value h together with the public key and the private key;

[0049] According to the pre-computed value h, each participant uses an improved Paillier homomorphic encryption algorithm to encrypt the locally updated model parameters to obtain encrypted model parameters, where the encryption formula is represented by the following formula (3):

[0050] (3)

[0051] Among them, m represents the model parameters, r represents the pre-generated random number, h represents the pre-computed value, n represents a part of the encryption public key; c represents the encrypted model parameters.

[0052] Optionally, the aggregation server performs an aggregation process on the encrypted model parameters to obtain aggregated encrypted model parameters, including:

[0053] The aggregation server sums up the encrypted model parameters to obtain aggregated encrypted model parameters; among them, the process of obtaining the aggregated encrypted model parameters is represented by the following formula (3):

[0054] (4)

[0055] Among them, represents the incremental update of the model of the i-th client; represents the incremental update of the global model after aggregation; i represents the i-th client.

[0056] Optionally, after the step of the aggregation server updating the global model according to the aggregated encrypted model parameters to obtain the updated encrypted global model, the method further includes:

[0057] The aggregation server transmits the updated encrypted global model to each participant;

[0058] Each participant uses the private key and the improved Paillier homomorphic encryption algorithm to decrypt the encrypted parameters of the updated encrypted global model to obtain the decrypted global model; merges the local difference values according to the decrypted global model to update the local model; the participant uses the updated local model for the next round of training.

[0059] Optionally, when each participant uses the private key and the improved Paillier homomorphic encryption algorithm to decrypt the encrypted parameters of the updated encrypted global model to obtain the decrypted global model, it includes:

[0060] Decompose the modulus of the improved Paillier homomorphic encryption algorithm into multiple small moduli, perform decryption calculations on each small modulus, and obtain the decomposition corresponding to each small modulus;

[0061] Among them, the decryption process is represented by the following formula (5):

[0062] (5)

[0063] Among them, is an auxiliary function; is the first parameter of the private key; is the second parameter of the private key; is the modulus of the improved Paillier homomorphic encryption algorithm; represents the ciphertext; represents the plaintext after decryption;

[0064] Use the Chinese Remainder Theorem to combine the decompositions corresponding to each small modulus to obtain the final decryption result.

[0065] On the other hand, a federated learning privacy protection device based on homomorphic encryption and incremental update is provided. The federated learning privacy protection device based on homomorphic encryption and incremental update includes: a processor; a memory, and computer-readable instructions are stored on the memory. When the computer-readable instructions are executed by the processor, any one of the methods in the above-mentioned federated learning privacy protection method based on homomorphic encryption and incremental update is implemented.

[0066] On the other hand, a computer-readable storage medium is provided, in which at least one instruction is stored, and the at least one instruction is loaded and executed by a processor to implement any one of the above-mentioned federated learning privacy protection methods based on homomorphic encryption and incremental update.

[0067] The beneficial effects brought by the technical solutions provided in the embodiments of the present invention at least include:

[0068] Based on the federated learning framework, each participant trains the local initial model with local training data to generate local initial model parameters; according to the initial model parameters, an incremental update mechanism is adopted to calculate the difference value of the initial model parameters; according to the difference value of the initial model parameters, the initial model parameters are updated to obtain updated model parameters; an improved Paillier homomorphic encryption algorithm is adopted to encrypt the updated model parameters to obtain encrypted model parameters; each participant transmits the encrypted model parameters to an aggregation server, and the aggregation server performs aggregation processing on the encrypted model parameters to obtain aggregated encrypted model parameters; the aggregation server updates the global model according to the aggregated encrypted model parameters to obtain an updated encrypted global model.

[0069] By adopting the incremental update mechanism and the improved Paillier homomorphic encryption algorithm, this application can ensure the privacy security of the model during the transmission and aggregation processes, avoiding the risk of exposure of plaintext data; through the incremental update mechanism, only the difference values of the model parameters are transmitted, and the difference values are quantized and integer encrypted, reducing the data transmission volume and computational complexity. Through the encryption and decryption processes of the improved Paillier homomorphic encryption algorithm, the computational complexity of encryption and decryption can be reduced, and the computational efficiency of the system can be improved; the embodiments of the present invention combine additive homomorphic encryption and incremental update technologies, ensuring data privacy while reducing the computational overhead brought by the encryption and decryption processes, and solving the problems of privacy leakage risk and computational bottleneck in traditional methods. BRIEF DESCRIPTION OF THE DRAWINGS

[0070] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following will briefly introduce the drawings required for the description of the embodiments. Obviously, the following drawings are only some embodiments of the present invention. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative efforts.

[0071] Figure 1 It is a flowchart of a federated learning privacy protection method based on homomorphic encryption and incremental update provided by an embodiment of the present invention;

[0072] Figure 2It is a schematic structural diagram of a federated learning privacy protection system based on homomorphic encryption and incremental update provided by an embodiment of the present invention;

[0073] Figure 3 It is a block diagram of a federated learning privacy protection system based on homomorphic encryption and incremental update provided by an embodiment of the present invention;

[0074] Figure 4 It is a schematic structural diagram of a federated learning privacy protection device based on homomorphic encryption and incremental update provided by an embodiment of the present invention. Detailed implementation manners

[0075] Next, the technical solutions in the present invention will be described with reference to the accompanying drawings.

[0076] In the embodiments of the present invention, words such as "exemplarily" and "for example" are used to represent examples, illustrations or explanations. Any embodiment or design solution described as an "example" in the present invention should not be construed as being more preferred or more advantageous than other embodiments or design solutions. Exactly speaking, the use of the word "example" is intended to present concepts in a specific manner. In addition, in the embodiments of the present invention, the meaning expressed by "and / or" can be both, or either one of the two.

[0077] In the embodiments of the present invention, "image" and "picture" can sometimes be used interchangeably. It should be noted that when their differences are not emphasized, the meanings they express are the same. "(of)", "corresponding", and "corresponding" can sometimes be used interchangeably. It should be noted that when their differences are not emphasized, the meanings they express are the same.

[0078] In the embodiments of the present invention, sometimes subscripts such as W 1 may be written in a non-subscript form such as W1. When their differences are not emphasized, the meanings they express are the same.

[0079] To make the technical problems, technical solutions and advantages to be solved by the present invention clearer, the following will be described in detail with reference to the accompanying drawings and specific embodiments.

[0080] The embodiments of the present invention provide a federated learning privacy protection method based on homomorphic encryption and incremental update. This method can be implemented by a federated learning privacy protection device based on homomorphic encryption and incremental update. The federated learning privacy protection device based on homomorphic encryption and incremental update can be a terminal or a server. As Figure 1 shown in the flowchart of the federated learning privacy protection method based on homomorphic encryption and incremental update, the processing flow of this method can include the following steps:

[0081] S1. Based on the federated learning framework, each participant trains the local initial model using local training data to generate local initial model parameters.

[0082] In a feasible implementation, the data of each participant does not need to be uploaded to the aggregation server but is processed locally, which can avoid the leakage of sensitive data.

[0083] Among them, each participant trains a machine learning model using local data to generate local initial model parameters; for example, each participant uses a deep learning algorithm to train a neural network model and calculates the parameters of the neural network model.

[0084] Among them, different machine learning algorithms can be used during local training, including: neural network, decision tree algorithm, support vector machine algorithm, etc. The specific algorithm is selected according to specific business requirements, and the present invention does not make limitations here.

[0085] Among them, the business requirements may include: collaborative computing of unmanned aerial vehicle (UAV) swarms and autonomous driving, etc.; among them, multiple UAVs complete complex computing tasks by sharing computing tasks and data under a certain cooperation mechanism. The collective tasks may include: target recognition and tracking, image recognition, and positioning and navigation, etc. UAV swarms need to collect and share a large amount of data, such as geographical locations and sensor data, etc. when performing tasks, so privacy protection is required.

[0086] S2. According to the initial model parameters, using the incremental update mechanism, calculate the difference value of the initial model parameters; according to the difference value of the initial model parameters, update the initial model parameters with the difference value of the initial model parameters to obtain updated model parameters.

[0087] Optionally, the process of calculating the difference value of the initial model parameters according to the initial model parameters using the incremental update mechanism in S2 is represented by the following formula (1):

[0088] (1)

[0089] Among them, represents the model parameters of the previous round; represents the current model parameters; represents the difference value of the initial model parameters.

[0090] Among them, each participant calculates the increment of the initial model parameters locally, that is, the difference value between the current model and the previous model; among them, the incremental update mechanism only transmits and aggregates the difference values, rather than the complete model parameters.

[0091] Optionally, before the step of updating the initial model parameters according to the difference value of the initial model parameters in S2 to obtain updated model parameters, it further includes: quantifying the difference value of the initial model parameters using fixed-point quantization technology to obtain a quantized difference value.

[0092] In a feasible implementation, through fixed-point quantization technology, the difference value is converted into an integer, reducing the complexity of floating-point calculations, ensuring that the data meets the requirements of the homomorphic encryption algorithm during encryption, improving the calculation efficiency, and increasing the security of the encryption process.

[0093] S3. Use an improved Paillier homomorphic encryption algorithm to encrypt the updated model parameters to obtain encrypted model parameters.

[0094] In a feasible implementation, after each participant completes local model training, an improved Paillier homomorphic encryption algorithm is used to encrypt the obtained model parameters.

[0095] Among them, during the key generation process, the constants r and h are pre-computed and saved together with the public key and the private key for subsequent use.

[0096] Among them, the pre-generated constant h in the key generation phase can be directly used in the encryption phase, making the encryption process only contain multiplication and modulo operations, avoiding complex exponential operations, thereby improving the encryption efficiency.

[0097] Optionally, the specific implementation process of S3 may include S31 - S33:

[0098] S31. Generate a key by dynamically pre-selecting a specified number of random numbers r;

[0099] S32. Calculate the pre-computed value h according to the random number r; among them, the process of obtaining the pre-computed value h is represented by the following formula (2):

[0100] (2)

[0101] Among them, the pre-computed value h is saved together with the public key and the private key;

[0102] S33. According to the pre-computed value h, each participant uses an improved Paillier homomorphic encryption algorithm to encrypt the locally updated model parameters to obtain encrypted model parameters, where the encryption formula is represented by the following formula (3):

[0103] (3)

[0104] Among them, m represents model parameters, r represents a pre-generated random number, h represents a pre-computed value, n represents a part of the encryption public key; c represents the encrypted model parameters.

[0105] Among them, the pre-computed value h is used to replace , where Usually select , that is , after further simplifying the formula, the above encryption formula is obtained.

[0106] S4. Each participant transmits the encrypted model parameters to the aggregation server, and the aggregation server performs aggregation processing on the encrypted model parameters to obtain the aggregated encrypted model parameters; the aggregation server updates the global model according to the aggregated encrypted model parameters to obtain the updated encrypted global model.

[0107] Optionally, the aggregation server in S4 performs aggregation processing on the encrypted model parameters to obtain the aggregated encrypted model parameters, including:

[0108] The aggregation server sums the encrypted model parameters to obtain the aggregated encrypted model parameters; among them, the process of obtaining the aggregated encrypted model parameters is represented by the following formula (3):

[0109] (4)

[0110] Among them, represents the incremental update of the model of the i-th client; represents the incremental update of the aggregated global model; i represents the i-th client.

[0111] In a feasible implementation, under the homomorphic encryption algorithm, the aggregation server can directly sum the encrypted model parameters to calculate the parameters of the global model; among them, the aggregation server performs addition calculation on the encrypted model parameters without decrypting; since the Paillier homomorphic encryption algorithm supports addition operations, the aggregation server can perform model aggregation operations in the encrypted state, which can ensure the security of private data.

[0112] Among them, the aggregation server updates the global model through the aggregated model parameters; the encrypted global model can be generated through the homomorphic encryption algorithm, providing useful information while maintaining privacy.

[0113] Optionally, after the step where the aggregation server in S4 updates the global model according to the aggregated encrypted model parameters to obtain the updated encrypted global model, it further includes:

[0114] The aggregation server transmits the updated encrypted global model to each participant;

[0115] Each participant uses its private key and the improved Paillier homomorphic encryption algorithm to decrypt the encrypted parameters of the updated encrypted global model, obtaining the decrypted global model; the participant combines the local difference values based on the decrypted global model to update the local model; the participant uses the updated local model for the next round of training.

[0116] Among them, the above process is carried out in a multi-round iterative manner until the model converges. Among them, in each round of iteration, the incremental update mechanism and the improved Paillier homomorphic encryption algorithm can ensure the protection of data privacy and improve the computational efficiency at the same time.

[0117] Optionally, each participant uses its private key and the improved Paillier homomorphic encryption algorithm to decrypt the encrypted parameters of the updated encrypted global model, obtaining the decrypted global model, including:

[0118] Decompose the modulus of the improved Paillier homomorphic encryption algorithm into multiple small moduli, perform decryption calculations on each small modulus, and obtain the decomposition corresponding to each small modulus;

[0119] In a feasible implementation, decompose the modulus of Paillier into multiple small moduli as such that ;

[0120] Among them, the decryption process is represented by the following formula (5):

[0121] (5)

[0122] Among them, is an auxiliary function; is the first parameter of the private key; is the second parameter of the private key; is the modulus of the improved Paillier homomorphic encryption algorithm; represents the ciphertext; represents the decrypted plaintext;

[0123] Use the Chinese Remainder Theorem to combine the decompositions corresponding to each small modulus to obtain the final decryption result.

[0124] Among them, in the decryption process, the modular operation is decomposed into multiple small moduli for calculation, and parallel acceleration is achieved through the Chinese Remainder Theorem.

[0125] Among them, the Chinese Remainder Theorem is a method mastered by those skilled in the art, and will not be further elaborated in this application.

[0126] Among them, as Figure 2The following is a schematic structural diagram of a federated learning privacy protection system based on homomorphic encryption and incremental update provided by an embodiment of the present invention; in a feasible implementation manner, multiple clients train an initial model to generate parameters of the initial model, perform differential update on the parameters of the initial model, obtain updated model parameters by quantifying the difference values, and encrypt the updated model parameters by using an improved Paillier homomorphic encryption algorithm to obtain encrypted model parameters; each participating party transmits the encrypted model parameters to an aggregation server, and the aggregation server performs aggregation processing on the encrypted model parameters to obtain a global aggregated model.

[0127] Among them, the present application is applicable to fields such as medical health, financial risk control, and intelligent transportation.

[0128] Among them, in order to ensure that the model updates of the participating parties cannot leak any private information, the present invention adopts a secure aggregation algorithm based on homomorphic encryption and local model incremental update. By encrypting and aggregating the encrypted model parameters uploaded by each participating party, an encrypted global model is finally obtained to ensure privacy protection during the model aggregation process.

[0129] Among them, the parameter incremental update in the federated learning adopted in the present application can reduce the amount of encrypted data. Since the model gradually tends to be stable during the training process, the incremental update of its parameters will gradually tend to 0. Therefore, the characteristic of small incremental update enables only a small amount of data to be encrypted, thus significantly reducing the computing and communication overheads. Since model parameters usually adopt floating-point data types, while the Paillier encryption algorithm can originally only process integers, it is necessary to encrypt the integer part and the decimal part of the floating-point type respectively. In the present application, the floating-point number is converted into a fixed-point number through fixed-point quantization technology, so that the decimal part of the floating-point number can be encrypted in integer form, which can effectively adapt to the integer operation of Paillier encryption, not only ensuring the efficiency of the encryption process, but also ensuring that while meeting the privacy protection requirements, the size of the encrypted data is reduced, and the efficiency and scalability of the encryption operation during the model training process are improved.

[0130] In the embodiments of the present invention, based on the federated learning framework, each participant trains the local initial model using local training data to generate local initial model parameters; according to the initial model parameters, an incremental update mechanism is adopted to calculate the difference values of the initial model parameters; according to the difference values of the initial model parameters, the initial model parameters are updated to obtain updated model parameters; an improved Paillier homomorphic encryption algorithm is used to encrypt the updated model parameters to obtain encrypted model parameters; each participant transmits the encrypted model parameters to the aggregation server, and the aggregation server performs aggregation processing on the encrypted model parameters to obtain aggregated encrypted model parameters; the aggregation server updates the global model according to the aggregated encrypted model parameters to obtain an updated encrypted global model.

[0131] By adopting the incremental update mechanism and the improved Paillier homomorphic encryption algorithm, this application can ensure the privacy and security of the model during the transmission and aggregation processes, avoiding the risk of exposure of plaintext data; through the incremental update mechanism, only the difference values of the model parameters are transmitted, and the difference values are quantized and integer encrypted, reducing the data transmission volume and computational complexity. Through the encryption and decryption processes of the improved Paillier homomorphic encryption algorithm, the computational complexity of encryption and decryption can be reduced, and the computational efficiency of the system can be improved; the embodiments of the present invention combine additive homomorphic encryption and incremental update technologies, ensuring data privacy while reducing the computational overhead brought by the encryption and decryption processes, and solving the problems of privacy leakage risk and computational bottleneck in traditional methods.

[0132] Figure 3 It is a block diagram of a federated learning privacy protection system based on homomorphic encryption and incremental update shown according to an exemplary embodiment. This system is used for the federated learning privacy protection method based on homomorphic encryption and incremental update. Refer to Figure 3 , this system includes a participant 310 and an aggregation server 320. Among them:

[0133] The participant 310 is used to, based on the federated learning framework, each participant trains the local initial model using local training data to generate local initial model parameters; according to the initial model parameters, an incremental update mechanism is adopted to calculate the difference values of the initial model parameters; according to the difference values of the initial model parameters, the initial model parameters are updated to obtain updated model parameters; an improved Paillier homomorphic encryption algorithm is used to encrypt the updated model parameters to obtain encrypted model parameters;

[0134] The aggregation server 320 is used for each participating party to transmit the encrypted model parameters to the aggregation server, and the aggregation server performs aggregation processing on the encrypted model parameters to obtain aggregated encrypted model parameters; the aggregation server updates the global model according to the aggregated encrypted model parameters to obtain an updated encrypted global model.

[0135] Optionally, the process of calculating the difference value of the initial model parameters by using an incremental update mechanism according to the initial model parameters is represented by the following formula (1):

[0136] (1)

[0137] Wherein, represents the model parameters of the previous round; represents the current model parameters; represents the difference value of the initial model parameters.

[0138] Optionally, before the step of updating the initial model parameters according to the difference value of the initial model parameters to obtain updated model parameters, it further includes: quantifying the difference value of the initial model parameters by using a fixed-point quantization technique to obtain a quantized difference value.

[0139] Optionally, the step of encrypting the updated model parameters by using an improved Paillier homomorphic encryption algorithm to obtain encrypted model parameters includes:

[0140] Generating a key by dynamically pre-selecting a specified number of random numbers r;

[0141] Calculating a pre-computed value h according to the random number r; wherein the process of obtaining the pre-computed value h is represented by the following formula (2):

[0142] (2)

[0143] Wherein, the pre-computed value h is saved together with the public key and the private key;

[0144] According to the pre-computed value h, each participating party uses an improved Paillier homomorphic encryption algorithm to encrypt the locally updated model parameters to obtain encrypted model parameters, wherein the encryption formula is represented by the following formula (3):

[0145] (3)

[0146] Wherein, m represents the model parameters, r represents the pre-generated random number, h represents the pre-computed value, n represents a part of the encryption public key; c represents the encrypted model parameters.

[0147] Optionally, the aggregation server performs aggregation processing on the encrypted model parameters to obtain aggregated encrypted model parameters, including:

[0148] The aggregation server sums up the encrypted model parameters to obtain aggregated encrypted model parameters; wherein, the process of obtaining the aggregated encrypted model parameters is represented by the following formula (3):

[0149] (4)

[0150] Wherein, represents the incremental update of the model of the i-th client; represents the incremental update of the global model after aggregation; i represents the i-th client.

[0151] Optionally, after the step of the aggregation server updating the global model according to the aggregated encrypted model parameters to obtain the updated encrypted global model, it further includes:

[0152] The aggregation server transmits the updated encrypted global model to each participant;

[0153] Each participant uses the private key and the improved Paillier homomorphic encryption algorithm to decrypt the encrypted parameters of the updated encrypted global model to obtain the decrypted global model; merges the local difference values according to the decrypted global model to update the local model; the participant uses the updated local model for the next round of training.

[0154] Optionally, each participant uses the private key and the improved Paillier homomorphic encryption algorithm to decrypt the encrypted parameters of the updated encrypted global model to obtain the decrypted global model, including:

[0155] Decompose the modulus of the improved Paillier homomorphic encryption algorithm into multiple small moduli, and perform decryption calculations on each small modulus to obtain the decomposition corresponding to each small modulus;

[0156] Wherein, the decryption process is represented by the following formula (5):

[0157] (5)

[0158] Wherein, is an auxiliary function; is the first parameter of the private key; is the second parameter of the private key; is the modulus of the improved Paillier homomorphic encryption algorithm; represents the ciphertext; represents the plaintext after decryption;

[0159] The Chinese Remainder Theorem is used to combine the decompositions corresponding to each small modulus to obtain the final decryption result.

[0160] In the embodiment of the present invention, based on the federated learning framework, each participant uses local training data to train the local initial model to generate local initial model parameters; according to the initial model parameters, an incremental update mechanism is adopted to calculate the difference values of the initial model parameters; according to the difference values of the initial model parameters, the initial model parameters are updated to obtain updated model parameters; an improved Paillier homomorphic encryption algorithm is used to encrypt the updated model parameters to obtain encrypted model parameters; each participant transmits the encrypted model parameters to the aggregation server, and the aggregation server performs aggregation processing on the encrypted model parameters to obtain aggregated encrypted model parameters; the aggregation server updates the global model according to the aggregated encrypted model parameters to obtain an updated encrypted global model.

[0161] By adopting the incremental update mechanism and the improved Paillier homomorphic encryption algorithm, this application can ensure the privacy and security of the model during the transmission and aggregation processes, avoiding the risk of exposure of plaintext data; through the incremental update mechanism, only the difference values of the model parameters are transmitted, and the difference values are quantized and encrypted as integers, reducing the data transmission volume and computational complexity. Through the encryption and decryption processes of the improved Paillier homomorphic encryption algorithm, the computational complexity of encryption and decryption can be reduced, improving the computational efficiency of the system; the embodiment of the present invention combines additive homomorphic encryption and incremental update technologies, while ensuring data privacy, reducing the computational overhead brought by the encryption and decryption processes, and solving the problems of privacy leakage risk and computational bottleneck in traditional methods.

[0162] Figure 4 FIG. is a schematic structural diagram of a federated learning privacy protection device based on homomorphic encryption and incremental update provided by an embodiment of the present invention. As Figure 4 shown, the federated learning privacy protection device based on homomorphic encryption and incremental update may include the above-mentioned Figure 3 shown federated learning privacy protection system based on homomorphic encryption and incremental update. Optionally, the federated learning privacy protection device 410 based on homomorphic encryption and incremental update may include a first processor 2001.

[0163] Optionally, the federated learning privacy protection device 410 based on homomorphic encryption and incremental update may further include a memory 2002 and a transceiver 2003.

[0164] Among them, the first processor 2001 is connected to the memory 2002 and the transceiver 2003, such as through a communication bus.

[0165] Next, in combination with Figure 4Specifically introduce each component of the federated learning privacy protection device 410 based on homomorphic encryption and incremental update:

[0166] Among them, the first processor 2001 is the control center of the federated learning privacy protection device 410 based on homomorphic encryption and incremental update. It can be a single processor or a collective term for multiple processing elements. For example, the first processor 2001 is one or more central processing units (CPUs), or it can be an application specific integrated circuit (ASIC), or one or more integrated circuits configured to implement the embodiments of the present invention. For example: one or more digital signal processors (DSPs), or one or more field programmable gate arrays (FPGAs).

[0167] Optionally, the first processor 2001 can execute various functions of the federated learning privacy protection device 410 based on homomorphic encryption and incremental update by running or executing software programs stored in the memory 2002 and calling data stored in the memory 2002.

[0168] In a specific implementation, as an embodiment, the first processor 2001 may include one or more CPUs, such as Figure 4 the CPU0 and CPU1 shown in

[0169] In a specific implementation, as an embodiment, the federated learning privacy protection device 410 based on homomorphic encryption and incremental update may also include multiple processors, such as Figure 4 the first processor 2001 and the second processor 2004 shown in. Each of these processors can be a single-core processor (single-CPU) or a multi-core processor (multi-CPU). Here, the processor can refer to one or more devices, circuits, and / or processing cores for processing data (such as computer program instructions).

[0170] Among them, the memory 2002 is used to store software programs for executing the solution of the present invention and is controlled by the first processor 2001 for execution. The specific implementation method can refer to the above method embodiments and will not be elaborated here.

[0171] Optionally, the memory 2002 may be a read-only memory (ROM) or other type of static storage device that can store static information and instructions, a random access memory (RAM) or other type of dynamic storage device that can store information and instructions, or may also be an electrically erasable programmable read-only memory (EEPROM), a compact disc read-only memory (CD-ROM), or other optical disc storage, optical disc storage (including compact discs, laser discs, optical discs, digital versatile discs, Blu-ray discs, etc.), magnetic disk storage media, or other magnetic storage devices, or any other medium that can be used to carry or store the desired program code in the form of instructions or data structures and can be accessed by a computer, but is not limited thereto. The memory 2002 may be integrated with the first processor 2001 or may exist independently and be coupled to the first processor 2001 through an interface circuit ( Figure 4 not shown) of the federated learning privacy protection device 410 based on homomorphic encryption and incremental updates. The embodiments of the present invention do not make specific limitations on this.

[0172] The transceiver 2003 is used to communicate with a network device or with a terminal device.

[0173] Optionally, the transceiver 2003 may include a receiver and a transmitter ( Figure 4 not shown separately). Among them, the receiver is used to implement the receiving function, and the transmitter is used to implement the sending function.

[0174] Optionally, the transceiver 2003 may be integrated with the first processor 2001 or may exist independently and be coupled to the first processor 2001 through an interface circuit ( Figure 4 not shown) of the federated learning privacy protection device 410 based on homomorphic encryption and incremental updates. The embodiments of the present invention do not make specific limitations on this.

[0175] It should be noted that Figure 4 the structure of the federated learning privacy protection device 410 based on homomorphic encryption and incremental updates shown does not constitute a limitation on the router. The actual knowledge structure recognition device may include more or fewer components than shown, or combine certain components, or have different component arrangements.

[0176] In addition, the technical effects of the federated learning privacy protection device 410 based on homomorphic encryption and incremental updates may refer to the technical effects of the federated learning privacy protection method based on homomorphic encryption and incremental updates described in the above method embodiments, and will not be elaborated here.

[0177] It should be understood that the first processor 2001 in the embodiments of the present invention may be a central processing unit (CPU), and the processor may also be other general-purpose processors, digital signal processors (DSPs), application specific integrated circuits (ASICs), field programmable gate arrays (FPGAs) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor may be a microprocessor or the processor may also be any conventional processor, etc.

[0178] It should also be understood that the memory in the embodiments of the present invention may be a volatile memory or a non-volatile memory, or may include both volatile and non-volatile memories. Among them, the non-volatile memory may be a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), or a flash memory. The volatile memory may be a random access memory (RAM), which is used as an external cache. By way of example but not limitation, many forms of random access memory (RAM) are available, such as static random access memory (SRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), double data rate synchronous dynamic random access memory (DDR SDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchlink dynamic random access memory (SLDRAM), and direct rambus random access memory (DR RAM).

[0179] The above embodiments can be implemented in whole or in part by software, hardware (such as circuits), firmware, or any combination thereof. When implemented using software, the above embodiments can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions or computer programs. When the computer instructions or computer programs are loaded or executed on a computer, the processes or functions described in the embodiments of the present invention are generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable systems. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center by wired (such as infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium that can be accessed by a computer or a data storage device such as a server or data center that contains one or more collections of available media. The available media can be magnetic media (such as floppy disks, hard disks, magnetic tapes), optical media (such as DVDs), or semiconductor media. The semiconductor media can be a solid-state drive.

[0180] It should be understood that the term "and / or" in this document is merely a description of the association relationship between associated objects, indicating that three relationships can exist. For example, A and / or B can represent: A exists alone, A and B exist simultaneously, and B exists alone. Here, A and B can be singular or plural. In addition, the character " / " in this document generally represents an "or" relationship between the associated objects before and after, but it may also represent an "and / or" relationship, which can be specifically understood by referring to the context before and after.

[0181] In the present invention, "at least one" means one or more, and "a plurality" means two or more. "At least one of the following" or its similar expressions refer to any combination of these items, including any combination of single items or plural items. For example, at least one of a, b, or c can represent: a, b, c, a - b, a - c, b - c, or a - b - c, where a, b, and c can be single or multiple.

[0182] It should be understood that in various embodiments of the present invention, the magnitudes of the sequence numbers of the above processes do not mean the order of execution. The order of execution of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present invention.

[0183] Those of ordinary skill in the art can realize that the units and algorithm steps of each example described in combination with the embodiments disclosed herein can be implemented by electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are executed in a hardware or software manner depends on the specific application and design constraints of the technical solution. Professional technicians can use different methods to implement the described functions for each specific application, but such implementation should not be considered to exceed the scope of the present invention.

[0184] Those skilled in the art can clearly understand that for the convenience and conciseness of description, the specific working processes of the devices, systems, and units described above can refer to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0185] In several embodiments provided by the present invention, it should be understood that the disclosed devices, systems, and methods can be implemented in other ways. For example, the system embodiments described above are merely illustrative. For example, the division of the units is only a logical function division, and there can be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another device, or some features can be ignored or not executed. Another point is that the displayed or discussed mutual coupling, direct coupling, or communication connection can be through some interfaces. The indirect coupling or communication connection of the systems or units can be in an electrical, mechanical, or other form.

[0186] The units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they can be located in one place, or distributed to multiple network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0187] In addition, the functional units in each embodiment of the present invention can be integrated in a processing unit, or each unit can exist physically alone, or two or more units can be integrated in one unit.

[0188] When the above-mentioned functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on such understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes several instructions for causing a computer device (which may be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods described in various embodiments of the present invention. The foregoing storage medium includes: various media that can store program codes, such as USB flash drives, mobile hard disks, read-only memories (ROMs), random access memories (RAMs), magnetic disks, or optical discs.

[0189] As described above, the above are only specific embodiments of the present invention, but the protection scope of the present invention is not limited thereto. Any person skilled in the art within the technical scope disclosed by the present invention can easily think of changes or substitutions, which should all be covered by the protection scope of the present invention. Therefore, the protection scope of the present invention should be subject to the protection scope of the claims.

Claims

1. A privacy protection method for federated learning based on homomorphic encryption and incremental update, characterized in that: The method for protecting the privacy of federated learning based on homomorphic encryption and incremental update is implemented by a system for protecting the privacy of federated learning based on homomorphic encryption and incremental update, including: multiple participants and an aggregation server; the method includes: S1. Based on the federated learning framework, each participant uses local training data to train the local initial model to generate local initial model parameters; S2. Calculate the difference values ​​of the initial model parameters using an incremental update mechanism according to the initial model parameters; update the initial model parameters according to the difference values ​​of the initial model parameters to obtain updated model parameters; S3. Using an improved Paillier homomorphic encryption algorithm, encrypt the updated model parameters to obtain encrypted model parameters; S4. Each participant transmits the encrypted model parameters to the aggregation server, and the aggregation server aggregates the encrypted model parameters to obtain aggregated encrypted model parameters; the aggregation server updates the global model according to the aggregated encrypted model parameters to obtain an updated encrypted global model.

2. The method for protecting privacy in federated learning based on homomorphic encryption and incremental update according to claim 1, characterized in that: The process of calculating the difference value of the initial model parameter using an incremental update mechanism according to the initial model parameter in S2 is represented by the following formula (1): （1） in, Represents the model parameters of the previous round; Represents the current model parameters; Represents the difference values ​​of the initial model parameters.

3. The method for protecting privacy in federated learning based on homomorphic encryption and incremental update according to claim 1, characterized in that: Before the step of updating the initial model parameters according to the differential values ​​of the initial model parameters to obtain updated model parameters in S2, the step also includes: quantizing the differential values ​​of the initial model parameters using fixed-point quantization technology to obtain quantized differential values.

4. The method for privacy protection of federated learning based on homomorphic encryption and incremental update according to claim 1, characterized in that: The S3 adopts an improved Paillier homomorphic encryption algorithm to encrypt the updated model parameters to obtain the encrypted model parameters, including: S31, generate a key by dynamically pre-selecting a specified number of random numbers r; S32. Calculate the pre-calculated value h according to the random number r; wherein the process of obtaining the pre-calculated value h is represented by the following formula (2): （2） Wherein, the pre-calculated value h is saved together with the public key and the private key; S33. According to the pre-calculated value h, each participant uses the improved Paillier homomorphic encryption algorithm to encrypt the locally updated model parameters to obtain the encrypted model parameters, where the encryption formula is expressed by the following formula (3): （3） Among them, m represents the model parameters, r represents the pre-generated random number, h represents the pre-calculated value, n represents a part of the encrypted public key; and c represents the encrypted model parameters.

5. The method for protecting privacy in federated learning based on homomorphic encryption and incremental update according to claim 1, characterized in that: The aggregation server of S4 performs aggregation processing on the encrypted model parameters to obtain aggregated encrypted model parameters, including: The aggregation server adds the encrypted model parameters to obtain the aggregated encrypted model parameters; wherein the process of obtaining the aggregated encrypted model parameters is represented by the following formula (3): （4） in, represents the incremental update of the model of the i-th client; Represents the incremental update of the global model after aggregation; i represents the i-th client.

6. The method for privacy protection of federated learning based on homomorphic encryption and incremental update according to claim 1, characterized in that: After the step of the aggregation server updating the global model according to the aggregated encrypted model parameters to obtain the updated encrypted global model in S4, the method further includes: The aggregation server transmits the updated encrypted global model to each participant; Each participant uses a private key and an improved Paillier homomorphic encryption algorithm to decrypt the encrypted parameters of the updated encrypted global model to obtain a decrypted global model; local differential values ​​are merged according to the decrypted global model to update the local model; the participant uses the updated local model to perform the next round of training.

7. The method for protecting privacy in federated learning based on homomorphic encryption and incremental updates according to claim 6, characterized in that: Each participant uses a private key and an improved Paillier homomorphic encryption algorithm to decrypt the encrypted parameters of the updated encrypted global model to obtain a decrypted global model, including: Decompose the modulus of the improved Paillier homomorphic encryption algorithm into multiple small moduli, perform decryption calculation on each small modulus, and obtain the decomposition corresponding to each small modulus; The decryption process is expressed by the following formula (5): （5） in, As a helper function; is the first parameter of the private key; is the second parameter of the private key; is the modulus of the improved Paillier homomorphic encryption algorithm; Represents ciphertext; Represents the decrypted plaintext; The Chinese remainder theorem is used to combine the decompositions corresponding to each small modulus to obtain the final decryption result.

8. A federated learning privacy protection system based on homomorphic encryption and incremental update, wherein the federated learning privacy protection system based on homomorphic encryption and incremental update is used to implement the federated learning privacy protection method based on homomorphic encryption and incremental update as claimed in any one of claims 1 to 7, characterized in that: The system comprises: The participants are used to train the local initial model based on the federated learning framework, and each participant uses local training data to generate local initial model parameters; according to the initial model parameters, an incremental update mechanism is used to calculate the differential value of the initial model parameters; according to the differential value of the initial model parameters, the initial model parameters are updated to obtain updated model parameters; and the updated model parameters are encrypted using an improved Paillier homomorphic encryption algorithm to obtain encrypted model parameters; The aggregation server is used for each participant to transmit the encrypted model parameters to the aggregation server, and the aggregation server aggregates the encrypted model parameters to obtain aggregated encrypted model parameters; the aggregation server updates the global model according to the aggregated encrypted model parameters to obtain an updated encrypted global model.

9. A privacy protection device for federated learning based on homomorphic encryption and incremental update, characterized in that: The federated learning privacy protection device based on homomorphic encryption and incremental update includes: processor; A memory having computer-readable instructions stored thereon, wherein when the computer-readable instructions are executed by the processor, the method according to any one of claims 1 to 7 is implemented.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores program codes, which can be called by a processor to execute the method according to any one of claims 1 to 7.

Images