A privacy-preserving federated learning method, system and device

Abstract

This application discloses a privacy-preserving federated learning method, system, and device. It employs a dual-server, dual-trapdoor homomorphic encryption scheme, which calculates two cosine similarities in a ciphertext environment to serve as the client's confidence level. Lower confidence levels indicate a higher likelihood that the client is initiating a poisoning attack, leading to the discarding of gradients uploaded by such clients and effectively resisting poisoning attacks. Simultaneously, since the servers only access ciphertext data, they cannot infringe on client privacy, effectively resisting privacy inference attacks. This solves the technical problems of semi-honest servers potentially guessing user privacy information when receiving gradients, leading to privacy leaks, and the inability of servers to determine whether a user has initiated a poisoning attack based on the dataset due to the nature of federated learning, thus preventing defective aggregated models or hindering accuracy improvement.

Description

Technical Field

[0001] This application relates to the field of blockchain technology, and in particular to a privacy-preserving federated learning method, system, and device. Background Technology

[0002] The explosive growth of big data has further propelled the development of artificial intelligence. However, while the volume of data is increasing rapidly, it is often fragmented, and due to the existence of certain institutions (finance, healthcare, etc.), their data exists in the form of "data silos." Although this data is not shared, there is an urgent need in real life to use it to build machine learning models to solve problems. Driven by this need, some have proposed distributed learning, which collects these scattered data into a huge dataset for retraining. However, this approach can lead to the leakage of some private data. Further, federated learning has been proposed.

[0003] Data owners can train a common model locally using their own datasets, and then a server aggregates these trained models. In this process, each data provider only shares the trained model and does not need to provide the original local training data. Due to this privacy-preserving training model, especially given the current privacy laws and regulations in many countries, federated learning has been deployed by numerous companies.

[0004] While federated learning offers significant advantages, it also carries numerous potential drawbacks, such as privacy breaches and model poisoning. When a semi-honest server receives gradients from a user, it may attempt to guess the user's private information. Simultaneously, federated learning faces another critical security threat—poisoning attacks. Due to the unique nature of federated learning, the server cannot determine from the dataset whether a user has launched a poisoning attack, potentially leading to flawed aggregated models or a failure to improve accuracy. Furthermore, during training, some clients may engage in free-riding attacks, reaping the benefits without participating in the training themselves, yet sharing the results of others. Summary of the Invention

[0005] This application provides a privacy-preserving federated learning method, system, and device, which solves the problems of privacy leakage caused by semi-honest servers guessing user privacy information when receiving gradients from users, and the inability of servers to determine whether users have launched poisoning attacks based on the dataset due to the special nature of federated learning, resulting in defective aggregated models or inability to improve accuracy. At the same time, it addresses the technical problem that some clients launch free-riding attacks during training, reaping the benefits without participating in training themselves, but sharing the results of others.

[0006] In view of this, the first aspect of this application provides a privacy-preserving federated learning method, the method comprising:

[0007] S1. The key distribution center generates a public key and a private key, and then splits the private key and distributes it to the main server, auxiliary server and client;

[0008] S2. The client obtains the global model hash value of the current training round from the blockchain and stores it in the file storage system. After obtaining the global gradient through the hash value in name access mode, it completes the training locally, performs preset processing on the local training gradient, obtains the updated encrypted gradient, and uploads it to the blockchain.

[0009] S3. The main server and the auxiliary server communicate based on the blockchain and file storage system to determine whether the encryption gradient uploaded by the client to the blockchain is normalized.

[0010] S4. The main server and auxiliary server communicate based on blockchain and file storage system to calculate the client's confidence level.

[0011] S5. Calculate the current client's contribution in this round. If the contribution is lower than the first preset threshold, mark the current client as a client that initiated a free-rider attack and kick the client out of the blockchain.

[0012] S6. The main server calculates the aggregate weights of each client based on the confidence level, completes the aggregation of the global model, and updates the global gradient of the global model to the file storage system.

[0013] S7. Return to repeat steps S1 to S6 until the global model converges.

[0014] Optionally, step S1 specifically includes:

[0015] The key distribution center generates strong prime numbers p and q, and sets the public key pk = N = p·q and the private key sk = λ = lcm(p-1, q-1);

[0016] The key distribution center splits the public and private keys, generating key pairs for the master server and auxiliary server, and for the master server and client, respectively, as follows: sk1 = λ1, sk2 = λ2, λ1 + λ2 ≡ 0 mod λ, λ1 + λ2 ≡ 1 mod N. After completion, let... sk1 and sk2 are the private key pair between the i-th user and the master server, and the private key pair between the master server and the auxiliary server.

[0017] Optionally, the pre-processing of the local training gradient in step S2 specifically includes:

[0018] The local training gradients are clipped by setting gradients whose absolute values ​​are less than a second preset threshold to 0.

[0019] The local training gradients are normalized, specifically through the expression... Where g is the gradient vector of the client, g i Let be the i-th vector in g;

[0020] The local training gradient is encrypted using the public key generated by the key distribution center, resulting in the encrypted gradient, specifically expressed as [[g]]=(N+1). g ·r N modN 2 , (r∈[1,N]), where [[g]] is the encryption gradient, r is a positive integer less than N and coprime to N, and N is the public key.

[0021] Optionally, step S3 specifically includes:

[0022] The main server adds random noise to the encrypted gradient [[g]] uploaded by the user to obtain... Then, partial decryption is performed using the key matched with the auxiliary server to obtain...

[0023] The main server will hash value and After the hash value is added to the chain, it will be stored in key-value pairs. as well as Store in a file storage system;

[0024] After the auxiliary server obtains the corresponding hash value from the blockchain, it downloads the gradient from the file storage system. as well as

[0025] The auxiliary server decrypts using its own private key. get After complete decryption, the noisy gradient plaintext is obtained.

[0026] The auxiliary server calculates the sum of squares for each gradient. The resulting hash value is then uploaded to the blockchain and stored in the file storage system;

[0027] After retrieving the result hash value from the blockchain, the main server downloads the sum of squares of the gradients from the file storage system.

[0028] The main server removes noise to obtain s, specifically through the expression:

[0029]

[0030]

[0031] in, For the k-th noisy component in the gradient vector uploaded by the client, r i , (i∈[1,n]) represents the noise added to the i-th component. If the calculated s is 1, the gradient uploaded by the user is accepted; otherwise, it is discarded.

[0032] Optionally, partial decryption and full decryption are specifically as follows:

[0033]

[0034]

[0035] Where [[g]] is the encryption gradient, [g]1 and [g]2 are the results after decryption using a single private key, and N is the public key.

[0036] Optionally, step S4 specifically includes:

[0037] Assume the encryption gradient uploaded by the user in round t+1 is... The global model gradient in round t is The gradient of the root dataset trained on the main server in round t+1 is: Noise is added to the encrypted gradient uploaded by the user, specifically as follows:

[0038]

[0039] in, Let be the cosine similarity between the i-th client and the global gradient from the previous round. Let cosine similarity be the value between the i-th client and the root dataset maintained by the server.

[0040] Main server upload and The hash value is put on the blockchain and the data is stored there. Stored in the file storage system as key-value pairs;

[0041] After the auxiliary server obtains the corresponding hash value from the blockchain, it partially decrypts it using its own private key, uploads the decryption result hash value to the blockchain, and stores it in the file storage system.

[0042] After the main server obtains the corresponding decryption result hash value from the blockchain, it downloads a portion of the decryption result from the file storage system and then uses full decryption to obtain the two cosine similarities.

[0043] The main server calculates the confidence score of the corresponding client using two cosine similarities, specifically:

[0044] R i =α·cos(θ) 1,i )+β·cos(θ 2,i );

[0045] R i Let α be the confidence level of the i-th user, and β be the corresponding weighting factors.

[0046] Optionally, the calculation of the current client's contribution in this round in step S5 is as follows:

[0047] τ1>τ2>τ3 and

[0048] Where τ is the time decay factor, These represent the contributions in the current round and the previous two rounds, respectively.

[0049] Optionally, in step S6, the master server calculates the aggregate weight of each client based on the confidence level to complete the aggregation of the global model, specifically as follows:

[0050]

[0051] Among them Wei i The weights aggregated for the i-th user.

[0052] A second aspect of this application provides a privacy-preserving federated learning system, the system comprising:

[0053] Key distribution center, main server, auxiliary server, and client;

[0054] The key distribution center generates public and private keys, and then splits the private key and distributes it to the main server, auxiliary server, and client.

[0055] The client obtains the global model hash value of the current training round from the blockchain and stores it in the file storage system. After obtaining the global gradient through the hash value in name access mode, it completes the training locally, performs preset processing on the local training gradient, obtains the updated encrypted gradient, and uploads it to the blockchain.

[0056] The main server and the auxiliary server communicate based on the blockchain and file storage system to determine whether the encryption gradient of the client's uploaded blockchain is normalized;

[0057] The primary server and secondary servers communicate based on blockchain and file storage systems to calculate the client's confidence level.

[0058] The master server calculates the current client's contribution in this round. If the contribution is lower than the first preset threshold, the current client is marked as a client that launched a free-rider attack and is kicked out of the blockchain.

[0059] The main server calculates the aggregate weights of each client based on the confidence level, completes the aggregation of the global model, and updates the global gradient of the global model to the file storage system;

[0060] Return to the key distribution center and repeat all steps until the global model converges.

[0061] A third aspect of this application provides a privacy-preserving federated learning device, the device comprising a processor and a memory:

[0062] The memory is used to store program code and transmit the program code to the processor;

[0063] The processor is configured to execute the steps of the privacy-preserving federated learning method as described in the first aspect above, according to the instructions in the program code.

[0064] As can be seen from the above technical solutions, the embodiments of this application have the following advantages:

[0065] This application provides a privacy-preserving federated learning method, system, and device. It employs a dual-server, dual-trapdoor homomorphic encryption scheme, which calculates two cosine similarities in a ciphertext environment to serve as the client's confidence level. Lower confidence levels indicate a higher likelihood that the client is initiating a poisoning attack, leading to the discarding of gradients uploaded by such clients and effectively resisting poisoning attacks. Simultaneously, since the servers only access ciphertext data, they cannot infringe on client privacy, effectively resisting privacy inference attacks. This solves the technical problems of semi-honest servers potentially guessing user privacy information when receiving gradients, leading to privacy leaks, and the inability of servers to determine whether a user has initiated a poisoning attack based on the dataset due to the nature of federated learning, thus preventing defective aggregated models or hindering accuracy improvement. Attached Figure Description

[0066] Figure 1 This is a flowchart of the privacy-preserving federated learning method in the embodiments of this application;

[0067] Figure 2 This is a schematic diagram of the privacy-preserving federated learning system in the embodiments of this application;

[0068] Figure 3 This is a schematic diagram of the privacy-preserving federated learning device in the embodiments of this application. Detailed Implementation

[0069] To enable those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present application, and not all embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of the present application.

[0070] This application designs a privacy-preserving federated learning method, system, and device, which solves the problems of privacy leakage caused by semi-honest servers guessing user privacy information when receiving gradients from users, and the inability of servers to determine whether users have launched poisoning attacks based on the dataset due to the special nature of federated learning, resulting in defective aggregated models or inability to improve accuracy. At the same time, it addresses the technical problem of some clients launching free-riding attacks during training, reaping the benefits without participating in training themselves, but sharing the results of others.

[0071] The double trapdoor homomorphic encryption scheme is as follows:

[0072] 1. Key generation: Generate two strong prime numbers p and q with a bit length of ε bits, then let N = p·q, λ = lcm(p-1, q-1), public key pk = (N, N+1), private key sk = λ;

[0073] 2. Private key splitting: Split sk to obtain two partial keys sk1=λ1, sk2=λ2, where λ1+λ2≡0modλ, λ1+λ2≡1modN;

[0074] 3. Plaintext Encryption: The data to be encrypted is m∈Z N The encryption process is then [[m]] = (N+1) m ·r N modN 2 Where [[m]] is the encrypted ciphertext, (r is a positive integer less than N and relatively prime to N);

[0075] 4. Partial Decryption: Input ciphertext and part of the private key sk i (i∈[1,2]), after decryption we get

[0076] 5. Full Decryption: Input the two partially decrypted ciphertexts [m]1 and [m]2 to perform full decryption:

[0077] This scheme supports additive homomorphism, i.e., [[m1]]·[[m2]]=[[m1+m2]]. It also supports exponentiation, [[m]] k=[[k·m]],k∈Z N In particular, [[m]] N-1 =[[-m]].

[0078] For easier understanding, please refer to Figure 1 , Figure 1 This is a flowchart of the privacy-preserving federated learning method in the embodiments of this application, such as... Figure 1 As shown, specifically:

[0079] S1. The key distribution center generates a public key and a private key, and then splits the private key and distributes it to the main server, auxiliary server and client;

[0080] Step S1 specifically includes:

[0081] The key distribution center generates strong prime numbers p and q, and sets the public key pk = N = p·q and the private key sk = λ = lcm(p-1, q-1);

[0082] The key distribution center splits the public and private keys, generating key pairs for the master server and auxiliary server, and for the master server and client, respectively, as follows: sk1 = λ1, sk2 = λ2, λ1 + λ2 ≡ 0 mod λ, λ1 + λ2 ≡ 1 mod N. After completion, let... sk1 and sk2 are the private key pair between the i-th user and the master server, and the private key pair between the master server and the auxiliary server.

[0083] S2. The client obtains the global model hash value of the current training round from the blockchain and stores it in the file storage system. After obtaining the global gradient through the hash value in name access mode, it completes the training locally, performs preset processing on the local training gradient, obtains the updated encrypted gradient, and uploads it to the blockchain.

[0084] Step S2, which involves pre-setting the local training gradient, specifically includes:

[0085] The local training gradients are clipped by setting gradients whose absolute values ​​are less than a second preset threshold to 0.

[0086] It should be noted that in order to reduce the communication volume in the federated learning process, the gradients need to be clipped first. When the absolute value of a gradient is less than a set threshold, it is set to 0.

[0087] The local training gradients are normalized, specifically through the expression... Where g is the gradient vector of the client, g i Let be the i-th vector in g;

[0088] It should be noted that the gradient can be viewed as a vector with magnitude and direction. Therefore, the magnitude of the gradient also affects the global model. Clients launching poison attacks will use this to send larger gradients to influence the global model as much as possible. Therefore, we need to normalize the gradient before sending it to the main server.

[0089] The local training gradient is encrypted using the public key generated by the key distribution center, resulting in the encrypted gradient, specifically expressed as [[g]]=(N+1). g ·r N modN 2 , (r∈[1,N]), where [[g]] is the encryption gradient, r is a positive integer less than N and coprime to N, and N is the public key.

[0090] S3. The main server and the auxiliary server communicate based on the blockchain and file storage system to determine whether the encryption gradient uploaded by the client to the blockchain is normalized.

[0091] Step S3 specifically includes:

[0092] The main server adds random noise to the encrypted gradient [[g]] uploaded by the user to obtain... Then, partial decryption is performed using the key matched with the auxiliary server to obtain...

[0093] The main server will hash value and After the hash value is added to the chain, it will be stored in key-value pairs. as well as Store in a file storage system;

[0094] After the auxiliary server obtains the corresponding hash value from the blockchain, it downloads the gradient from the file storage system. as well as

[0095] The auxiliary server decrypts using its own private key. get After complete decryption, the noisy gradient plaintext is obtained.

[0096] The auxiliary server calculates the sum of squares for each gradient. The resulting hash value is then uploaded to the blockchain and stored in the file storage system;

[0097] After retrieving the result hash value from the blockchain, the main server downloads the sum of squares of the gradients from the file storage system.

[0098] The main server removes noise to obtain s, specifically through the expression:

[0099]

[0100]

[0101] in, For the k-th noisy component in the gradient vector uploaded by the client, r i ,(i∈[1,n]) represents the noise added to the i-th component. If the calculated s is 1, the gradient uploaded by the user is accepted; otherwise, it is discarded.

[0102] Partial decryption and complete decryption are as follows:

[0103]

[0104]

[0105] Where [[g]] is the encryption gradient, [g]1 and [g]2 are the results after decryption using a single private key, and N is the public key.

[0106] S4. The main server and auxiliary server communicate based on blockchain and file storage system to calculate the client's confidence level.

[0107] Step S4 is as follows:

[0108] Assume the encryption gradient uploaded by the user in round t+1 is... The global model gradient in round t is The gradient of the root dataset trained on the main server in round t+1 is: Noise is added to the encrypted gradient uploaded by the user, specifically as follows:

[0109]

[0110] in, Let be the cosine similarity between the i-th client and the global gradient from the previous round. Let cosine similarity be the value between the i-th client and the root dataset maintained by the server.

[0111] Main server upload and The hash value is put on the blockchain and the data is stored there. Stored in the file storage system as key-value pairs;

[0112] After the auxiliary server obtains the corresponding hash value from the blockchain, it partially decrypts it using its own private key, uploads the decryption result hash value to the blockchain, and stores it in the file storage system.

[0113] After the main server obtains the corresponding decryption result hash value from the blockchain, it downloads a portion of the decryption result from the file storage system and then uses full decryption to obtain the two cosine similarities.

[0114] The main server calculates the confidence score of the corresponding client using two cosine similarities, specifically:

[0115] R i =α·cos(θ) 1,i )+β·cos(θ 2,i );

[0116] R i Let α be the confidence level of the i-th user, and β be the corresponding weighting factors.

[0117] S5. Calculate the current client's contribution in this round. If the contribution is lower than the first preset threshold, mark the current client as a client that initiated a free-rider attack and kick the client out of the blockchain.

[0118] Step S5 calculates the current client's contribution in this round as follows:

[0119] τ1>τ2>τ3 and

[0120] Where τ is the time decay factor, These represent the contributions in the current round and the previous two rounds, respectively.

[0121] S6. The main server calculates the aggregate weights of each client based on the confidence level, completes the aggregation of the global model, and updates the global gradient of the global model to the file storage system.

[0122] In step S6, the main server calculates the aggregate weight of each client based on the confidence level to complete the aggregation of the global model.

[0123]

[0124] Among them Wei i The weights aggregated for the i-th user.

[0125] S7. Return to repeat steps S1 to S6 until the global model converges.

[0126] Please see Figure 2 , Figure 2 This is a schematic diagram of the privacy-preserving federated learning system in the embodiments of this application, such as... Figure 2 As shown, specifically:

[0127] Key distribution center 201, main server 202, auxiliary server 203, and client 204;

[0128] The key distribution center 201 generates a public key and a private key, and then splits the private key and distributes it to the main server 202, the auxiliary server 203 and the client 204;

[0129] Client 204 retrieves the global model hash value of the current training round from the blockchain and stores it in the file storage system 205. After obtaining the global gradient through the hash value in name access mode, it completes the training locally, performs preset processing on the global gradient, obtains the updated encrypted gradient, and uploads it to the blockchain 206.

[0130] The main server 202 and the auxiliary server 203 communicate based on the blockchain 206 and the file storage system 205 to determine whether the encryption gradient uploaded by the client 204 to the blockchain 206 is normalized.

[0131] The main server 202 and the auxiliary server 203 communicate based on the blockchain 206 and the file storage system 205 to calculate the confidence level of the client 204;

[0132] The master server 202 calculates the contribution of the current client 204 in this round. If the contribution is lower than the first preset threshold, the current client 204 is marked as the client 204 that initiated the free-rider attack and is kicked out of the blockchain.

[0133] The master server 202 calculates the aggregate weights of each client 204 based on the confidence level, completes the aggregation of the global model, and updates the global gradient of the global model to the file storage system 205;

[0134] Return to key distribution center 201 and repeat all steps until the global model converges.

[0135] This application also provides another privacy-preserving federated learning device, such as... Figure 3 As shown, for ease of explanation, only the parts related to the embodiments of this application are shown. For specific technical details not disclosed, please refer to the method section of the embodiments of this application. The terminal can be any terminal device including mobile phones, tablets, personal digital assistants (PDAs), point-of-sale (POS) terminals, in-vehicle computers, etc. Taking a mobile phone as an example:

[0136] Figure 3 This is a block diagram illustrating a portion of the structure of a mobile phone related to the terminal provided in the embodiments of this application. (Reference) Figure 3The mobile phone includes: a radio frequency (RF) circuit 1010, a memory 1020, an input unit 1030, a display unit 1040, a sensor 1050, an audio circuit 1060, a wireless fidelity (WiFi) module 1070, a processor 1080, and a power supply 1090, etc. Those skilled in the art will understand that... Figure 3 The mobile phone structure shown does not constitute a limitation on the mobile phone and may include more or fewer components than shown, or combine certain components, or have different component arrangements.

[0137] The following is combined with Figure 3 A detailed introduction to each component of a mobile phone:

[0138] The RF circuit 1010 can be used for receiving and transmitting signals during information transmission or calls. Specifically, it receives downlink information from the base station and processes it with the processor 1080; additionally, it transmits uplink data to the base station. Typically, the RF circuit 1010 includes, but is not limited to, an antenna, at least one amplifier, a transceiver, a coupler, a low-noise amplifier (LNA), a duplexer, etc. Furthermore, the RF circuit 1010 can also communicate wirelessly with networks and other devices. The aforementioned wireless communications may use any communication standard or protocol, including but not limited to Global System for Mobile Communications (GSM), General Packet Radio Service (GPRS), Code Division Multiple Access (CDMA), Wideband Code Division Multiple Access (WCDMA), Long Term Evolution (LTE), email, and Short Messaging Service (SMS).

[0139] The memory 1020 can be used to store software programs and modules. The processor 1080 executes various mobile phone functions and data processing by running the software programs and modules stored in the memory 1020. The memory 1020 may mainly include a program storage area and a data storage area. The program storage area may store the operating system, applications required for at least one function (such as sound playback function, image playback function, etc.), etc.; the data storage area may store data created according to the use of the mobile phone (such as audio data, phonebook, etc.). In addition, the memory 1020 may include high-speed random access memory, and may also include non-volatile memory, such as at least one disk storage device, flash memory device, or other volatile solid-state storage device.

[0140] The input unit 1030 can be used to receive input numerical or character information, and to generate key signal inputs related to user settings and function control of the mobile phone. Specifically, the input unit 1030 may include a touch panel 1031 and other input devices 1032. The touch panel 1031, also known as a touch screen, can collect touch operations performed by the user on or near it (such as operations performed by the user using a finger, stylus, or any suitable object or accessory on or near the touch panel 1031), and drive the corresponding connection devices according to a pre-set program. Optionally, the touch panel 1031 may include two parts: a touch detection device and a touch controller. The touch detection device detects the user's touch position and the signal generated by the touch operation, and transmits the signal to the touch controller; the touch controller receives touch information from the touch detection device, converts it into touch point coordinates, and sends it to the processor 1080, and can also receive and execute commands sent by the processor 1080. In addition, the touch panel 1031 can be implemented using various types such as resistive, capacitive, infrared, and surface acoustic wave. In addition to the touch panel 1031, the input unit 1030 may also include other input devices 1032. Specifically, other input devices 1032 may include, but are not limited to, one or more of the following: physical keyboard, function keys (such as volume control buttons, power buttons, etc.), trackball, mouse, joystick, etc.

[0141] The display unit 1040 can be used to display information input by the user or information provided to the user, as well as various menus of the mobile phone. The display unit 1040 may include a display panel 1041, which may optionally be configured as a Liquid Crystal Display (LCD), Organic Light-Emitting Diode (OLED), or similar display panel 1041. Further, a touch panel 1031 may cover the display panel 1041. When the touch panel 1031 detects a touch operation on or near it, it transmits the information to the processor 1080 to determine the type of touch event. Subsequently, the processor 1080 provides corresponding visual output on the display panel 1041 according to the type of touch event. Although in Figure 3 In this embodiment, the touch panel 1031 and the display panel 1041 are two separate components to realize the input and output functions of the mobile phone. However, in some embodiments, the touch panel 1031 and the display panel 1041 can be integrated to realize the input and output functions of the mobile phone.

[0142] The mobile phone may also include at least one sensor 1050, such as a light sensor, a motion sensor, and other sensors. Specifically, the light sensor may include an ambient light sensor and a proximity sensor. The ambient light sensor can adjust the brightness of the display panel 1041 according to the ambient light level, and the proximity sensor can turn off the display panel 1041 and / or the backlight when the phone is moved to the ear. As a type of motion sensor, an accelerometer sensor can detect the magnitude of acceleration in various directions (generally three axes). When stationary, it can detect the magnitude and direction of gravity and can be used for applications that recognize the phone's posture (such as landscape / portrait switching, related games, magnetometer posture calibration), vibration recognition-related functions (such as pedometer, taps), etc. Other sensors that may be configured in the mobile phone, such as gyroscopes, barometers, hygrometers, thermometers, and infrared sensors, will not be described in detail here.

[0143] The audio circuit 1060, speaker 1061, and microphone 1062 provide an audio interface between the user and the mobile phone. The audio circuit 1060 converts the received audio data into electrical signals and transmits them to the speaker 1061, where the speaker 1061 converts them into sound signals for output. On the other hand, the microphone 1062 converts the collected sound signals into electrical signals, which are then received by the audio circuit 1060, converted into audio data, and then processed by the processor 1080 before being transmitted via the RF circuit 1010 to, for example, another mobile phone, or the audio data can be output to the memory 1020 for further processing.

[0144] WiFi is a short-range wireless transmission technology. Through the WiFi module 1070, mobile phones can help users send and receive emails, browse web pages, and access streaming media, providing users with wireless broadband internet access. Although Figure 3 The WiFi module 1070 is shown, but it is understood that it is not an essential component of a mobile phone and can be omitted as needed without changing the essence of the invention.

[0145] The processor 1080 is the control center of the mobile phone, connecting various parts of the phone through various interfaces and lines. It executes software programs and / or modules stored in the memory 1020 and calls data stored in the memory 1020 to perform various functions and process data, thereby providing overall monitoring of the phone. Optionally, the processor 1080 may include one or more processing units; preferably, the processor 1080 may integrate an application processor and a modem processor, wherein the application processor mainly handles the operating system, user interface, and applications, while the modem processor mainly handles wireless communication. It is understood that the modem processor may not be integrated into the processor 1080.

[0146] The mobile phone also includes a power supply 1090 (such as a battery) that supplies power to various components. Preferably, the power supply can be logically connected to the processor 1080 through a power management system, thereby enabling functions such as charging, discharging, and power consumption management through the power management system.

[0147] Although not shown, mobile phones may also include a camera, Bluetooth module, etc., which will not be described in detail here.

[0148] In this embodiment of the application, the processor 1080 included in the terminal also has the following functions:

[0149] S1. The key distribution center generates a public key and a private key, and then splits the private key and distributes it to the main server, auxiliary server and client;

[0150] S2. The client obtains the global model hash value of the current training round from the blockchain and stores it in the file storage system. After obtaining the global gradient through the hash value in name access mode, it completes the training locally, performs preset processing on the local training gradient, obtains the updated encrypted gradient, and uploads it to the blockchain.

[0151] S3. The main server and the auxiliary server communicate based on the blockchain and file storage system to determine whether the encryption gradient uploaded by the client to the blockchain is normalized.

[0152] S4. The main server and auxiliary server communicate based on blockchain and file storage system to calculate the client's confidence level.

[0153] S5. Calculate the current client's contribution in this round. If the contribution is lower than the first preset threshold, mark the current client as a client that initiated a free-rider attack and kick the client out of the blockchain.

[0154] S6. The main server calculates the aggregate weights of each client based on the confidence level, completes the aggregation of the global model, and updates the global gradient of the global model to the file storage system.

[0155] S7. Return to repeat steps S1 to S6 until the global model converges.

[0156] This application also provides a computer-readable storage medium for storing program code that executes any one of the implementation methods of the privacy-preserving federated learning method described in the foregoing embodiments.

[0157] This application provides a privacy-preserving federated learning method, system, and device. It employs a dual-server, dual-trapdoor homomorphic encryption scheme, which calculates two cosine similarities in a ciphertext environment to serve as the client's confidence level. Lower confidence levels indicate a higher likelihood that the client is initiating a poisoning attack, leading to the discarding of gradients uploaded by such clients and effectively resisting poisoning attacks. Simultaneously, since the server only accesses ciphertext data, it cannot infringe on the client's privacy, effectively resisting privacy inference attacks. A time decay factor is used to weight and calculate a user's contribution in the current round. If the contribution falls below a set threshold, the user is removed from the blockchain network, and their deposit in the blockchain is confiscated to reward users with outstanding contributions, thus resisting free-rider attacks based on this elimination mechanism. This addresses the issues of privacy leaks caused by semi-honest servers guessing user privacy information when receiving gradients, the inability of servers to determine whether users have launched poisoning attacks based on the dataset due to the nature of federated learning, resulting in defective aggregated models or inability to improve accuracy, and the free-riding attacks by some clients during training, allowing them to benefit from others' work without participating in training themselves.

[0158] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working processes of the systems, devices, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0159] The terms “first,” “second,” “third,” “fourth,” etc. (if present) in the specification and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this application described herein can be implemented, for example, in orders other than those illustrated or described herein. Furthermore, the terms “comprising” and “having,” and any variations thereof, are intended to cover a non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0160] It should be understood that in this application, "at least one (item)" means one or more, and "more than" means two or more. "And / or" is used to describe the relationship between related objects, indicating that three relationships can exist. For example, "A and / or B" can represent three cases: only A exists, only B exists, and both A and B exist simultaneously, where A and B can be singular or plural. The character " / " generally indicates that the preceding and following related objects are in an "or" relationship. "At least one (item) of the following" or similar expressions refer to any combination of these items, including any combination of single or plural items. For example, at least one (item) of a, b, or c can represent: a, b, c, "a and b", "a and c", "b and c", or "a and b and c", where a, b, and c can be single or multiple.

[0161] In the several embodiments provided in this application, it should be understood that the disclosed systems, apparatuses, and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be an indirect coupling or communication connection between apparatuses or units through some interfaces, and may be electrical, mechanical, or other forms.

[0162] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0163] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0164] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes: USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, optical disks, and other media capable of storing program code.

[0165] The above-described embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application.

Claims

1. A privacy-preserving federated learning method, characterized in that, include: S1. The key distribution center generates a public key and a private key, and then splits the private key and distributes it to the main server, auxiliary server and client; S2. The client obtains the global model hash value of the current training round from the blockchain and stores it in the file storage system. After obtaining the global gradient through the hash value in name access mode, it completes the training locally, performs preset processing on the local training gradient, obtains the updated encrypted gradient, and uploads it to the blockchain. S3. The main server and the auxiliary server communicate based on the blockchain and file storage system to determine whether the encryption gradient uploaded by the client to the blockchain is normalized. S4. The main server and auxiliary server communicate based on blockchain and file storage system to calculate the client's confidence level. S5. Calculate the current client's contribution in this round. If the contribution is lower than the first preset threshold, mark the current client as a client that initiated a free-rider attack and kick the client out of the blockchain. S6. The main server calculates the aggregate weights of each client based on the confidence level, completes the aggregation of the global model, and updates the global gradient of the global model to the file storage system. S7. Return to repeat steps S1 to S6 until the global model converges; Step S2, which involves pre-setting the local training gradient, specifically includes: The local training gradients are clipped by setting gradients whose absolute values ​​are less than a second preset threshold to 0. The local training gradients are normalized, specifically through the expression... ,in, For the gradient vector of the client, for The i-th vector in; The encrypted gradient is obtained by encrypting the locally trained gradient using the public key generated by the key distribution center, specifically through the expression. ,in, For the encryption gradient, r is a positive integer less than N and coprime to N, and N is the public key; Step S3 specifically includes: The main server will upload encrypted gradients by users. Adding random noise to obtain Then, partial decryption is performed using the key matched with the auxiliary server to obtain... ; The main server will hash value and After the hash value is added to the chain, it will be stored in key-value pairs. as well as Store in a file storage system; The auxiliary server downloads the gradient from the file storage system after obtaining the corresponding hash value from the blockchain and ; The auxiliary server decrypts using its own private key. get After complete decryption, the noisy gradient plaintext is obtained. ; The auxiliary server calculates the sum of squares of each gradient The result hash value is then chained and stored in the file storage system. The master server downloads the sum of squares of gradients from the file storage system after obtaining the result hash value from the blockchain ; The main server obtains the noise removed , specifically through the expression: in, The k-th component of the gradient vector uploaded by the client has been given noise. For the noise added to the i-th component, when the calculated If the value is 1, the gradient uploaded by the user is accepted; otherwise, it is discarded.

2. The privacy-preserving federated learning method of claim 1, wherein, Step S1 specifically includes: The key distribution center generates strong prime numbers p and q, and lets public key , and private key ; The key distribution center splits the public and private keys, generating a key pair for the master server and auxiliary server, and for the master server and client, respectively: , After completion, order Let i be the private key pair between the i-th user and the master server. The private key pair between the primary server and the secondary server.

3. The privacy-preserving federated learning method of claim 1, wherein, Partial decryption and complete decryption are as follows: ； wherein, is the encrypted gradient, is the result of decryption using a single private key, N is the public key.

4. The privacy-preserving federated learning method of claim 1, wherein, Step S4 is as follows: Assume the encryption gradient uploaded by the user in round t+1 is... The global model gradient in round t is The gradient of the root dataset trained on the main server in round t+1 is: Noise is added to the encrypted gradient uploaded by the user, specifically: in, For the first The cosine similarity between each client and the previous round of global gradient. For the first Cosine similarity between the root dataset maintained by each client and the server; Main server upload and The hash value is added to the blockchain, and the data {hash( ), }、{hash( ), Stored in the file storage system as key-value pairs; After the auxiliary server obtains the corresponding hash value from the blockchain, it partially decrypts it using its own private key, uploads the decryption result hash value to the blockchain, and stores it in the file storage system. After the main server obtains the corresponding decryption result hash value from the blockchain, it downloads a portion of the decryption result from the file storage system and then uses full decryption to obtain the two cosine similarities. The main server calculates the confidence score of the corresponding client using two cosine similarities, specifically: ； the confidence for the i-th user, is a corresponding weight factor.

5. The privacy-preserving federated learning method of claim 4, wherein, Step S5 calculates the current client's contribution in this round as follows: ， ； wherein, is a time decay factor, are the contribution amounts of the current round and the previous two rounds, respectively.

6. The privacy-preserving federated learning method of claim 5, wherein, In step S6, the main server calculates the aggregate weight of each client based on the confidence level to complete the aggregation of the global model. wherein is the weight aggregated for the i-th user.

7. A privacy-preserving federated learning system, comprising: include: Key distribution center, main server, auxiliary server, and client; The key distribution center generates public and private keys, and then splits the private key and distributes it to the main server, auxiliary server, and client. The client obtains the global model hash value of the current training round from the blockchain and stores it in the file storage system. After obtaining the global gradient through the hash value in name access mode, it completes the training locally, performs preset processing on the local training gradient, obtains the updated encrypted gradient, and uploads it to the blockchain. The main server and the auxiliary server communicate based on the blockchain and file storage system to determine whether the encryption gradient of the client's uploaded blockchain is normalized; The primary server and secondary servers communicate based on blockchain and file storage systems to calculate the client's confidence level. The master server calculates the current client's contribution in this round. If the contribution is lower than the first preset threshold, the current client is marked as a client that launched a free-rider attack and is kicked out of the blockchain. The main server calculates the aggregate weights of each client based on the confidence level, completes the aggregation of the global model, and updates the global gradient of the global model to the file storage system; Return to the key distribution center and repeat all steps until the global model converges; Specifically, the client-side pre-processing of local training gradients includes: The local training gradients are clipped by setting gradients whose absolute values ​​are less than a second preset threshold to 0. The local training gradients are normalized, specifically through the expression... ,in, For the gradient vector of the client, for The i-th vector in; The encrypted gradient is obtained by encrypting the locally trained gradient using the public key generated by the key distribution center, specifically through the expression. ,in, For the encryption gradient, r is a positive integer less than N and coprime to N, and N is the public key; The primary and secondary servers communicate based on blockchain and file storage systems, determining whether the encryption gradient uploaded by the client to the blockchain is normalized. Specifically, this includes: The main server will upload encrypted gradients by users. Adding random noise to obtain Then, partial decryption is performed using the key matched with the auxiliary server to obtain... ; The main server will hash value and After the hash value is added to the chain, it will be stored in key-value pairs. as well as Store in a file storage system; The auxiliary server downloads the gradient from the file storage system after obtaining the corresponding hash value from the blockchain and ; The auxiliary server decrypts by its own private key obtained After the full decryption, the noisy gradient plaintext is obtained ; The auxiliary server calculates the sum of squares for each gradient. The resulting hash value is then uploaded to the blockchain and stored in the file storage system; The master server downloads the sum of squares of gradients from the file storage system after obtaining the result hash value from the blockchain ; The main server obtains the noise removed , in particular by the expression: in, The k-th component of the gradient vector uploaded by the client has been given noise. For the noise added to the i-th component, when the calculated If the value is 1, the gradient uploaded by the user is accepted; otherwise, it is discarded.

8. A privacy-preserving federated learning device, comprising: The device includes a processor and a memory: The memory is used to store program code and transmit the program code to the processor; The processor is configured to execute the privacy-preserving federated learning method according to any one of claims 1-6, based on instructions in the program code.

Images