Federated learning global model parameter acquisition method and system, and electronic device

Abstract

The application provides a federal learning global model parameter acquisition method and system and an electronic device. The method comprises the following steps: a first electronic device encrypts local model parameters participating in federal learning by using an encryption key to obtain encrypted local model parameters; the encryption key is split to obtain a plurality of sub-keys which are the same in number as the first electronic devices in the first electronic device group, one of the sub-keys is used as a first sub-key, and the other sub-keys are used as second sub-keys which are sent to other first electronic devices; the second sub-keys sent are received, and the first sub-key and the second sub-keys are aggregated to obtain a first aggregation key; and the encrypted local model parameters and the first aggregation key are sent to a second electronic device. The second electronic device obtains global model parameters according to the received encrypted local model parameters and the first aggregation key, thereby increasing the accuracy and security of the obtained global model parameters.

Description

Technical Field

[0001] This application relates to the field of federated machine learning technology, and in particular to a method, system, and electronic device for obtaining global model parameters for federated learning. Background Technology

[0002] In the training process of federated learning models, techniques such as differential privacy, homomorphic encryption, and secret sharing are often combined with federated learning to protect the privacy data during training. However, differential privacy requires adding noise to protect privacy, which leads to a loss of accuracy in the final model. Homomorphic encryption has a high computational cost and is only suitable for curious but honest adversaries; if an attacker can disrupt the participants' compliance with the protocol, the honest participants' model updates will be leaked. Secret sharing involves a large amount of communication, is time-consuming, and has relatively low efficiency in collaborative training. Therefore, existing methods for protecting privacy data in federated learning suffer from problems such as long training time, low accuracy of the obtained model parameters, and insufficient security in the transmission of model parameters. Summary of the Invention

[0003] This application provides a method for obtaining global model parameters in federated learning. In the process of obtaining global model parameters of a federated learning model, it can save model parameter acquisition time, ensure the security of model parameter transmission, and improve the accuracy of the obtained global model parameters.

[0004] To address the aforementioned technical problems, in a first aspect, this application provides a method for obtaining global model parameters in federated learning, applied to a federated learning system. The federated learning system includes a first electronic device group and a second electronic device group. The first electronic device group is a device group corresponding to the participants in the federated learning, and includes multiple first electronic devices. The second electronic device group is a device corresponding to the coordinator participating in the federated learning. The method includes: the first electronic device encrypting the local model parameters participating in the federated learning using an encryption key to obtain encrypted local model parameters; and the first electronic device splitting the encryption key to obtain multiple subkeys, the number of subkeys being the same as the number of first electronic devices in the first electronic device group; the first electronic device then... One of the multiple subkeys is saved locally as the first subkey. Each of the other subkeys, excluding the first subkey, is sent as a second subkey to the other first electronic devices in the first electronic device group. The first electronic device receives the second subkeys sent by the other first electronic devices in the first electronic device group. The first electronic device aggregates the first subkey and the second subkey to obtain a first aggregate key. The first electronic device sends the encrypted local model parameters and the first aggregate key to the second electronic device. The second electronic device receives the encrypted local model parameters and the first aggregate key sent by each first electronic device in the first electronic device group, and obtains the global model parameters based on the encrypted local model parameters and the first aggregate key.

[0005] In this implementation, the local model parameters participating in federated learning are encrypted using an encryption key to obtain encrypted local model parameters, which are then sent to the second electronic device. Encryption during the transmission of local model parameters ensures their privacy and security. Furthermore, each first electronic device splits and aggregates the encryption key used for encryption, then sends the resulting first aggregated key to the second electronic device. The second electronic device obtains the global model parameters based on multiple encrypted local model parameters and multiple first aggregated keys. This process only requires multiple encrypted local model parameters and multiple first aggregated keys to obtain the global model parameters, eliminating the need to split and combine the first aggregated keys to obtain the original encryption key. This saves parameter acquisition time, increases the security of obtaining global model parameters, and improves the accuracy of the global model parameters.

[0006] In one possible implementation of the first aspect above, the first electronic device splits the encryption key to obtain multiple subkeys, including: the first electronic device splits the encryption key according to a preset splitting step size to obtain multiple subkeys.

[0007] In this application, the encryption key is split according to a preset splitting step size to reduce the amount of network data transmission.

[0008] In one possible implementation of the first aspect above, obtaining global model parameters by encrypting local model parameters and a first aggregation key includes: aggregating each of the first aggregation keys to obtain a second aggregation key; aggregating each of the encrypted local model parameters to obtain aggregated local model parameters; and obtaining global model parameters based on the aggregated local model parameters and the second aggregation key.

[0009] In the implementation of this application, the global model parameters are obtained by aggregating the local model parameters and the second aggregation key, which increases the security of the process of obtaining the global model parameters and improves the accuracy of the global model parameters.

[0010] In one possible implementation of the first aspect above, the first electronic device encrypts the local model parameters participating in federated learning using an encryption key, including: the first electronic device adding the encryption key and the local model parameters; the first electronic device aggregating the first subkey and the second subkey, including: the first electronic device adding the first subkey and each of the second subkeys; and the second electronic device obtaining global model parameters based on the aggregated local model parameters and the second aggregated key, including: the second electronic device subtracting the aggregated local model parameters and the second aggregated key to obtain the global model parameters.

[0011] In this implementation, the encrypted local model parameters are obtained by adding the encryption key and the local model parameters. Then, the first subkey and each of the second subkeys are added to obtain the first aggregate key. Finally, the aggregated local model parameters and the second aggregate key are subtracted to obtain the global model parameters. By using addition and subtraction during the aggregation process to obtain the global model parameters, the accuracy of the obtained global model parameters is guaranteed.

[0012] In one possible implementation of the first aspect above, the global model parameters are obtained using the following formula:

[0013]

[0014] Among them, W g These are global model parameters. To aggregate local model parameters, Let k be the second aggregation key, k = 1, 2, ..., n, where n is the number of the first electronic devices.

[0015] In one possible implementation of the first aspect described above, the encryption key is obtained by generating a random vector and using the random vector as the encryption key. Using the generated random vector as the encryption key enhances the security of the encryption key.

[0016] Secondly, embodiments of this application provide a federated learning system for executing the federated learning global model parameter acquisition method described in the first aspect above. The federated learning system includes a first electronic device group and a second electronic device group. The first electronic device group is a device group corresponding to the participants in the federated learning, and the first electronic device group includes multiple first electronic devices. The second electronic device group is a device corresponding to the coordinator participating in the federated learning. The first electronic device is used to encrypt the local model parameters participating in the federated learning using an encryption key to obtain encrypted local model parameters. It also splits the encryption key to obtain multiple subkeys, the number of which is the same as the number of first electronic devices in the first electronic device group. Furthermore, one of the multiple subkeys is saved locally as the first subkey, and each of the other subkeys (excluding the first subkey) is sent as a second subkey to the other first electronic devices in the first electronic device group. The system also receives the second subkeys sent by the other first electronic devices in the first electronic device group. The first and second subkeys are aggregated to obtain a first aggregated key. The encrypted local model parameters and the first aggregated key are sent to the second electronic devices. The second electronic devices receive the encrypted local model parameters and the first aggregated key sent by each first electronic device in the first electronic device group, and obtain the global model parameters based on the encrypted local model parameters and the first aggregated key.

[0017] Thirdly, embodiments of this application provide an electronic device, including: a memory for storing a computer program, the computer program including program instructions; and a processor for executing the program instructions to cause the electronic device to perform the federated learning global model parameter acquisition method provided by the first aspect and / or any possible implementation of the first aspect.

[0018] Fourthly, embodiments of this application provide a computer-readable storage medium storing a computer program, the computer program including program instructions, the program instructions being executed by an electronic device to perform the federated learning global model parameter acquisition method provided by the first aspect and / or any possible implementation of the first aspect.

[0019] Fifthly, embodiments of this application provide a computer program product, including a computer program / instruction, which, when executed by a processor, implements the federated learning global model parameter acquisition method provided by the first aspect and / or any possible implementation of the first aspect.

[0020] The beneficial effects of this application are:

[0021] The method for obtaining global model parameters in federated learning provided in this application encrypts the local model parameters participating in federated learning using an encryption key to obtain encrypted local model parameters, which are then sent to a second electronic device. Encryption during the transmission of local model parameters ensures their privacy and security. Furthermore, each first electronic device splits and aggregates the encryption key used for encryption, then sends the resulting first aggregated key to the second electronic device. The second electronic device obtains the global model parameters based on multiple encrypted local model parameters and multiple first aggregated keys. This process only requires multiple encrypted local model parameters and multiple first aggregated keys to obtain the global model parameters, eliminating the need to split and combine the first aggregated keys to obtain the original encryption key. This saves parameter acquisition time, increases the security of the global model parameter acquisition process, and improves the accuracy of the global model parameters. Attached Figure Description

[0022] To more clearly illustrate the technical solution of this application, the accompanying drawings used in the description of the embodiments will be briefly introduced below.

[0023] Figure 1 This is a flowchart illustrating a method for obtaining global model parameters in federated learning, based on some implementations of this application.

[0024] Figure 2 This is a flowchart illustrating a process for obtaining global model parameters based on encrypted local model parameters and a first aggregation key, according to some implementation methods of this application.

[0025] Figure 3 This is a schematic diagram illustrating a method for obtaining global model parameters in federated learning, based on some implementations of this application.

[0026] Figure 4 This is a schematic diagram illustrating a federated machine learning method for aggregating security policies without model parameters, based on some implementations of this application.

[0027] Figure 5 This is a schematic diagram showing a comparison of the parameter training and data transmission time between a federated learning global model parameter acquisition method and the original federated machine learning method without a model aggregation security policy, based on some implementation methods of this application.

[0028] Figure 6 This is a schematic diagram of a federated learning system based on some implementations of this application;

[0029] Figure 7 This is a schematic diagram of the structure of an electronic device according to some implementations of this application. Detailed Implementation

[0030] The technical solution of this application will be described in further detail below with reference to the accompanying drawings.

[0031] First, a brief introduction to the relevant concepts of federated learning will be given.

[0032] Federated learning (also known as federated machine learning) is a mechanism that coordinates multiple participants to jointly train a model. It allows each participant to keep their private data locally, ensuring their control over it. However, federated machine learning introduces a model parameter exchange process. During this process, attackers can infer sensitive information in a user's data based on the user's model parameters, and may even be able to reverse engineer the user's private data.

[0033] Currently, technologies such as differential privacy, homomorphic encryption, and secret sharing are combined with federated learning to prevent attackers from learning sensitive data from the local model parameter updates of participants, thus protecting privacy during the secure aggregation process of federated model training.

[0034] Differential privacy is a prominent technique that protects privacy by adding noise. While differential privacy protects the privacy of participants' private and sensitive data, noise can lead to a loss of accuracy in the final model. The stronger the robustness of differential privacy guarantees, the more noise needs to be added to the data, and the more severe the loss of accuracy in the final model.

[0035] Homomorphic encryption provides a cryptographic solution that allows a terminal to encrypt data before sending it to a third party. The third party can then perform specific operations on the encrypted data without obtaining the plaintext information. The terminal receives the encrypted operation result, decrypts it, and finds that the result matches the result of the specific operation performed on the plaintext. Because each participant encrypts their model update before sending it to the server, the server cannot obtain any information from the encrypted model update. However, homomorphic encryption is computationally expensive, and this method is only suitable for curious but honest adversary models. That is, if an attacker can compromise the participants' compliance with the protocol, the honest participant's model update will be leaked by the attacker.

[0036] Secret sharing is a crucial mechanism in secure multi-party computation. The idea behind secret sharing is to divide a secret into appropriate fragments, each managed by a different participant. A single participant cannot recover the secret information; only through collaboration among several participants can the secret message be restored. Taking a threshold secret sharing scheme as an example: the protocol divides a secret J into w secret fragments s, which are sent to w users for safekeeping. The secret can only be recovered when at least f (f ≤ w) users collaborate. When secret sharing is combined with federated machine learning, model parameters are the objects that need to be secretly shared. In deep learning, model parameters often start in the millions. After training, the additional process of parameter splitting and sharing further increases the communication volume and time consumption of secret sharing, impacting the efficiency of collaborative training.

[0037] In summary, existing methods for protecting privacy data in federated learning suffer from problems such as long processing time, low accuracy of the obtained model parameters, and insufficient security in the transfer of model parameters.

[0038] Based on this, this application provides a method for obtaining global model parameters in federated learning. In the process of obtaining global model parameters of a federated learning model, it can save model parameter acquisition time, ensure the security of model parameter transmission, and improve the accuracy of the obtained global model parameters.

[0039] Next, refer to Figures 1-5 This paper describes in detail the implementation process and advantages of the method for obtaining global model parameters in federated learning.

[0040] In one implementation of this application, the method for obtaining global model parameters of federated learning is applied to a federated learning system. The federated learning system includes a first electronic device group and a second electronic device group. The first electronic device group is the device group corresponding to the participants in the federated learning and includes multiple first electronic devices. The second electronic device group is the device corresponding to the coordinator in the federated learning. Each first electronic device in the first electronic device group can communicate with each other, and each first electronic device and the second electronic device can also communicate with each other.

[0041] like Figure 1 As shown, the method for obtaining global model parameters in this federated learning approach includes the following steps:

[0042] S100: The first electronic device encrypts the local model parameters participating in federated learning using an encryption key to obtain encrypted local model parameters.

[0043] S200: The first electronic device splits the encryption key to obtain multiple subkeys, the number of which is the same as the number of first electronic devices in the first electronic device group.

[0044] S300: The first electronic device saves one of the multiple subkeys as the first subkey locally, and sends each of the other subkeys (excluding the first subkey) as a second subkey to the other first electronic devices in the first electronic device group.

[0045] S400: The first electronic device receives the second subkey sent by other first electronic devices in the first electronic device group.

[0046] S500: The first electronic device aggregates the first subkey and the second subkey to obtain the first aggregated key.

[0047] S600: The first electronic device sends the encrypted local model parameters and the first aggregation key to the second electronic device (i.e., the device corresponding to the coordinator).

[0048] S700: The second electronic device receives the encrypted local model parameters and the first aggregation key sent by each first electronic device in the first electronic device group, and obtains the global model parameters based on the encrypted local model parameters and the first aggregation key.

[0049] This method for obtaining global model parameters in federated learning encrypts the local model parameters participating in the federated learning process using an encryption key, resulting in encrypted local model parameters. These encrypted local model parameters are then sent to a second electronic device. Encryption during the transmission of local model parameters ensures their privacy and security. Furthermore, each first electronic device splits and aggregates the encryption key used for encryption, then sends the resulting first aggregated key to the second electronic device. The second electronic device then obtains the global model parameters based on multiple encrypted local model parameters and multiple first aggregated keys. This process eliminates the need to split and combine the first aggregated keys to obtain the original encryption key, saving parameter acquisition time, increasing the security of the global model parameter acquisition process, and improving the accuracy of the global model parameters.

[0050] The following provides a more detailed description of S100-S700. In the following text, the first electronic device will be referred to as the participant and the second electronic device as the coordinator.

[0051] First, in step S100, the first electronic device encrypts the local model parameters participating in federated learning using an encryption key, and then provides a detailed description of the encrypted local model parameters.

[0052] Here, the first electronic device refers to the first electronic device corresponding to all participants in federated learning in the first electronic device group. Each first electronic device needs to encrypt the local model parameters to obtain encrypted local model parameters.

[0053] In one implementation of this application, the encryption key is obtained by: a first electronic device generating a random vector and using the random vector as the encryption key.

[0054] For all participants, a random vector is constructed. The local model parameters are obfuscated (i.e., encrypted) using this noise vector before being sent to the coordinator, resulting in encrypted local model parameters. These encrypted local model parameters are then sent to the coordinator. Because the actual local model parameters have been obfuscated, other parties cannot know the true local model parameters, thus ensuring the security of the participants' local model parameters. The random vector generated by each first electronic device can be the same or different.

[0055] Next, in steps S200-S500, the first electronic device splits the encryption key to obtain multiple subkeys. The first electronic device saves one of the multiple subkeys as the first subkey locally, and sends each of the other subkeys (excluding the first subkey) as a second subkey to other first electronic devices in the first electronic device group. It also receives the second subkeys sent by other first electronic devices in the first electronic device group. Finally, the first subkey and the second subkey are aggregated to obtain the content of the first aggregated key, which will be explained in detail.

[0056] Specifically, the participants split the encryption key into multiple subkeys, for example, n parts. Each participant keeps one part, and the remaining n-1 parts are sent to the other n-1 participants. Simultaneously, each participant receives a second subkey from the other participants and aggregates the received n-1 subkeys with its own first subkey to obtain a first aggregated key, which is then sent to the coordinator. The aggregation process can specifically involve adding the received n-1 subkeys to its own first subkey.

[0057] It should be noted that steps S300 and S400 do not have a fixed order. While the first electronic device is splitting its local encryption key to obtain a first subkey and a second subkey, and sending the second subkey to other first electronic devices, it will also receive a second subkey from other first electronic devices. Alternatively, the first electronic device may have already received the second subkey from other first electronic devices before starting to split its local encryption key.

[0058] In this process, each participant is unaware of the other participants' actual encryption keys. However, unlike secretly sharing model parameters, this method secretly shares the encryption key. The generation of the encryption key and the training of local model parameters can occur simultaneously. Furthermore, the precision requirement of the encryption key can be set arbitrarily. In some implementations, the precision of the encryption key can be set to be lower than that of the local model parameters. For example, the encryption key can be represented using 2 bytes (half precision), while the local model parameters require single precision (4 bytes) or double precision (8 bytes). Secretly sharing the encryption key has significant advantages over secretly sharing local model parameters in terms of network data transmission volume and runtime.

[0059] In generating multiple subkeys, this application introduces a random value step size λ to reduce the amount of network data transmission. That is, in one implementation of this application, the first electronic device splits the encryption key to obtain multiple subkeys, including: the first electronic device splits the encryption key according to a preset splitting step size to obtain multiple subkeys.

[0060] Specifically, the preset splitting step size can be obtained using the following formula:

[0061] When s = 1, λ = 1; when s > 1, 1 ≤ λ ≤ H (H is the maximum step size), where s is the number of subkeys and λ is the preset splitting step size.

[0062] Correspondingly, the network transmission share for each subkey is:

[0063]

[0064] (Where: j∈[1,n]&j≠k, s∈(1,n])

[0065] in, The encryption key for the first electronic device. This is the first subkey obtained after splitting the encryption key. It is a second subkey other than the first subkey. The first aggregation key obtained by the first electronic device. The first subkey stored locally in the first electronic device. This is the second subkey received from another first electronic device.

[0066] Next, in step S600, the first electronic device sends the encrypted local model parameters and the first aggregation key to the second electronic device.

[0067] That is, multiple participants send their respective encrypted local model parameters and first aggregation key to the coordinator.

[0068] It should be noted that in step S100, after the first electronic device obtains the encrypted local model parameters, it can directly send the encrypted local model parameters to the second electronic device, and then perform encryption key splitting and aggregation processing, sending the first aggregated key to the second electronic device. Alternatively, after splitting and aggregating the encryption key, the encrypted local model parameters and the first aggregated key can be sent together to the second electronic device.

[0069] Finally, a detailed explanation is given of step S700, in which the second electronic device receives the encrypted local model parameters and the first aggregation key sent by each first electronic device in the first electronic device group, and obtains the content of the global model parameters based on the encrypted local model parameters and the first aggregation key.

[0070] In one implementation of this application, global model parameters are obtained based on the encrypted local model parameters and the first aggregation key, such as... Figure 2 As shown, it includes the following steps:

[0071] S710: Aggregate the first aggregation keys to obtain the second aggregation key.

[0072] S720: Aggregate the parameters of each encrypted local model to obtain aggregated local model parameters.

[0073] S730: Obtain global model parameters based on the aggregated local model parameters and the second aggregated key.

[0074] Understandably, the S720 and S730 can process data in parallel, without regard to the order of processing. Global model parameters refer to a set of local model parameters received by the coordinator from each participant.

[0075] The coordinator receives encrypted local model parameters from multiple participants and aggregates them to obtain aggregated local model parameters. Simultaneously, the coordinator also receives a first aggregation key from multiple participants and aggregates it to obtain a second aggregation key. Subtracting the second aggregation key from the aggregated local model parameters reconstructs the true global model parameters. This addresses the issue of model accuracy loss in differential privacy and increases the accuracy of the global model parameters.

[0076] In one implementation of this application, a first electronic device encrypts the local model parameters participating in federated learning using an encryption key, including: the first electronic device adding the encryption key and the local model parameters; the first electronic device aggregating a first subkey and a second subkey, including: the first electronic device adding the first subkey and each of the second subkeys; and a second electronic device obtaining global model parameters based on the aggregated local model parameters and the second aggregated key, including: the second electronic device subtracting the aggregated local model parameters and the second aggregated key to obtain the global model parameters.

[0077] In one implementation of this application, the global model parameters are obtained using the following formula:

[0078]

[0079] Among them, W g These are global model parameters. Encrypted local model parameters received by the second electronic device To aggregate local model parameters, The first aggregation key, Let k be the second aggregation key, k = 1, 2, ..., n, where n is the number of the first electronic devices.

[0080] Next, in one implementation of this application, combined with a specific application scenario, the method for obtaining global model parameters of federated learning provided in this application will be explained.

[0081] First, the symbols used in this implementation are explained as shown in the table below:

[0082]

[0083] The method for obtaining global model parameters for federated learning provided in this application includes:

[0084] For all participating parties p k ∈P (e.g., participant j, participant k, participant n, i.e., participant j, participant k, participant n), construct a random noise vector. (i.e., the encryption key, which is also the key of participant p) k The local model parameters are obfuscated with this random noise vector before being sent to the coordinator C. For example, such as... Figure 3 As shown, after participant j generates local model parameters... Next, construct a random noise vector. Local model parameters use replace, Will Send to the coordinator. Similarly, participant k generates local model parameters. Next, construct a random noise vector. Local model parameters use replace, Will Send to the coordinator, participant n generates local model parameters. Next, construct a random noise vector. Local model parameters use replace, Will The parameters are sent to the coordinator. Because the actual local model parameters have been obfuscated with noise, other parties cannot know the true local model parameters, thus ensuring the safety of the participating parties' model parameters.

[0085] Next, the participants will use random noise vectors Decompose into n parts, that is Participants keep a copy for themselves. (i.e., the first subkey), the remaining n-1 copies The second subkey (i.e., the second subkey) is sent to the remaining n-1 participants. At the same time, each participant will also receive a noise share from the other participants and combine the received n-1 shares with its own first noise share. Adding them together gives (i.e., the first aggregation key), that is (where: j∈[1,n]&j≠k,s∈(1,n]), then... Send it to the coordinator. For example, such as... Figure 3 As shown, participants j, k, and n each transmit their own encryption keys. The three keys obtained from the decomposition are each kept as a separate entity, with the remaining two shares being sent between participants. Participant j will receive multiple key shares obtained from the decomposition. The keys are sent to participant k and participant n respectively. Participant n will then decompose the keys into multiple key shares. The received key shares are sent to participant j and participant k respectively. Then, participant j, participant k, and participant n add the received key shares together and send them to the coordinator.

[0086] Regarding the generation of noise share, for When s = 1, λ = 1; when s > 1, 1 ≤ λ ≤ H (H is the maximum step size). Therefore, the share of network transmission is:

[0087] The coordinator receives obfuscation parameters from n participants. And aggregate them to get At the same time, the coordinator also receives a share of the noise aggregation from the n participants. And aggregate them to obtain N g The aggregation of obfuscation parameters Subtract the noise N after aggregation g This allows us to reconstruct the true global model parameters W. g .in, Where: k∈[1,n], Where: k∈[1,n], For example, such as Figure 3 As shown, global model parameters

[0088] It should be noted that each participant's local model parameter training and random noise-related transactions operate in two separate processes, running completely in parallel. The local model parameter training process utilizes GPU resources, while the local random noise-related transactions utilize CPU resources. Process P N The task is to model the random noise vector. Generation; random noise share Transmission and reception; aggregation of random noise components and its Sending; Receiving process P W Request local model random noise and send the local model random noise via pipe or shared memory. Process P W The task is to train local model parameters based on the global model parameters from the previous round and local private data; and to train local model parameters from local process P. N The random noise of the current round model parameters is obtained and calculated. And send it to the coordinator.

[0089] The method for obtaining global model parameters for federated learning provided in this application, also known as a model secure aggregation method based on secret shared noise and data obfuscation, does not suffer any loss in model accuracy and can be deduced from the following formula:

[0090]

[0091]

[0092]

[0093] Therefore, the coordinating party conducts After the operation, the global parameters W of the model can be obtained without any loss. g .

[0094] Furthermore, such as Figure 4As shown, a federated training method without a security policy is provided, in which participants j, k, and n respectively transmit their local model parameters. Send them separately to the coordinator, who will then... Additive aggregation is performed to obtain global model parameters. During the process of participants j, k, and n sending their local model parameters to the coordinator, no encryption or obfuscation of the local model parameters is performed. Therefore, the local model parameters are easily guessed by attackers during transmission.

[0095] The federated learning global model parameter acquisition method provided in this application differs from the secret-sharing model parameter method. In the secret-sharing method, model parameter training and secret sharing must be completed sequentially, while this application allows them to be completed in parallel. Therefore, in terms of training time, such as... Figure 5 As shown, the training and obfuscated parameter transmission time of the federated machine learning model parameters in this method is similar to that of the federated machine learning model parameter training and transmission time without a security policy. That is, because this application uses two parallel processes, GPU and CPU, the model security aggregation method based on secret shared noise and data obfuscation is essentially equivalent to the original federated training method without a security policy in terms of training time.

[0096] The model-based secure aggregation method using secret sharing noise and data obfuscation is provably secure. As shown in the above formula, so, In this application, each participating party only releases the following data: Therefore, this application can guarantee the security of local model parameters if other participants do not collude. Even in the event of collusion, this application can still guarantee the security of the honest participant's model parameters as long as at least one other participant is honest. Assuming the participant set P is divided into two parts: one part of the participants are honest and will not publish or disclose their true model parameters or random noise vectors to other participants; the other part of the participants are dishonest and will publish or disclose their model parameters or random noise vectors to other participants. The security can be proven as follows:

[0097] Ρ = honest∪dishonest

[0098]

[0099] The above formula shows that as long as the number of participants in the honest participants' set is greater than or equal to 2, the participants in the dishonest participants' set cannot obtain the local model parameters in the honest participants' set.

[0100] Secondly, the implementation of this application provides a federated learning system, please refer to [link to relevant documentation]. Figure 6 This system is used to execute the federated learning global model parameter acquisition method described in the first aspect above. The federated learning system includes a first electronic device group and a second electronic device group. The first electronic device group is the device group corresponding to the participants in the federated learning, and includes multiple first electronic devices. The second electronic device is the device corresponding to the coordinator participating in the federated learning. The first electronic device is used to encrypt the local model parameters participating in the federated learning using an encryption key to obtain encrypted local model parameters; and to split the encryption key to obtain multiple subkeys, the number of which is the same as the number of first electronic devices in the first electronic device group; and to save one of the multiple subkeys as the first subkey locally, and to send each of the other subkeys (excluding the first subkey) as a second subkey to the other first electronic devices in the first electronic device group; and to receive the second subkeys sent by the other first electronic devices in the first electronic device group; to aggregate the first subkeys and the second subkeys to obtain a first aggregate key; and to send the encrypted local model parameters and the first aggregate key to the second electronic device. The second electronic device is used to receive the encrypted local model parameters and the first aggregate key sent by each first electronic device in the first electronic device group, and to obtain the global model parameters based on the encrypted local model parameters and the first aggregate key.

[0101] Understandable, Figure 6 The first electronic device group shown only includes 3 first electronic devices, but in practice the number of first electronic devices can be adjusted according to actual needs.

[0102] Please see Figure 7 , Figure 7 The diagram shown is a structural block diagram of an electronic device provided in this application. The electronic device may include one or more processors 1002, system control logic 1008 connected to at least one of the processors 1002, system memory 1004 connected to the system control logic 1008, non-volatile memory (NVM) 1006 connected to the system control logic 1008, and network interface 1010 connected to the system control logic 1008.

[0103] Processor 1002 may include one or more single-core or multi-core processors. Processor 1002 may include any combination of general-purpose processors and special-purpose processors (e.g., graphics processors, application processors, baseband processors, etc.). In the implementation described herein, processor 1002 may be configured to perform a federated learning global model parameter acquisition method.

[0104] In some implementations, system control logic 1008 may include any suitable interface controller to provide any suitable interface to at least one of the processors 1002 and / or any suitable device or component communicating with system control logic 1008.

[0105] In some implementations, system control logic 1008 may include one or more memory controllers to provide an interface to system memory 1004. System memory 1004 may be used to load and store data and / or instructions. In some implementations, system memory 1004 of the electronic device may include any suitable volatile memory, such as suitable dynamic random access memory (DRAM).

[0106] NVM / memory 1006 may include one or more tangible, non-transitory computer-readable media for storing data and / or instructions. In some implementations, NVM / memory 1006 may include any suitable non-volatile memory such as flash memory and / or any suitable non-volatile storage device, such as at least one of a hard disk drive (HDD), a compact disc (CD) drive, and a digital versatile disc (DVD) drive.

[0107] NVM / Memory 1006 may include a portion of storage resources installed on an electronic device, or it may be accessible by the device, but is not necessarily part of the device. For example, NVM / Memory 1006 may be accessed over a network via network interface 1010.

[0108] Specifically, system memory 1004 and NVM / memory 1006 may each include a temporary copy and a permanent copy of instruction 1020. Instruction 1020 may include instructions that, when executed by at least one of processors 1002, cause the electronic device to implement the aforementioned federated learning global model parameter acquisition method. In some implementations, instruction 1020, hardware, firmware, and / or its software components may additionally / alternatively reside in system control logic 1008, network interface 1010, and / or processor 1002.

[0109] Network interface 1010 may include a transceiver for providing a radio interface for an electronic device, enabling communication with any other suitable device (such as a front-end module, antenna, etc.) via one or more networks. In some implementations, network interface 1010 may be integrated into other components of the electronic device. For example, network interface 1010 may be integrated into at least one of processor 1002, system memory 1004, NVM / memory 1006, and firmware device (not shown) with instructions, wherein when at least one of processor 1002 executes the instructions, the electronic device implements the aforementioned federated learning global model parameter acquisition method.

[0110] The network interface 1010 may further include any suitable hardware and / or firmware to provide a multiple-input multiple-output radio interface. For example, the network interface 1010 may be a network adapter, a wireless network adapter, a telephone modem, and / or a wireless modem.

[0111] In one implementation, at least one of the processors 1002 may be packaged together with the logic of one or more controllers for system control logic 1008 to form a system-in-a-package (SiP). In another implementation, at least one of the processors 1002 may be integrated on the same die with the logic of one or more controllers for system control logic 1008 to form a system-on-a-chip (SoC).

[0112] The electronic device may further include an input / output (I / O) device 1012. The I / O device 1012 may include a user interface enabling a user to interact with the electronic device; the peripheral component interface is designed to allow peripheral components to also interact with the electronic device. In some implementations, the electronic device may also include sensors for determining at least one type of environmental condition and location information relevant to the electronic device.

[0113] In some implementations, the user interface may include, but is not limited to, a display (e.g., a liquid crystal display, a touch screen display, etc.), a speaker, a microphone, one or more cameras (e.g., a still image camera and / or a video camera), a flashlight (e.g., a light-emitting diode flash), and a keyboard.

[0114] In some implementations, the peripheral component interface may include, but is not limited to, non-volatile memory ports, audio jacks, and power interfaces.

[0115] In some implementations, the sensors may include, but are not limited to, gyroscope sensors, accelerometers, proximity sensors, ambient light sensors, and positioning units. The positioning unit may also be part of or interact with network interface 1010 to communicate with components of the positioning network (e.g., Global Positioning System (GPS) satellites).

[0116] It is understood that the illustrative structures of the embodiments of the present invention do not constitute a specific limitation on the electronic device. In other embodiments of this application, the electronic device may include more or fewer components than illustrated, or combine some components, or split some components, or have different component arrangements. The illustrated components may be implemented in hardware, software, or a combination of software and hardware.

[0117] Program code can be applied to input instructions to perform the functions described herein and generate output information. The output information can be applied to one or more output devices in a known manner. For the purposes of this application, the processing system includes any system having a processor such as, for example, a digital signal processor (DSP), a microcontroller, an application-specific integrated circuit (ASIC), or a microprocessor.

[0118] The program code can be implemented using a high-level procedural language or an object-oriented programming language to communicate with the processing system. Assembly language or machine language can also be used when needed. In fact, the mechanisms described in this paper are not limited to any particular programming language. In either case, the language can be a compiled language or an interpreted language.

[0119] One or more aspects of at least one implementation can be implemented by representational instructions stored on a computer-readable storage medium, the instructions representing various logics in a processor, which, when read by a machine, cause the machine to create logic for performing the techniques described herein. These representations, referred to as “IP cores,” can be stored on tangible computer-readable storage media and provided to multiple customers or production facilities for loading into manufacturing machines that actually manufacture the logic or processor.

[0120] It should be noted that some structural or methodological features may be shown in the accompanying drawings in a specific arrangement and / or order. However, it should be understood that such a specific arrangement and / or order may not be necessary. Rather, in some implementations, these features may be arranged in a different manner and / or order than those shown in the illustrative drawings. Furthermore, including structural or methodological features in a particular figure does not imply that such features are required in all implementations, and in some implementations, these features may be omitted or may be combined with other features.

[0121] It should be noted that the terms "first" and "second" are used only to distinguish descriptions and should not be interpreted as indicating or implying relative importance.

[0122] It should be noted that some structural or methodological features may be shown in the accompanying drawings in a specific arrangement and / or order. However, it should be understood that such a specific arrangement and / or order may not be necessary. Rather, in some embodiments, these features may be arranged in a manner and / or order different from that shown in the illustrative drawings. Furthermore, including structural or methodological features in a particular figure does not imply that such features are required in all embodiments, and in some embodiments, these features may be omitted or may be combined with other features.

[0123] Although this application has been illustrated and described with reference to certain preferred embodiments, those skilled in the art should understand that the above description is a further detailed explanation of the application in conjunction with specific embodiments, and should not be construed as limiting the specific implementation of the application to these descriptions. Those skilled in the art can make various changes in form and detail, including some simple deductions or substitutions, without departing from the spirit and scope of this application.

Claims

1. A method for obtaining global model parameters in federated learning, characterized in that, The method is applied to a federated learning system, which includes a first electronic device group and a second electronic device group. The first electronic device group is a device group corresponding to the participants in the federated learning, and the first electronic device group includes multiple first electronic devices. The second electronic device group is a device corresponding to the coordinator participating in the federated learning. The first electronic device encrypts the local model parameters participating in federated learning using an encryption key to obtain encrypted local model parameters; and The first electronic device splits the encryption key into multiple sub-keys, and the number of sub-keys is the same as the number of the first electronic devices in the group of first electronic devices; The first electronic device saves one of the plurality of subkeys as a first subkey locally, and sends each of the other subkeys (excluding the first subkey) as a second subkey to the other first electronic devices in the group of first electronic devices; and The first electronic device receives the second subkey sent by other first electronic devices in the first electronic device group; The first electronic device aggregates the first subkey and the second subkey to obtain a first aggregated key; The first electronic device sends the encrypted local model parameters and the first aggregation key to the second electronic device; The second electronic device receives the encrypted local model parameters and the first aggregation key sent by each of the first electronic devices in the first electronic device group, and obtains the global model parameters based on the encrypted local model parameters and the first aggregation key.

2. The method for obtaining global model parameters in federated learning as described in claim 1, characterized in that, The first electronic device splits the encryption key to obtain multiple sub-keys, including: The first electronic device splits the encryption key according to a preset splitting step size to obtain the plurality of subkeys.

3. The method for obtaining global model parameters in federated learning as described in claim 1 or 2, characterized in that, Based on the encrypted local model parameters and the first aggregation key, global model parameters are obtained, including: The first aggregated keys are aggregated to obtain the second aggregated key; and The encrypted local model parameters are aggregated to obtain aggregated local model parameters. The global model parameters are obtained based on the aggregated local model parameters and the second aggregated key.

4. The method for obtaining global model parameters in federated learning as described in claim 3, characterized in that, The first electronic device encrypts the local model parameters participating in federated learning using an encryption key, including: the first electronic device performing an addition operation on the encryption key and the local model parameters; The first electronic device performs aggregation processing on the first subkey and the second subkey, including: The first electronic device performs addition on the first subkey and each of the second subkeys; The second electronic device obtains the global model parameters based on the aggregated local model parameters and the second aggregated key, including: The second electronic device performs a subtraction operation between the aggregated local model parameters and the second aggregated key to obtain the global model parameters.

5. The method for obtaining global model parameters in federated learning as described in claim 4, characterized in that, The global model parameters are obtained using the following formula: Among them, W g For the global model parameters, For the aggregated local model parameters, Let k be the second aggregation key, k = 1, 2, ..., n, where n is the number of the first electronic devices.

6. The method for obtaining global model parameters in federated learning as described in claim 5, characterized in that, The encryption key is obtained in the following way: The first electronic device generates a random vector and uses the random vector as the encryption key.

7. A federated learning system, characterized in that, The federated learning system includes a first electronic device group and a second electronic device group. The first electronic device group is the device group corresponding to the participants in the federated learning, and the first electronic device group includes multiple first electronic devices. The second electronic device group is the device corresponding to the coordinator participating in the federated learning. The first electronic device is used to encrypt the local model parameters participating in federated learning using an encryption key to obtain encrypted local model parameters; and to split the encryption key to obtain multiple subkeys, the number of which is the same as the number of first electronic devices in the first electronic device group; and to save one of the multiple subkeys as a first subkey locally, and to send each of the multiple subkeys other than the first subkey as a second subkey to the other first electronic devices in the first electronic device group; and to receive the second subkeys sent by the other first electronic devices in the first electronic device group; to aggregate the first subkey and the second subkey to obtain a first aggregate key; and to send the encrypted local model parameters and the first aggregate key to the second electronic device. The second electronic device is used to receive the encrypted local model parameters and the first aggregation key sent by each of the first electronic devices in the first electronic device group, and to obtain global model parameters based on the encrypted local model parameters and the first aggregation key.

8. An electronic device, characterized in that, include: A memory for storing computer programs, the computer programs including program instructions; A processor is configured to execute the program instructions to cause the electronic device to perform the federated learning global model parameter acquisition method as described in any one of claims 1-6.

9. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program, which includes program instructions that are executed by an electronic device to cause the electronic device to perform the federated learning global model parameter acquisition method as described in any one of claims 1-7.

10. A computer program product, characterized in that, It includes a computer program / instruction that, when executed by an electronic device, implements the federated learning global model parameter acquisition method as described in any one of claims 1-7.

Images