Privacy-preserving federated learning method and related equipment

Abstract

The disclosed embodiment provides a method for federated learning based on privacy protection and related equipment. The method includes: obtaining a row sampling sequence set and a column sampling sequence set according to the weight matrix of the neural network; generating a low-dimensional row representation matrix and a low-dimensional column feature matrix according to the row sampling sequence set and the column sampling sequence set; receiving a public key sent by the edge node, encrypting the symmetric key according to the public key, obtaining a ciphertext, and sending the ciphertext to the edge node; generating a symmetric key, and encrypting the row sampling sequence set and the column sampling sequence set according to the symmetric key to obtain an encrypted row sampling sequence set and a column sampling sequence set; generating an initialization parameter set according to the low-dimensional row representation matrix, the low-dimensional column feature matrix, the encrypted row sampling sequence set and the encrypted column sampling sequence set, and sending the initialization parameter set to the central server.

Description

Technical Field

[0001] The present disclosure relates to the field of computer technology, and in particular to a privacy-protection-based federated learning method and related equipment. Background Art

[0002] In recent years, with the rapid development of artificial intelligence and the popularity of mobile devices, machine learning methods have shown obvious advantages in data analysis, security diagnosis, automation systems and other fields. However, machine learning methods need to collect a large amount of user data to train models. When training models in a multi-user environment, users must share data with the central server. This data sharing poses a huge threat to the interests of each user. Federated Learning (FL) can effectively reduce the risk of local data privacy leakage by sharing local model weights instead of sharing local data. It is currently one of the most popular distributed machine learning architectures. However, current research shows that shared models can easily cause federated learning methods to be attacked by inference, resulting in the problem of local data privacy leakage. Summary of the invention

[0003] In view of this, the purpose of the present disclosure is to propose a federated learning method and related equipment based on privacy protection.

[0004] In a first aspect, the present disclosure provides a method for federated learning based on privacy protection, which is applied to a target node, including:

[0005] According to the weight matrix of the neural network, a row sampling sequence set and a column sampling sequence set are obtained;

[0006] Generate a low-dimensional row characterization matrix and a low-dimensional column characterization matrix according to the row sampling sequence set and the column sampling sequence set;

[0007] Receiving a public key sent by the edge node, encrypting the symmetric key according to the public key to obtain a ciphertext, and sending the ciphertext to the edge node;

[0008] Generate a symmetric key, and encrypt the row sampling sequence set and the column sampling sequence set according to the symmetric key to obtain an encrypted row sampling sequence set and a column sampling sequence set;

[0009] An initialization parameter set is generated according to the low-dimensional row characterization matrix, the low-dimensional column characterization matrix, the encrypted row sampling sequence set and the encrypted column sampling sequence set, and the initialization parameter set is sent to the central server.

[0010] In a second aspect, the present disclosure further provides a privacy-preserving federated learning method, which is applied to a central server and includes:

[0011] Receiving an initialization parameter set sent by a target node, and sending the initialization parameter set to an edge node;

[0012] A row edge characterization matrix and a column edge characterization matrix sent by the edge node are received, a global characterization matrix parameter set is obtained according to the row edge characterization matrix and the column edge characterization matrix, and the global characterization matrix parameter set is sent to the edge node.

[0013] In a third aspect, the present disclosure further provides a method for federated learning based on privacy protection, which is applied to an edge node, including:

[0014] Generate an asymmetric key pair; wherein the asymmetric key pair includes: a public key and a private key;

[0015] Sending the public key to the target node;

[0016] Receive the ciphertext sent by the target node, and decrypt the ciphertext using the private key to obtain a symmetric key;

[0017] Receive the initialization parameter set sent by the central server, and decrypt the encrypted row sampling sequence set and the encrypted column sampling sequence set in the initialization parameter set according to the symmetric key to obtain the decrypted row sampling sequence set and the column sampling sequence set;

[0018] Calculate a core matrix according to the decrypted row sampling sequence set and column sampling sequence set, and the low-dimensional row characterization matrix and low-dimensional column characterization matrix in the initialization parameter set;

[0019] According to the core matrix and in combination with the local privacy data set of the edge node, a local row edge representation matrix and a local column edge representation matrix corresponding to the edge node are trained to obtain a row edge representation matrix and a column edge representation matrix corresponding to the edge node;

[0020] The row edge representation matrix and the column edge representation matrix are sent to the central server.

[0021] In a fourth aspect, the present disclosure also provides a federated learning system, which includes: a target node end, a central server end and an edge node end; the target node end is used to execute the privacy protection-based federated learning method as described in the first aspect above; the central server end is used to execute the privacy protection-based federated learning method as described in the second aspect above; the edge node end is used to execute the privacy protection-based federated learning method as described in the third aspect above.

[0022] In a fifth aspect, the present disclosure further provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, the federated learning method as described in any one of the above items is implemented.

[0023] As can be seen from the above, the disclosure provides a privacy-preserving federated learning method. In the implementation process, the target node decomposes the weight matrix of a large-scale neural network into a small-scale low-dimensional row representation matrix and a low-dimensional column feature matrix, and obtains its row sampling sequence set and column sampling sequence set. The row sampling sequence set and the column sampling sequence set can be used as an index set of row sampling sequences and column sampling sequences. The target node generates a symmetric key, encrypts the index set, and obtains an encrypted row sampling sequence set and an encrypted column sampling sequence set. The target node generates an initialization parameter set based on the low-dimensional row representation matrix, the low-dimensional column feature matrix, the encrypted row sampling sequence set, and the encrypted column sampling sequence set. The initialization parameter set replaces the weight matrix of the complete large-scale neural network, and the target node sends the initialization parameter set to the central server, which receives the initialization parameter set. Because there is no corresponding decryption key, the central server cannot decrypt the encrypted row sampling sequence set and the encrypted column sampling sequence set in the initialization parameter set, that is, the index set cannot be obtained. Therefore, the data privacy of the target node can be effectively protected. At the edge node, an asymmetric key is generated, and the public key in the asymmetric key is provided to the target node for it to encrypt the symmetric key to obtain the ciphertext. The target node sends the ciphertext to the edge node. The edge node receives the ciphertext sent by the target node, and decrypts the ciphertext through the private key to obtain the symmetric key. The target node decrypts the encrypted row sampling sequence set and the encrypted column sampling sequence set in the initialization parameter set through the symmetric key to obtain the decrypted row sampling sequence set and the encrypted column sampling sequence set, that is, to obtain the index set. The target node and the edge node communicate through the encrypted symmetric key, which effectively guarantees the privacy of data transmission between the two ends. For the edge node, only the row edge representation matrix and the column edge representation matrix are sent to the central server for the central server to obtain the global representation matrix parameter set. In the privacy-preserving federated learning method provided by the present disclosure, the central server cannot obtain any information about the core matrix of the target node and the edge node from beginning to end. The central server cannot complete the inference attack based on the partial weight information of the neural network obtained. Therefore, the method provided by the present disclosure can protect the privacy leakage of local data very well. At the same time, compared with other federated learning encryption mechanisms, the present disclosure only needs to encrypt the row sampling sequence set and the column sampling sequence set, which greatly reduces the computational complexity in the encryption process. The method provided by the present disclosure can accurately and efficiently solve the problem of data privacy leakage generated by the interaction between the target node and the edge node and the central server during the training process of federated learning, and has very important theoretical value and application potential in future distributed federated learning. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] In order to more clearly illustrate the technical solutions in the present disclosure or the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings in the following description are only embodiments of the present disclosure. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0025] Figure 1 A flowchart of the target node side of the privacy protection-based federated learning method provided in an embodiment of the present disclosure.

[0026] Figure 2 A schematic diagram of the central server-side flow chart of the privacy-preserving federated learning method provided in an embodiment of the present disclosure.

[0027] Figure 3 A schematic diagram of the edge node process of the privacy-preserving federated learning method provided in an embodiment of the present disclosure.

[0028] Figure 4 A schematic diagram of the interaction process among the target node, central server and edge node of the privacy-preserving federated learning method provided in an embodiment of the present disclosure.

[0029] Figure 5 A schematic diagram of application scenarios of the target node side, central server side and edge node side of the privacy protection-based federated learning method provided in an embodiment of the present disclosure.

[0030] Figure 6 It is a schematic diagram of security performance analysis when the row sampling sequence set and the column sampling sequence set are not encrypted.

[0031] Figure 7 A schematic diagram of security performance analysis when encrypting a row sampling sequence set and a column sampling sequence set using a privacy-preserving federated learning method provided in an embodiment of the present disclosure.

[0032] Figure 8 A schematic diagram of a federal system based on privacy protection provided for an embodiment of the present disclosure.

[0033] Fig. 9 It is a schematic diagram of the structure of an electronic device according to an embodiment of the present disclosure. DETAILED DESCRIPTION

[0034] In order to make the objectives, technical solutions and advantages of the present disclosure more clearly understood, the present disclosure is further described in detail below in combination with specific embodiments and with reference to the accompanying drawings.

[0035] It should be noted that, unless otherwise defined, the technical terms or scientific terms used in the embodiments of the present disclosure should be understood by people with ordinary skills in the field to which the present disclosure belongs. The "first", "second" and similar words used in the embodiments of the present disclosure do not indicate any order, quantity or importance, but are only used to distinguish different components. "Including" or "comprising" and similar words mean that the elements or objects appearing before the word cover the elements or objects listed after the word and their equivalents, without excluding other elements or objects. "Connect" or "connected" and similar words are not limited to physical or mechanical connections, but can include electrical connections, whether direct or indirect. "Up", "down", "left", "right" and the like are only used to indicate relative positional relationships. When the absolute position of the described object changes, the relative positional relationship may also change accordingly.

[0036] As described in the background technology section, federated learning, as a distributed computing framework, can train neural network models on multiple distributed edge nodes without transmitting their data to a third-party central server. It can protect the privacy of local user data very well, and therefore has good application prospects in distributed data processing and distributed machine learning. However, recent studies have shown that shared model updates can also cause federated learning methods to be subject to inference attacks, such as attribute inference attacks and data reconstruction attacks. Attribute inference attacks can infer sensitive information attributes of node data during the update process of the federated learning model. Data reconstruction attacks can reconstruct node privacy data based on the update gradient of the edge node.

[0037] In view of this, the present disclosure provides a method for federated learning based on privacy protection. In the implementation process, the target node decomposes the weight matrix of a large-scale neural network into a small-scale low-dimensional row representation matrix and a low-dimensional column feature matrix, and obtains its row sampling sequence set and column sampling sequence set. The row sampling sequence set and the column sampling sequence set can be used as the index set of the row sampling sequence and the column sampling sequence. The target node generates a symmetric key, encrypts the index set, and obtains the encrypted row sampling sequence set and the encrypted column sampling sequence set. The target node generates an initialization parameter set based on the low-dimensional row representation matrix, the low-dimensional column feature matrix, the encrypted row sampling sequence set, and the encrypted column sampling sequence set. The initialization parameter set replaces the weight matrix of the complete large-scale neural network, and the target node sends the initialization parameter set to the central server, which receives the initialization parameter set. Because there is no corresponding decryption key, the central server cannot decrypt the encrypted row sampling sequence set and the encrypted column sampling sequence set in the initialization parameter set, that is, the index set cannot be obtained. Therefore, the data privacy of the target node can be effectively protected. At the edge node, an asymmetric key is generated, and the public key in the asymmetric key is provided to the target node for it to encrypt the symmetric key to obtain the ciphertext. The target node sends the ciphertext to the edge node. The edge node receives the ciphertext sent by the target node, and decrypts the ciphertext through the private key to obtain the symmetric key. The target node decrypts the encrypted row sampling sequence set and the encrypted column sampling sequence set in the initialization parameter set through the symmetric key to obtain the decrypted row sampling sequence set and the encrypted column sampling sequence set, that is, to obtain the index set. The target node and the edge node communicate through the encrypted symmetric key, which effectively guarantees the privacy of data transmission between the two ends. For the edge node, only the row edge representation matrix and the column edge representation matrix are sent to the central server for the central server to obtain the global representation matrix parameter set. In the privacy-preserving federated learning method provided by the present disclosure, the central server cannot obtain any information about the core matrix of the target node and the edge node from beginning to end. The central server cannot complete the inference attack based on the partial weight information of the neural network obtained. Therefore, the method provided by the present disclosure can protect the privacy leakage of local data very well. At the same time, compared with other federated learning encryption mechanisms, the present disclosure only needs to encrypt the row sampling sequence set and the column sampling sequence set, which greatly reduces the computational complexity in the encryption process. The method provided by the present disclosure can accurately and efficiently solve the problem of data privacy leakage generated by the interaction between the target node and the edge node and the central server during the training process of federated learning, and has very important theoretical value and application potential in future distributed federated learning.

[0038] The technical solution of the present disclosure is further described in detail below through specific embodiments.

[0039] refer to Figure 1 , which is a target node-side flow chart of the privacy-preserving federated learning method provided in an embodiment of the present disclosure.

[0040] Step S101, obtaining a row sampling sequence set and a column sampling sequence set according to a weight matrix of a neural network.

[0041] Step S102: generating a low-dimensional row characterization matrix and a low-dimensional column characterization matrix according to the row sampling sequence set and the column sampling sequence set.

[0042] Step S103: receiving the public key sent by the edge node, encrypting the symmetric key according to the public key to obtain a ciphertext, and sending the ciphertext to the edge node.

[0043] Step S104: Generate a symmetric key, and encrypt the row sampling sequence set and the column sampling sequence set according to the symmetric key to obtain an encrypted row sampling sequence set and a column sampling sequence set.

[0044] Step S105, generating an initialization parameter set according to the low-dimensional row characterization matrix, the low-dimensional column characterization matrix, the encrypted row sampling sequence set and the encrypted column sampling sequence set, and sending the initialization parameter set to the central server.

[0045] refer to Figure 2 , which is a schematic diagram of the central server-side process of the privacy-preserving federated learning method provided in an embodiment of the present disclosure.

[0046] Step S201: receiving an initialization parameter set sent by a target node, and sending the initialization parameter set to an edge node.

[0047] Step S202: receiving a row edge characterization matrix and a column edge characterization matrix sent by the edge node, obtaining a global characterization matrix parameter set according to the row edge characterization matrix and the column edge characterization matrix, and sending the global characterization matrix parameter set to the edge node.

[0048] refer to Figure 3 , which is a schematic diagram of the edge node process of the privacy-preserving federated learning method provided in an embodiment of the present disclosure.

[0049] Step S301, generating an asymmetric key pair; wherein the asymmetric key pair includes: a public key and a private key.

[0050] Step S302: sending the public key to the target node.

[0051] Step S303: receiving the ciphertext sent by the target node, and decrypting the ciphertext using the private key to obtain a symmetric key.

[0052] Step S304, receiving the initialization parameter set sent by the central server, and decrypting the encrypted row sampling sequence set and the encrypted column sampling sequence set in the initialization parameter set according to the symmetric key to obtain the decrypted row sampling sequence set and the column sampling sequence set.

[0053] Step S305 , calculating a core matrix according to the decrypted row sampling sequence set and column sampling sequence set, and the low-dimensional row characterization matrix and low-dimensional column characterization matrix in the initialization parameter set.

[0054] Step S306: Based on the core matrix and in combination with the local privacy data set of the edge node, the local row edge representation matrix and the local column edge representation matrix corresponding to the edge node are trained to obtain the row edge representation matrix and the column edge representation matrix corresponding to the edge node.

[0055] Step S307: sending the row edge representation matrix and the column edge representation matrix to the central server.

[0056] refer to Figure 4 , which is a schematic diagram of the interaction process of the target node end, the central server end and the edge node end of the privacy protection-based federated learning method provided in an embodiment of the present disclosure, including the following steps:

[0057] Step 11, the target node end 100 obtains a row sampling sequence set and a column sampling sequence set according to the weight matrix of the neural network.

[0058] In a specific implementation, for each layer of the neural network, a set of row sampling sequence sets and column sampling sequence sets will be obtained.

[0059] In the embodiment of the present disclosure, for the weight matrix of layer l in the neural network;

[0060] like Figure 5 As shown, it is a schematic diagram of the application scenarios of the target node side, the central server side and the edge node side of the privacy protection-based federated learning method provided in an embodiment of the present disclosure.

[0061] Randomly select one of the edge nodes as the target node n a , target node end n a Initialize the weight matrix of the neural network through Xavier Among them, M l , N l Represents the weight matrix of the neural network layer l The input and output dimensions of the weight matrix is a high-dimensional matrix.

[0062] After initialization, the initialization weight matrix is ​​obtained

[0063] Initialize the weight matrix The elements of obey the following uniform distribution:

[0064]

[0065] Among them, i, j are initialization weight matrices The elements in .

[0066] Initialize the weight matrix Low-rank approximation is performed through singular value decomposition (SVD) to perform low-rank approximation and feature dimension reduction to obtain an approximate weight matrix

[0067] By approximating the weight matrix The characteristic matrices U and V of , get the number of row samples s of the approximate weight matrix l，r (s l，r <<M l ) and the number of column samples s l，c (s l，c <<N l );

[0068] According to the number of row samples s of the approximate weight matrix l，r (s l，r <<M l ) and the number of column samples s l，c (s l，c <<N l ) for the approximate weight matrix Perform probability sampling to obtain a set of row sampling sequences and column sampling sequence set The row sampling sequence set obtained at this time and column sampling sequence set It can be used as the index set of row sampling sequence and column sampling sequence;

[0069]

[0070] Step 12: The target node 100 generates a low-dimensional row characterization matrix and a low-dimensional column characterization matrix according to the row sampling sequence set and the column sampling sequence set.

[0071] In the embodiment of the present disclosure, according to the row sampling sequence set and column sampling sequence set Generate row sampling matrix and the column sampling matrix

[0072] According to the row sampling matrix and the column sampling matrix For the approximate weight matrix Perform sampling to generate a low-dimensional row representation matrix and the low-dimensional list feature matrix

[0073]

[0074]

[0075] In a specific implementation, the core matrix can also be generated by sampling the weight matrix of the neural network through a row sampling sequence set and a column sampling sequence set.

[0076] In step 13, the edge node 300 generates an asymmetric key pair, wherein the asymmetric key pair includes a public key and a private key.

[0077] In the specific implementation, an asymmetric encryption algorithm is used, which requires two keys: a public key (publickey: referred to as the public key) and a private key (privatekey: referred to as the private key). The public key and the private key are a pair. If the public key is used to encrypt data, it can only be decrypted with the corresponding private key.

[0078] In the embodiment of the present disclosure, there are several edge nodes 300, and for each edge node 300, an asymmetric key pair is generated. in, is the public key, For the private key.

[0079] Step 14: The edge node 300 sends the public key to the target node 100.

[0080] Step 15: The target node 100 generates a symmetric key, and encrypts the row sampling sequence set and the column sampling sequence set according to the symmetric key to obtain the encrypted row sampling sequence set and the column sampling sequence set.

[0081] In the embodiment of the present disclosure, the target node 100 generates a symmetric key Based on the symmetric key The row sampling sequence set and column sampling sequence set Encrypt to obtain the encrypted row sampling sequence set and column sampling sequence set

[0082]

[0083] Among them, L is the total number of neural network layers, and l is the lth layer of the neural network.

[0084] In step 16, the target node 100 receives the public key sent by the edge node 300, and encrypts the symmetric key according to the public key to obtain a ciphertext.

[0085] In the specific implementation, a symmetric encryption algorithm is used. In the symmetric encryption algorithm, the data sender processes the plaintext (original data) and the encryption key (mi yao) together through a special encryption algorithm, and then turns it into a complex encrypted ciphertext and sends it out. After the recipient receives the ciphertext, if he wants to interpret the original text, he needs to use the encryption key and the inverse algorithm of the same algorithm to decrypt the ciphertext to restore it to readable plaintext. In the symmetric encryption algorithm, there is only one key used, and both the sender and the receiver use this key to encrypt and decrypt the data, which requires the decryption party to know the encryption key in advance.

[0086] The target node 100 uses the symmetric key The public key sent by the edge node 300 Encrypt and get the ciphertext It can be expressed as:

[0087]

[0088] Step 17: The target node 100 sends the ciphertext to the edge node 300.

[0089] Step 18: The target node 100 generates an initialization parameter set according to the low-dimensional row characterization matrix, the low-dimensional column characterization matrix, the encrypted row sampling sequence set, and the encrypted column sampling sequence set.

[0090] In the specific implementation, when generating the initialization parameter set When the edge node 300 sets the training period and training termination condition of the row edge characterization matrix and the column edge characterization matrix at the same time.

[0091]

[0092] in, is the encrypted row sampling sequence set and the encrypted column sampling sequence set, is a set of low-dimensional row characterization matrices and low-dimensional column characterization matrices, L is the total number of neural network layers, and l is the lth layer of the neural network.

[0093] Step 19: The target node 100 sends the initialization parameter set to the central server 200.

[0094] Step 20 : The central server 200 receives the initialization parameter set sent by the target node 100 , and sends the initialization parameter set to the edge node 300 .

[0095] In a specific implementation, the central server 200 receives an initialization parameter set Although the central server 200 can use the initialization parameter set However, due to the lack of the corresponding key, the central server 200 cannot initialize the parameter set The encrypted row sampling sequence set and column sampling sequence set Decryption is performed, that is, the index set of the row sampling sequence and the column sampling sequence cannot be obtained, so the central server 200 cannot infer the data sensitive information attributes and update gradients of the target node and the edge node by reorganizing the core matrix. In this transmission process, the privacy data of the target node and the edge node can be well protected.

[0096] In step 21, the edge node 300 receives the ciphertext sent by the target node 100, and decrypts the ciphertext using the private key to obtain a symmetric key.

[0097] In the embodiment of the present disclosure, the edge node 300 uses a private key Ciphertext Decrypt and get the symmetric key

[0098]

[0099] It can be seen that the target node 100 and the edge node 300 use the encrypted symmetric key Communication is carried out to ensure the security of private information transmission between the two nodes.

[0100] In step 22, the edge node 300 receives the initialization parameter set sent by the central server, and decrypts the encrypted row sampling sequence set and the encrypted column sampling sequence set in the initialization parameter set according to the symmetric key to obtain the decrypted row sampling sequence set and the column sampling sequence set.

[0101] In the specific implementation, because the edge node 300 has a symmetric key Therefore, the encrypted row sampling sequence set and the encrypted column sampling sequence set can be Decryption is performed, but the central server 200 cannot decrypt because it does not have the corresponding key, which clearly reflects the privacy protection feature of the privacy-protection-based federated learning method proposed in the present disclosure.

[0102] Step 23, the edge node 300 calculates a core matrix according to the decrypted row sampling sequence set and column sampling sequence set, and the low-dimensional row characterization matrix and low-dimensional column characterization matrix in the initialization parameter set;

[0103] The core matrix is The calculation process is as follows:

[0104]

[0105] Step 24, the edge node 300 trains the local row edge representation matrix and the local column edge representation matrix corresponding to the edge node according to the core matrix and in combination with the local privacy data set of the edge node, and obtains the row edge representation matrix and the column edge representation matrix corresponding to the edge node;

[0106] In the specific implementation, due to the limited resource bandwidth of federated learning, although all edge nodes can be trained to obtain their corresponding row edge representation matrix and column edge representation matrix, it is difficult for all edge nodes to send the row edge representation matrix and column edge representation matrix to the central server at the same time. In order to solve the above problem, we select all edge nodes with probability pc, which can be expressed as:

[0107]

[0108] in, and They represent the set of all edge nodes and the set of selected edge nodes respectively.

[0109] For the selected edge node, specifically, the core matrix Local privacy dataset And the low-dimensional row representation matrix in the initialization parameter set Low dimensional list feature matrix Multiply and add the bias vector Get the calculation result, pass the calculation result through the activation function, and get the first output data set

[0110]

[0111] Compute the gradient of the row edge representation matrix via rank-restricted backward error propagation and the column margins represent the gradient of the matrix

[0112] For the convenience of calculation, assume that s l，c =s l，r =s l, in the error propagation process, let the error matrix of the l+1 layer of the neural network be The error matrix of the activation function of layer l of the back-propagation neural network is According to the chain derivation rule, the gradient of the row edge representation matrix is ​​obtained and the column margins represent the gradient of the matrix

[0113]

[0114]

[0115] Without loss of generality Because it involves a complex pseudo-inverse derivative formula, Python can directly perform derivatives to calculate the gradient of the edge representation matrix and the column margins represent the gradient of the matrix

[0116] Based on the first output data set Combine the row edges to represent the gradient of the matrix and the column margins represent the gradient of the matrix The local row edge representation matrix corresponding to the edge node end and the local column edge representation matrix Train according to the random gradient descent method to obtain the row edge representation matrix corresponding to the edge node end and the column edge representation matrix

[0117]

[0118]

[0119] Here, η is the update step size of the gradient.

[0120] Step 25 : The edge node 300 sends the row edge representation matrix and the column edge representation matrix to the central server 200 .

[0121] In the embodiment of the present disclosure, after all the selected edge nodes 300 complete the set training cycle, the row edge representation matrix and column edge representation matrix of all the trained edge nodes 300 are Sent to the central server 200, where t l The training cycle is set.

[0122] Step 26: The central server 200 receives the row edge characterization matrix and the column edge characterization matrix sent by the edge node, and obtains a global characterization matrix parameter set according to the row edge characterization matrix and the column edge characterization matrix.

[0123] In the embodiment of the present disclosure, the central server 200 integrates all the received row edge representation matrices and column edge representation matrices trained at the selected edge nodes to form a global row representation matrix and a global column representation matrix:

[0124]

[0125]

[0126] Where D represents the total number of training samples at the edge node end 300, It represents the number of samples in the training data set of the edge node, and t is the current execution cycle of the central server.

[0127]

[0128]

[0129] Step 27: Set the global characterization matrix parameter set Sent to the edge node.

[0130] refer to Figure 6 , which is a schematic diagram of security performance analysis when the row sampling sequence set and the column sampling sequence set are not encrypted, where iter represents the number of training times.

[0131] As can be seen from the picture, when the row sampling sequence set and the column sampling sequence set are not encrypted, the features in the picture fluctuate significantly, indicating that there is information privacy leakage in the federated learning process.

[0132] refer to Figure 7 , which is a schematic diagram of security performance analysis when the privacy-preserving federated learning method provided in an embodiment of the present disclosure encrypts a row sampling sequence set and a column sampling sequence set, wherein iter represents the number of training times.

[0133] It can be seen from the picture that after the row sampling sequence set and the column sampling sequence set are encrypted, the features in the figure are obvious, and they remain stable after multiple trainings, indicating that there is no privacy leakage of information in the privacy-preserving federated learning process provided in the embodiment of the present disclosure.

[0134] Based on the same inventive concept, the present disclosure also provides a federated learning system, which includes: a target node end 801, a central server end 802 and an edge node end 803; the target node end 801 is used to execute the privacy protection-based federated learning method as described in the first aspect above; the central server end 802 is used to execute the privacy protection-based federated learning method as described in the second aspect above; the edge node end 803 is used to execute the privacy protection-based federated learning method as described in the third aspect above.

[0135] refer to Figure 8 , which is a schematic diagram of a federal system based on privacy protection provided in an embodiment of the present disclosure.

[0136] Based on the same inventive concept, an embodiment of the present disclosure further provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, the privacy-preserving federated learning method described in any of the above embodiments is implemented.

[0137] Fig. 9 A more specific schematic diagram of the hardware structure of an electronic device provided by an embodiment of the present disclosure is shown, and the device may include: a processor 1010, a memory 1020, an input / output interface 1030, a communication interface 1040, and a bus 1050. The processor 1010, the memory 1020, the input / output interface 1030, and the communication interface 1040 are connected to each other through the bus 1050 in the device.

[0138] The processor 1010 can be implemented by a general-purpose CPU (Central Processing Unit), a microprocessor, an application-specific integrated circuit (ASIC), or one or more integrated circuits, and is used to execute relevant programs to implement the technical solutions provided in the embodiments of this specification.

[0139] The memory 1020 may be implemented in the form of ROM (Read Only Memory), RAM (Random Access Memory), static storage device, dynamic storage device, etc. The memory 1020 may store an operating system and other application programs. When the technical solutions provided in the embodiments of this specification are implemented by software or firmware, the relevant program codes are stored in the memory 1020 and are called and executed by the processor 1010.

[0140] The input / output interface 1030 is used to connect the input / output module to realize information input and output. The input / output module can be configured in the device as a component (not shown in the figure), or it can be externally connected to the device to provide corresponding functions. The input device may include a keyboard, a mouse, a touch screen, a microphone, various sensors, etc., and the output device may include a display, a speaker, a vibrator, an indicator light, etc.

[0141] The communication interface 1040 is used to connect a communication module (not shown) to realize communication interaction between the device and other devices. The communication module can realize communication through a wired mode (such as USB, network cable, etc.) or a wireless mode (such as mobile network, WIFI, Bluetooth, etc.).

[0142] The bus 1050 includes a path that transmits information between the various components of the device (eg, the processor 1010, the memory 1020, the input / output interface 1030, and the communication interface 1040).

[0143] It should be noted that, although the above device only shows the processor 1010, the memory 1020, the input / output interface 1030, the communication interface 1040 and the bus 1050, in the specific implementation process, the device may also include other components necessary for normal operation. In addition, it can be understood by those skilled in the art that the above device may also only include the components necessary for implementing the embodiments of the present specification, and does not necessarily include all the components shown in the figure.

[0144] Those skilled in the art should understand that the discussion of any of the above embodiments is merely illustrative and is not intended to imply that the scope of the present disclosure (including the claims) is limited to these examples. Based on the concept of the present disclosure, the technical features in the above embodiments or different embodiments may be combined, the steps may be implemented in any order, and there are many other variations of the different aspects of the embodiments of the present disclosure as described above, which are not provided in detail for the sake of simplicity.

[0145] In addition, to simplify the description and discussion, and in order not to make the embodiments of the present disclosure difficult to understand, the known power / ground connections to the integrated circuit (IC) chips and other components may or may not be shown in the provided figures. In addition, the device can be shown in the form of a block diagram to avoid making the embodiments of the present disclosure difficult to understand, and this also takes into account the fact that the details of the implementation of these block diagram devices are highly dependent on the platform on which the embodiments of the present disclosure will be implemented (that is, these details should be fully within the scope of understanding of those skilled in the art). Where specific details (e.g., circuits) are set forth to describe exemplary embodiments of the present disclosure, it is apparent to those skilled in the art that the embodiments of the present disclosure can be implemented without these specific details or with changes in these specific details. Therefore, these descriptions should be considered illustrative rather than restrictive.

[0146] Although the present application has been described in conjunction with the specific embodiments of the present disclosure, many replacements, modifications and variations of these embodiments will be apparent to those skilled in the art from the foregoing description. For example, other memory architectures (e.g., dynamic RAM (DRAM)) may use the embodiments discussed.

[0147] The embodiments of the present disclosure are intended to cover all such substitutions, modifications and variations that fall within the broad scope of the appended claims. Therefore, any omissions, modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the embodiments of the present disclosure should be included in the scope of protection of the present disclosure.

Claims

1. A privacy-preserving federated learning method, applied to a target node, comprising: According to the weight matrix of the neural network, a row sampling sequence set and a column sampling sequence set are obtained; Generate a low-dimensional row characterization matrix and a low-dimensional column characterization matrix according to the row sampling sequence set and the column sampling sequence set; Generate a symmetric key, and encrypt the row sampling sequence set and the column sampling sequence set according to the symmetric key to obtain an encrypted row sampling sequence set and a column sampling sequence set; the encrypted row sampling sequence set and the column sampling sequence set satisfy the following expression: ； in, is the symmetric key, is the row sampling sequence set, is the column sampling sequence set, is the total number of neural network layers, is the neural network layer l; Receiving a public key sent by an edge node, encrypting the symmetric key according to the public key to obtain a ciphertext, and sending the ciphertext to the edge node; wherein the target node is an edge node randomly selected from multiple edge nodes; Generate an initialization parameter set according to the low-dimensional row characterization matrix, the low-dimensional column characterization matrix, the encrypted row sampling sequence set and the encrypted column sampling sequence set, and send the initialization parameter set to the central server; The central server receives the initialization parameter set sent by the target node, and sends the initialization parameter set to the edge node; The edge node receives the initialization parameter set sent by the central server, and decrypts the encrypted row sampling sequence set and the encrypted column sampling sequence set in the initialization parameter set according to the symmetric key to obtain a decrypted row sampling sequence set and a column sampling sequence set; The edge node calculates a core matrix according to the decrypted row sampling sequence set and column sampling sequence set, and the low-dimensional row characterization matrix and the low-dimensional column characterization matrix in the initialization parameter set; According to the core matrix and in combination with the local privacy data set of the edge node, a local row edge representation matrix and a local column edge representation matrix corresponding to the edge node are trained to obtain a row edge representation matrix and a column edge representation matrix corresponding to the edge node; Sending the row edge representation matrix and the column edge representation matrix to the central server; The central server receives the row edge characterization matrix and the column edge characterization matrix sent by the edge node, obtains a global characterization matrix parameter set according to the row edge characterization matrix and the column edge characterization matrix, and sends the global characterization matrix parameter set to the edge node.

2. The method according to claim 1, characterized in that: According to the weight matrix of the neural network, a row sampling sequence set and a column sampling sequence set are obtained, which specifically include: Initialize the weight matrix of the neural network; Decompose the weight matrix of the initialized neural network to obtain an approximate weight matrix; The approximate weight matrix is ​​sampled to obtain the row sampling sequence set and the column sampling sequence set.

3. The method according to claim 2, characterized in that According to the row sampling sequence set and the column sampling sequence set, a low-dimensional row characterization matrix and a low-dimensional column characterization matrix are generated, specifically including: Generate a row sampling matrix and a column sampling matrix according to the row sampling sequence set and the column sampling sequence set; The approximate weight matrix is ​​sampled according to the row sampling matrix and the column sampling matrix to obtain a low-dimensional row characterization matrix and a low-dimensional column characterization matrix.

4. A privacy-preserving federated learning method, applied to a central server, comprising: The central server receives the initialization parameter set sent by the target node, and sends the initialization parameter set to the edge node; wherein the target node obtains a row sampling sequence set and a column sampling sequence set according to the weight matrix of the neural network; the target node generates a low-dimensional row representation matrix and a low-dimensional column characterization matrix according to the row sampling sequence set and the column sampling sequence set; the initialization parameter set includes the low-dimensional row representation matrix, the low-dimensional column characterization matrix, the encrypted row sampling sequence set and the encrypted column sampling sequence set; the encrypted row sampling sequence set and the column sampling sequence set satisfy the following expression: ； in, a symmetric key generated for the target node, is the row sampling sequence set, is the column sampling sequence set, is the total number of neural network layers, is the neural network layer l; The edge node receives the initialization parameter set sent by the central server, and decrypts the encrypted row sampling sequence set and the encrypted column sampling sequence set in the initialization parameter set according to the symmetric key to obtain a decrypted row sampling sequence set and a column sampling sequence set; Calculate a core matrix according to the decrypted row sampling sequence set and column sampling sequence set, and the low-dimensional row characterization matrix and low-dimensional column characterization matrix in the initialization parameter set; According to the core matrix and in combination with the local privacy data set of the edge node, a local row edge representation matrix and a local column edge representation matrix corresponding to the edge node are trained to obtain a row edge representation matrix and a column edge representation matrix corresponding to the edge node; Sending the row edge representation matrix and the column edge representation matrix to the central server; The central server receives the row edge characterization matrix and the column edge characterization matrix sent by the edge node, obtains a global characterization matrix parameter set according to the row edge characterization matrix and the column edge characterization matrix, and sends the global characterization matrix parameter set to the edge node; wherein the target node is an edge node randomly selected from multiple edge node ends.

5. The method according to claim 4, characterized in that Receiving a row edge characterization matrix and a column edge characterization matrix sent by the edge node, and obtaining a global characterization matrix parameter set according to the row edge characterization matrix and the column edge characterization matrix, specifically including: According to the row edge characterization matrix and the column edge characterization matrix, a global row characterization matrix and a global column characterization matrix are obtained; A global characterization matrix parameter set is generated according to the global row characterization matrix and the global column characterization matrix.

6. The method according to claim 4, characterized in that Also includes: An updated row edge characterization matrix and an updated column edge characterization matrix sent by the edge node are received, and the global characterization matrix parameter set is updated according to the updated row edge characterization matrix and the updated column edge characterization matrix.

7. A privacy-preserving federated learning method, applied to an edge node, comprising: Generate an asymmetric key pair; wherein the asymmetric key pair includes: a public key and a private key; Sending the public key to the target node; Receiving the ciphertext sent by the target node, and decrypting the ciphertext using the private key to obtain a symmetric key; The edge node receives the initialization parameter set sent by the central server, and decrypts the encrypted row sampling sequence set and the encrypted column sampling sequence set in the initialization parameter set according to the symmetric key to obtain the decrypted row sampling sequence set and the column sampling sequence set; the encrypted row sampling sequence set and the column sampling sequence set satisfy the following expression: ； in, is the symmetric key, is the row sampling sequence set, is the column sampling sequence set, is the total number of neural network layers, is a neural network layer 1; wherein, the target node end obtains a row sampling sequence set and a column sampling sequence set according to the weight matrix of the neural network; the target node end generates a low-dimensional row characterization matrix and a low-dimensional column characterization matrix according to the row sampling sequence set and the column sampling sequence set; the target node end generates an initialization parameter set according to the low-dimensional row characterization matrix, the low-dimensional column characterization matrix, the encrypted row sampling sequence set and the encrypted column sampling sequence set, and sends the initialization parameter set to the central server end; The edge node calculates a core matrix according to the decrypted row sampling sequence set and column sampling sequence set, and the low-dimensional row characterization matrix and the low-dimensional column characterization matrix in the initialization parameter set; According to the core matrix and in combination with the local privacy data set of the edge node, a local row edge representation matrix and a local column edge representation matrix corresponding to the edge node are trained to obtain a row edge representation matrix and a column edge representation matrix corresponding to the edge node; Sending the row edge representation matrix and the column edge representation matrix to the central server; The central server receives a row edge characterization matrix and a column edge characterization matrix sent by the edge node, obtains a global characterization matrix parameter set according to the row edge characterization matrix and the column edge characterization matrix, and sends the global characterization matrix parameter set to the edge node; wherein the target node is an edge node randomly selected from multiple edge node ends.

8. The method according to claim 7, characterized in that According to the core matrix and in combination with the local privacy data set of the edge node, the local row edge representation matrix and the local column edge representation matrix corresponding to the edge node are trained to obtain the row edge representation matrix and the column edge representation matrix corresponding to the edge node, specifically including: Multiply the core matrix, the local privacy data set, and the low-dimensional row characterization matrix and the low-dimensional column characterization matrix in the initialization parameter set, and add the bias vector to obtain a calculation result, and pass the calculation result through an activation function to obtain a first output data set; Calculate the gradient of the row edge representation matrix and the gradient of the column edge representation matrix; According to the first output data set, combined with the gradient of the row edge representation matrix and the gradient of the column edge representation matrix, the local row edge representation matrix and the local column edge representation matrix corresponding to the edge node end are trained to obtain the row edge representation matrix and the column edge representation matrix corresponding to the edge node end.

9. A privacy-preserving federated learning system, comprising: Target node side, central server side and edge node side; The target node is used to execute the method according to any one of claims 1 to 3; The central server is used to execute the method according to any one of claims 4 to 6; The edge node is used to execute the method according to any one of claims 7 or 8.

10. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the program, the method according to any one of claims 1 to 8 is implemented.

Images