Federated learning methods, systems, and storage media

Abstract

The application provides a federal learning method, a system and a storage medium, and belongs to the technical field of Internet application.The federal learning method provided by the application is applied to federal model construction, can be applied to two-party federal model construction, two-party logistic regression models are constructed through user data and labels, public key exchange is performed between two nodes, and data is twice encrypted using the public keys of the two parties in turn, each node can only decrypt the data encrypted by the public key of the own party using the private key of the own party, safe exchange of two-party data is realized, the two nodes can be provided with optimal parameters meeting preset conditions in a joint modeling mode, and the demand of a business scenario is met.The above scheme achieves the effect that any party cannot individually decrypt the ciphertext data, realizes that data interaction can be completed without a trusted third party and without data export, ensures the safety of the data, and has wide applicability and is suitable for more business scenarios.

Description

Technical Field

[0001] This invention relates to the field of Internet application technology, and in particular to a federated learning method, system, and storage medium. Background Technology

[0002] In recent years, the importance of data security has become self-evident. Major internet platforms, enterprises, and related security regulations have all attached great importance to data security.

[0003] Data security is particularly crucial in businesses involving multiple platforms. To ensure collaborative modeling operations are conducted in accordance with laws and regulations, technologies such as secure multi-party computation and federated learning will become essential in the data circulation field. By encrypting intermediate results during the interactive algorithm process, model building is achieved without data leaving the database. Among these, logistic regression models and decision tree models are the most commonly used and have wide applications across various fields.

[0004] However, the lack of a trusted third party or difficulties in data export during the encryption process still affects the development of this collaborative modeling business. Summary of the Invention

[0005] This invention proposes a federated learning method, system, and storage medium to address the problems existing in the prior art and provide a secure encryption method that does not require the participation of a third party.

[0006] To solve the above-mentioned technical problems, the present invention is implemented as follows:

[0007] In a first aspect, embodiments of the present invention provide a federated learning method, the method being applied to a first node in a joint modeling process, comprising:

[0008] Build a federated learning model based on user feature data and tags;

[0009] Generate a first public key and a first private key for homomorphic encryption, and send the first public key to the second node of the federated learning model;

[0010] Receive the second public key sent by the second node for homomorphic encryption;

[0011] Based on the user feature data, the weighted data of the user feature data of the first node is calculated, and the weighted data of the user feature data of the first node is encrypted twice before being sent to the second node. The second encryption is performed by first encrypting with the first public key and then encrypting with the second public key.

[0012] The system receives a first-encrypted loss function sent by the second node. The first-encrypted loss function is obtained by the second node decrypting the second-encrypted loss function using the second private key. The second-encrypted loss function is obtained by the second node training a model based on the weighted data of the second-encrypted user feature data from the first node and the weighted data of the second node's second-encrypted user feature data. Herein, the first encryption refers to encryption using the first public key.

[0013] Based on the user feature data and labels, a first gradient is calculated, which is the gradient of the first logistic regression model of the federated learning model.

[0014] The loss function is decrypted using the first private key to obtain the loss function; the parameters of the first logistic regression model are optimized and iterated according to the first gradient; when the value of the loss function meets the preset conditions, the iteration stops, and the coefficient vector of the user features of the first node is obtained at this time.

[0015] Optionally, calculating the first gradient includes:

[0016] Receive the residual of the federated learning model after secondary encryption sent by the second node. ;

[0017] According to the above Calculate the first gradient after secondary encryption ;

[0018] calculate And Transmit to the second node;

[0019] Where d is the residual, g A It is the first gradient, R A It is a random number, [[[[R] A ]] A ]] B It is a double-encrypted R A R A Used to protect gradient information;

[0020] This indicates encryption using the first public key. This indicates encryption using a second public key. This means that encryption is performed first using the first public key, and then encryption is performed again using the second public key.

[0021] Optionally, calculating the first gradient further includes:

[0022] Receive the message sent by the second node Data, the By the second node The result is obtained after decryption.

[0023] Optionally, the method further includes:

[0024] Receive the encrypted second gradient related data sent by the second node. , where g B The second gradient is the gradient of the second logistic regression model of the federated learning model. The second gradient after secondary encryption by the second node Decrypt and add a random number R B The result was obtained after calculation;

[0025] The second gradient correlation data after the first encryption Decryption once to obtain ;

[0026] Will Send to the second node.

[0027] Optionally, the step of optimizing and iterating the parameters of the first logistic regression model includes:

[0028] The parameters of the first node are optimized iteratively based on the following formula:

[0029] w A = w A - g A

[0030] Where w A Let be the coefficient vector of the user features of the first node. It is the learning rate parameter.

[0031] Secondly, embodiments of the present invention provide a federated learning method applied to a second node in joint modeling, the method comprising:

[0032] Build a federated learning model based on user feature data and tags;

[0033] Generate a second public key and a second private key for homomorphic encryption, and send the second public key to the first node;

[0034] Receive the weighted data of the first public key and the second-encrypted first node user feature data;

[0035] Calculate the weighted data of the user feature data of the second node, and then encrypt the weighted data of the user feature data of the second node twice; wherein, the second encryption is to first encrypt it with the first public key and then encrypt it with the second public key.

[0036] The weighted data of the first node user feature data after secondary encryption and the weighted data of the second node user feature data after secondary encryption are used to calculate the loss function and the second gradient after secondary encryption. The second gradient is the gradient of the second logistic regression model of the federated learning model.

[0037] The loss function, which has undergone secondary encryption, is decrypted using the second private key and then sent to the first node.

[0038] Calculate the second gradient based on the user feature data and tags;

[0039] Based on the second gradient, the parameters of the second logistic regression model are optimized iteratively. When the value of the loss function meets the preset conditions, the iteration stops, and the coefficient vector of the user features of the second node is obtained at this time.

[0040] Optionally, calculating the loss function and the second gradient after secondary encryption includes:

[0041] Based on the weighted data of the first node user feature data after secondary encryption, and the weighted data of the second node user feature data after secondary encryption, calculate:

[0042] The weighted data u of the merged user feature data is then encrypted again. yw T x Data after secondary encryption The data after secondary encryption of residual d and (w T x) 2 Data after secondary encryption ;

[0043] Where x is user feature data, y is label, and w is the coefficient vector of user features;

[0044] Through the above , , and Calculate the loss function after double encryption, and calculate the second gradient data after double encryption. .

[0045] Optionally, the calculation of the second gradient includes:

[0046] The second gradient data after secondary encryption Perform a decryption to obtain the encrypted second-gradient data. ;

[0047] calculate And transmit it to the first node;

[0048] Among them, R B These are random numbers, used to preserve gradient information. This indicates encryption using the first public key. This indicates encryption using a second public key. This means that encryption is performed first using the first public key, and then encryption is performed again using the second public key.

[0049] Optionally, the step of optimizing and iterating the parameters of the second logistic regression model includes:

[0050] The parameters of the second node are optimized iteratively based on the following formula:

[0051] w B = w B - g B

[0052] Where w B Let be the coefficient vector of the user features of the second node. It is the learning rate parameter.

[0053] Thirdly, embodiments of the present invention provide a federated learning modeling system for joint modeling, comprising:

[0054] The building module is used to construct federated learning models based on user feature data and labels;

[0055] The processing module is used to generate a first public key and a first private key for homomorphic encryption, and send the first public key to the second node of the federated learning model;

[0056] The receiving module is used to receive the second public key for homomorphic encryption sent by the second node;

[0057] The processing module is further configured to calculate the weighted data of the user feature data of the first node based on the user feature data, and to send the weighted data of the user feature data of the first node to the second node after performing secondary encryption, wherein the secondary encryption is to first encrypt using the first public key and then encrypt using the second public key.

[0058] The receiving module is further configured to receive a first-encrypted loss function sent by the second node. The first-encrypted loss function is obtained by the second node decrypting the second-encrypted loss function using the second private key. The second-encrypted loss function is obtained by the second node training a model based on the second-encrypted weighted user feature data from the first node and the second-encrypted weighted user feature data from the second node. Herein, the first encryption refers to encryption using the first public key.

[0059] The processing module is further configured to calculate a first gradient based on the user feature data and labels, wherein the first gradient is the gradient of the first logistic regression model of the federated learning model;

[0060] The encryption calculation module is also used to decrypt the encrypted loss function using the first private key to obtain the loss function;

[0061] The optimization module is used to optimize and iterate the parameters of the first logistic regression model based on the first gradient. When the value of the loss function meets the preset conditions, the iteration stops and the coefficient vector of the user features of the first node is obtained at this time.

[0062] Fourthly, embodiments of the present invention provide a federated learning modeling system for joint modeling, comprising:

[0063] The building module is used to construct a second-node logistic regression model based on user feature data and tags;

[0064] The processing module is used to generate a second public key and a second private key for homomorphic encryption, and to send the second public key to the first node;

[0065] The receiving module is used to receive the weighted data of the first public key and the second-encrypted first node user feature data;

[0066] The processing module is further configured to calculate the weighted data of the second node user feature data and perform secondary encryption on the weighted data of the second node user feature data; wherein, the secondary encryption is to first encrypt using the first public key and then encrypt using the second public key;

[0067] The processing module is further configured to calculate the loss function and the second gradient after secondary encryption based on the weighted data of the first node user feature data after secondary encryption and the weighted data of the second node user feature data after secondary encryption. The second gradient is the gradient of the second logistic regression model of the federated learning model.

[0068] The processing module is also used to decrypt the double-encrypted loss function using the second private key and then send it to the first node;

[0069] The processing module is further configured to calculate a second gradient based on the user feature data and tags;

[0070] The optimization module is used to optimize and iterate the parameters of the second logistic regression model according to the second gradient. When the value of the loss function meets the preset conditions, the iteration stops and the coefficient vector of the user features of the second node is obtained at this time.

[0071] Fifthly, embodiments of the present invention provide a network device, including: a processor, a memory, and a program stored in the memory and executable on the processor, wherein the program, when executed by the processor, implements the steps of the federated learning method provided in the first or second aspect.

[0072] In a sixth aspect, embodiments of the present invention provide a computer-readable storage medium storing a computer program, which, when executed by a processor, implements the steps of the federated learning method provided in the first or second aspect.

[0073] The beneficial effects of the technical solutions provided in the embodiments of the present invention include at least the following:

[0074] The federated learning method provided by this invention is applied to federated model construction, specifically to two-party federated models. It constructs a two-party logistic regression model using user data and labels. The two nodes exchange public keys and encrypt the data twice using both parties' public keys. Each node can only decrypt data encrypted with its own private key, achieving secure data exchange between the two parties. In a joint modeling mode, it provides optimal parameters to both nodes that meet preset conditions, satisfying the needs of various business scenarios. This solution achieves the effect that neither party can decrypt the encrypted data independently, enabling data interaction without a trusted third party or data outbound, ensuring data security, and possessing broad applicability suitable for a wider range of business scenarios. Attached Figure Description

[0075] Various other advantages and benefits will become apparent to those skilled in the art upon reading the following detailed description of preferred embodiments. The accompanying drawings are for illustrative purposes only and are not intended to limit the invention. Furthermore, the same reference numerals denote the same parts throughout the drawings. In the drawings:

[0076] Figure 1 A flowchart of the first node of a federated learning method provided in an embodiment of the present invention;

[0077] Figure 2 A flowchart of the second node of a federated learning method provided in an embodiment of the present invention;

[0078] Figure 3 This is a schematic diagram of node information interaction in a federated learning method provided by an embodiment of the present invention;

[0079] Figure 4 This is a schematic diagram of the first node structure of a federated learning system provided in an embodiment of the present invention;

[0080] Figure 5This is a schematic diagram of the second node structure of a federated learning system provided in an embodiment of the present invention;

[0081] Figure 6 This is a schematic diagram of the structure of a network device provided in an embodiment of the present invention. Detailed Implementation

[0082] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0083] The federated learning method provided in this invention embodiment realizes logistic regression modeling in a two-party scenario. The first node in the solution of this invention belongs to the first party, which is usually the data provider. This party has abundant data but lacks application scenarios and hopes to provide data services to the outside world to make the data generate value.

[0084] The second node belongs to the second party, which is usually the data requester. This party has a clear business scenario and needs data to assist them in carrying out their business. They build models based on the data from the first party and apply them to the actual business scenario.

[0085] Please refer to Figure 1 , Figure 1 This diagram illustrates the first node flowchart of a federated learning method provided in an embodiment of the present invention. Figure 1 The process described in this application refers to the operations performed by the first party, i.e., the first node, in the federated learning method provided by this application. The method includes:

[0086] Step 11: Construct a federated learning model based on user feature data and labels.

[0087] Specifically, after obtaining the user's feature data and tags, the first party constructs a first logistic regression model, and the second party constructs a second logistic regression model.

[0088] Specifically, before building the federated learning model, both parties need to collect, clean, and perform feature engineering on user data.

[0089] Specifically, data collection: The first party and the second party respectively collect multi-dimensional features of the user, such as behavior and communication. The second party also needs to collect user tags. The collected data is stored in the respective databases of the first party and the second party for further use.

[0090] Specifically, data cleaning: Since there may be missing or incorrect information during the collection process, noise removal is required in this step, missing data is filled or deleted, and the supplemented data is imported into the respective databases of both parties.

[0091] Specifically, feature engineering involves processing the massive amounts of collected data, further deriving features based on the original data, expanding feature dimensions, obtaining potentially useful user information, and generating user feature data.

[0092] Step 12: Generate a first public key and a first private key for homomorphic encryption, and send the first public key to the second node of the federated learning model;

[0093] Step 13: Receive the second public key for homomorphic encryption sent by the second node;

[0094] Specifically, steps 12 and 13 enable the exchange of public keys between the first and second parties. The first party generates a first public key and a first private key, while the second party generates a second public key and a second private key. Both parties send their respective public keys to each other. Through this step, in subsequent encryption, both the first and second parties can encrypt data using their respective first and second public keys. However, since the first and second parties only hold their own private keys, each party can only decrypt data encrypted with its own public key and cannot decrypt data encrypted with the other party's public key. This greatly ensures the security of the data.

[0095] Step 14: Based on the user feature data, calculate the weighted data of the user feature data of the first node, and then encrypt the weighted data of the user feature data of the first node twice before sending it to the second node. The second encryption is to first encrypt it with the first public key and then encrypt it with the second public key.

[0096] For example, the first party calculates the weighted data u of the user feature data of the first node. A and u A 2 ,in, , where x A w is the feature vector of first-party user data. A For x A The vector of coefficients, i.e., the vector of coefficients of the first-party user features, w A With x A One-to-one correspondence, here we take x A = (x1, x2, x3).

[0097] Specifically, in this specific implementation method, we will uniformly use... This indicates that encryption is performed using the first public key. This indicates that encryption is performed using the second public key. This means that encryption is performed first using the first public key, and then encryption is performed again using the second public key.

[0098] For example, according to the above representation rules, the first node here will Send to the second node.

[0099] Step 15: Receive the first-encrypted loss function sent by the second node. The first-encrypted loss function is obtained by the second node decrypting the second-encrypted loss function using the second private key. The second-encrypted loss function is obtained by the second node training a model based on the second-encrypted weighted user feature data from the first node and the second-encrypted weighted user feature data from the second node. Herein, the first encryption refers to encryption using the first public key.

[0100] Specifically, the loss function is determined by the second node based on... as well as Calculations are performed to obtain the second-order encrypted data with the loss function. The second node uses the second public key to... After decryption, the encrypted loss function data is obtained as follows: ,Will Transmit to the first node;

[0101] in, This is weighted data of the second node's user feature data obtained through double encryption. Here, user data features are represented by x B = (x4, x5) The two-stage encryption mentioned in this article refers to encryption first using the first public key and then using the second public key.

[0102] The first node receives the... .

[0103] Step 16: Calculate the first gradient based on the user feature data and labels. The first gradient is the gradient of the first logistic regression model of the federated learning model.

[0104] For example, user feature data is x, label is y, and the first gradient is g. A .

[0105] Step 17: Decrypt the encrypted loss function using the first private key to obtain the loss function.

[0106] Specifically, the first node receives the message sent by the second node. Then, the loss function data is encrypted once using the first private key. Decrypt the data to obtain the loss function.

[0107] Step 18: Optimize and iterate the parameters of the first logistic regression model based on the first gradient. When the value of the loss function meets the preset conditions, stop the iteration and obtain the coefficient vector of the user features of the first node at this time.

[0108] Specifically, it is determined whether the loss function loss is less than the preset threshold ops or whether the preset maximum number of iterations max_iter has been reached. If the preset conditions are met, the iteration stops; otherwise, the iteration continues.

[0109] When iteration stops, the parameters w of the first logistic regression model are... A These are the parameters required to conduct business.

[0110] Optionally, calculating the first gradient includes:

[0111] Receive the residual of the federated learning model after secondary encryption sent by the second node. ;

[0112] According to the above Calculate the first gradient after secondary encryption ;

[0113] calculate And Transmit to the second node;

[0114] Where d is the residual, g A It is the first gradient, R A It is a random number, [[[[R] A ]] A ]] B It is a double-encrypted R A R A Used to protect gradient information;

[0115] This indicates encryption using the first public key. This indicates encryption using a second public key. This means that encryption is performed first using the first public key, and then encryption is performed again using the second public key.

[0116] Specifically, These are parameters calculated by the second node, and the specific calculation process is described in the section on the second node below.

[0117] The first node is calculated according to the following formula. :

[0118] .

[0119] Where, x A This refers to user characteristic data from the first party (first node).

[0120] Optionally, calculating the first gradient further includes:

[0121] Receive the message sent by the second node Data, the By the second node The result is obtained after decryption.

[0122] Specifically, in this article, the decryption of the second node refers to the second node decrypting the data encrypted with the second public key using the second private key, that is, the second node decrypting the data after double encryption.

[0123] The first node's decryption refers to the first node decrypting the data encrypted with the first public key using the first private key; that is, the first node decrypting the data after it has been encrypted once.

[0124] The first node does not have the second private key, so it cannot decrypt the data after the second encryption. The second node does not have the first private key, so it cannot decrypt the data after the first encryption. This greatly ensures the security of the data.

[0125] Optionally, the method further includes:

[0126] Receive the encrypted second gradient related data sent by the second node. , where g B The second gradient is the gradient of the second logistic regression model of the federated learning model. The second gradient after secondary encryption by the second node Decrypt and add a random number R B The result was obtained after calculation;

[0127] The second gradient correlation data after the first encryption Decryption once to obtain ;

[0128] Will Send to the second node.

[0129] Specifically, the parameters of the first node related to the second gradient One encrypted data Decrypt using the first private key, and obtain the decrypted result. Send to the second node, due to the random number R B The existence of this information means it is transmitted to the second node in plaintext. It will also not expose the value of the second gradient.

[0130] Optionally, the step of optimizing and iterating the parameters of the first logistic regression model includes:

[0131] The parameters of the first node are optimized iteratively based on the following formula:

[0132] w A = w A - g A

[0133] Where w A Let be the coefficient vector of the user features of the first node. It is the learning rate parameter.

[0134] Specifically, based on the intermediate data of the optimization process, the value of the loss function is determined. When the loss function meets the preset conditions, the above iteration is stopped, and the parameters at this time are the optimized parameters of the first logistic regression model.

[0135] Secondly, refer to Figure 2 This invention provides a federated learning method applied to the second node of joint modeling, the method comprising:

[0136] Step 21: Construct a federated learning model based on user feature data and labels.

[0137] Specifically, the second party constructs a second logistic regression model at the second node.

[0138] Step 22: Generate a second public key and a second private key for homomorphic encryption, and send the second public key to the first node;

[0139] Step 23: Receive the first public key and the weighted data of the first node user feature data after secondary encryption.

[0140] Specifically, the second node receives the weighted data of the first node's user feature data after it has been encrypted twice. .

[0141] Step 24: Calculate the weighted data of the second node user feature data, and perform secondary encryption on the weighted data of the second node user feature data; wherein, the secondary encryption is to first encrypt it with the first public key and then encrypt it with the second public key.

[0142] Specifically, the weighted data of the second node user feature data is as follows: and calculation u B 2 , This is weighted data of the second node's user feature data obtained through secondary encryption. It is worth noting that this data is not sent to the first node.

[0143] Step 25: Calculate the loss function and the second gradient after secondary encryption based on the weighted data of the first node user feature data after secondary encryption and the weighted data of the second node user feature data after secondary encryption. The second gradient is the gradient of the second logistic regression model of the federated learning model.

[0144] Specifically, according to ,as well as Calculate the loss function data after secondary encryption. And calculate the second gradient data after secondary encryption. .

[0145] Specifically, in order to calculate as well as The second node is based on and First calculate , , , Intermediate values;

[0146] Where x is the user feature data, y is the label, w is the vector of coefficients of the user feature data, and d is the residual of the federated learning model;

[0147] The weighted data is the fusion of user feature data from the first logistic regression model and the second logistic regression model.

[0148] Step 26: Decrypt the double-encrypted loss function using the second private key and send it to the first node.

[0149] Specifically, the second node will calculate the... Decrypted using the second private key ,Will Send to the first node.

[0150] Step 27: Calculate the second gradient based on the user feature data and tags.

[0151] Step 28: Optimize and iterate the parameters of the second logistic regression model according to the second gradient. When the value of the loss function meets the preset conditions, stop the iteration and obtain the coefficient vector of the user features of the second node at this time.

[0152] Specifically, it is determined whether the loss function loss is less than the preset threshold ops or whether the preset maximum number of iterations max_iter has been reached. If the preset conditions are met, the iteration stops; otherwise, the iteration continues.

[0153] When iteration stops, the parameters w of the first logistic regression model are... B These are the parameters required to conduct business.

[0154] Optionally, calculating the loss function and the second gradient after secondary encryption includes:

[0155] Based on the weighted data of the first node user feature data after secondary encryption, and the weighted data of the second node user feature data after secondary encryption, calculate:

[0156] The weighted data u of the merged user feature data is then encrypted again. yw T x Data after secondary encryption The data after secondary encryption of residual d and (w T x) 2 Data after secondary encryption ;

[0157] Where x is user feature data, y is label, and w is the coefficient vector of user features;

[0158] Through the above , , and Calculate the loss function after double encryption, and calculate the second gradient data after double encryption. .

[0159] Specifically, the calculation process is as follows:

[0160] calculate According to the formula It can be seen that the following formula can be used for calculation:

[0161] .

[0162] calculate :

[0163] .

[0164] calculate :

[0165]

[0166] calculate According to the formula We can obtain:

[0167] .

[0168] calculate ,make ,So:

[0169] ,

[0170] According to the gradient calculation formula... We can obtain:

[0171] .

[0172] Optionally, the calculation of the second gradient includes:

[0173] The second gradient data after secondary encryption Perform a decryption to obtain the encrypted second-gradient data. ;

[0174] calculate And Transmit to the first node;

[0175] Among them, R B These are random numbers, used to protect gradient information. [[ ]] A This is encrypted data obtained by encrypting with the first public key.

[0176] Specifically, through the second private key pair Decrypt, obtain Here, only the encrypted data of the second gradient-related parameters, after adding random numbers, is sent to the first node. It can effectively ensure data security.

[0177] Optionally, the optimization and iteration of the parameters of the second logistic regression model includes: optimizing and iterating the parameters of the second node based on the following formula:

[0178] w B = w B - g B

[0179] Where w B Let be the coefficient vector of the user features of the second node. It is the learning rate parameter.

[0180] Specifically, based on the intermediate data of the optimization process, the value of the loss function is determined. When the loss function meets the preset conditions, the above iteration is stopped, and the parameters at this time are the optimized parameters of the second logistic regression model.

[0181] For details, please refer to Figure 3 This illustrates the information transfer and interaction between the first and second nodes during the construction of the federated model.

[0182] In summary, the federated learning method provided in this embodiment of the invention, applied to federated model construction, can be used for two-party federated model construction. It constructs a two-party logistic regression model using user data and labels, with public keys exchanged between the two nodes. The data is then encrypted twice using both nodes' public keys. Each node can only decrypt data encrypted with its own public key using its own private key, achieving secure data exchange between the two parties. In the joint modeling mode, it can provide optimal parameters to both nodes that meet preset conditions, satisfying the needs of business scenarios. The above solution achieves the effect that neither party can decrypt the encrypted data independently, enabling data interaction without a trusted third party or data outbound, ensuring data security, and has broad applicability suitable for more business scenarios.

[0183] Please refer to Figure 4 This invention provides a federated learning modeling system 40 for joint modeling, comprising:

[0184] Module 41 is used to build a federated learning model based on user feature data and labels;

[0185] Processing module 42 is used to generate a first public key and a first private key for homomorphic encryption, and send the first public key to the second node of the federated learning model;

[0186] Receiver module 43 is used to receive the second public key for homomorphic encryption sent by the second node;

[0187] The processing module 42 is further configured to calculate the weighted data of the user feature data of the first node based on the user feature data, and to send the weighted data of the user feature data of the first node to the second node after performing secondary encryption, wherein the secondary encryption is to first encrypt using the first public key and then encrypt using the second public key.

[0188] The receiving module 43 is further configured to receive a first-encrypted loss function sent by the second node. The first-encrypted loss function is obtained by the second node decrypting the second-encrypted loss function using the second private key. The second-encrypted loss function is obtained by the second node training a model based on the second-encrypted weighted user feature data from the first node and the second-encrypted weighted user feature data from the second node. Herein, the first encryption refers to encryption using the first public key.

[0189] The processing module 42 is further configured to calculate a first gradient based on the user feature data and labels, wherein the first gradient is the gradient of the first logistic regression model of the federated learning model;

[0190] The processing module 42 is further configured to decrypt the encrypted loss function using the first private key to obtain the loss function;

[0191] The optimization module 44 is used to optimize and iterate the parameters of the first logistic regression model according to the first gradient. When the value of the loss function meets the preset conditions, the iteration stops and the coefficient vector of the user features of the first node is obtained at this time.

[0192] Optionally, the processing module 42 is further configured to:

[0193] Receive the residual of the federated learning model after secondary encryption sent by the second node. ;

[0194] According to the above Calculate the first gradient after secondary encryption ;

[0195] calculate And Transmit to the second node;

[0196] Where d is the residual, g A It is the first gradient, R A It is a random number, [[[[R] A ]] A ]] B It is a double-encrypted R A R A Used to protect gradient information;

[0197] This indicates encryption using the first public key. This indicates encryption using a second public key. This means that encryption is performed first using the first public key, and then encryption is performed again using the second public key.

[0198] Optionally, the processing module 42 is further configured to:

[0199] Receive the message sent by the second node Data, the By the second node The result is obtained after decryption.

[0200] Optionally, the receiving module 43 is further configured to:

[0201] Receive the encrypted second gradient related data sent by the second node. , where g B The second gradient is the gradient of the second logistic regression model of the federated learning model. The second gradient after secondary encryption by the second node Decrypt and add a random number R B The result was obtained after calculation;

[0202] The processing module 42 is also used to process the second gradient correlation data after the first encryption. Decryption once to obtain ;

[0203] Will Send to the second node.

[0204] Optionally, the processing module 42 is further configured to:

[0205] The parameters of the first node are optimized iteratively based on the following formula:

[0206] w A = w A - g A

[0207] Where w A Let be the coefficient vector of the user features of the first node. It is the learning rate parameter.

[0208] Fourthly, embodiments of the present invention provide a federated learning modeling system 50, applied to joint modeling, comprising:

[0209] Module 51 is used to build a second-node logistic regression model based on user feature data and tags;

[0210] Processing module 52 is used to generate a second public key and a second private key for homomorphic encryption, and send the second public key to the first node;

[0211] The receiving module 53 is used to receive the weighted data of the first public key and the second-encrypted first node user feature data;

[0212] The processing module 52 is further configured to calculate the weighted data of the second node user feature data and perform secondary encryption on the weighted data of the second node user feature data; wherein, the secondary encryption is to first encrypt using the first public key and then encrypt using the second public key;

[0213] The processing module 52 is further configured to calculate the loss function and the second gradient after secondary encryption based on the weighted data of the first node user feature data after secondary encryption and the weighted data of the second node user feature data after secondary encryption, wherein the second gradient is the gradient of the second logistic regression model of the federated learning model.

[0214] The processing module 52 is further configured to decrypt the double-encrypted loss function using the second private key and then send it to the first node;

[0215] The processing module 52 is further configured to calculate a second gradient based on the user feature data and tags;

[0216] The optimization module 54 is used to optimize and iterate the parameters of the second logistic regression model according to the second gradient. When the value of the loss function meets the preset conditions, the iteration stops and the coefficient vector of the user features of the second node is obtained at this time.

[0217] Optionally, the processing module 52 is further configured to:

[0218] Based on the weighted data of the first node user feature data after secondary encryption, and the weighted data of the second node user feature data after secondary encryption, calculate:

[0219] The weighted data u of the merged user feature data is then encrypted again. yw T x Data after secondary encryption The data after secondary encryption of residual d and (w T x) 2 Data after secondary encryption ;

[0220] Where x is user feature data, y is label, and w is the coefficient vector of user features;

[0221] Through the above , , and Calculate the loss function after double encryption, and calculate the second gradient data after double encryption. .

[0222] Optionally, the processing module 52 is further configured to:

[0223] The second gradient data after secondary encryption Perform a decryption to obtain the encrypted second-gradient data. ;

[0224] calculate And transmit it to the first node;

[0225] Among them, R B These are random numbers, used to preserve gradient information. This indicates encryption using the first public key. This indicates encryption using a second public key. This means that encryption is performed first using the first public key, and then encryption is performed again using the second public key.

[0226] Optionally, the processing module 52 is further configured to:

[0227] The parameters of the second node are optimized iteratively based on the following formula:

[0228] w B = w B - g B

[0229] Where w B Let be the coefficient vector of the user features of the second node. It is the learning rate parameter.

[0230] Please refer to Figure 6 This invention also provides a network device 60, including a processor 61, a memory 62, and a computer program stored in the memory 62 and executable on the processor 61. When the computer program is executed by the processor 61, it implements the various processes of the above-described federated learning method embodiments and achieves the same technical effects. To avoid repetition, it will not be described again here.

[0231] This invention also provides a computer-readable storage medium storing a computer program. When executed by a processor, the computer program implements the various processes of the above-described federated learning method embodiments and achieves the same technical effects. To avoid repetition, it will not be described again here. The computer-readable storage medium may be a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk, etc.

[0232] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element.

[0233] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods of the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) and includes several instructions to cause a terminal (which may be a mobile phone, computer, server, air conditioner, or network device, etc.) to execute the methods described in the various embodiments of the present invention.

[0234] The embodiments of the present invention have been described above with reference to the accompanying drawings. However, the present invention is not limited to the specific embodiments described above. The specific embodiments described above are merely illustrative and not restrictive. Those skilled in the art can make many other forms under the guidance of the present invention without departing from the spirit and scope of the claims, and all of these forms are within the protection scope of the present invention.

Claims

1. A federated learning method, characterized in that, The method, applied to the first node of the joint modeling, includes: Build a federated learning model based on user feature data and tags; Generate a first public key and a first private key for homomorphic encryption, and send the first public key to the second node of the federated learning model; Receive the second public key sent by the second node for homomorphic encryption; Based on the user feature data, the weighted data of the user feature data of the first node is calculated, and the weighted data of the user feature data of the first node is encrypted twice before being sent to the second node. The second encryption is performed by first encrypting with the first public key and then encrypting with the second public key. The system receives a first-encrypted loss function sent by the second node. The first-encrypted loss function is obtained by the second node decrypting the second-encrypted loss function using the second private key. The second-encrypted loss function is obtained by the second node training a model based on the weighted data of the second-encrypted user feature data from the first node and the weighted data of the second node's second-encrypted user feature data. Herein, the first encryption refers to encryption using the first public key. Based on the user feature data and labels, a first gradient is calculated, which is the gradient of the first logistic regression model of the federated learning model. The loss function is decrypted using the first private key to obtain the loss function; the parameters of the first logistic regression model are optimized and iterated according to the first gradient; when the value of the loss function meets the preset conditions, the iteration stops, and the coefficient vector of the user features of the first node is obtained at this time.

2. The federated learning method according to claim 1, characterized in that, The calculation of the first gradient includes: Receive the residual of the federated learning model after secondary encryption sent by the second node. ; According to the above Calculate the first gradient after secondary encryption ; calculate And Transmit to the second node; Where d is the residual, g A It is the first gradient, R A It is a random number, [[[[R] A ]] A ]] B It is a double-encrypted R A R A Used to protect gradient information; This indicates encryption using the first public key. This indicates encryption using a second public key. This means that encryption is performed first using the first public key, and then encryption is performed again using the second public key.

3. The federated learning method according to claim 2, characterized in that, The calculation of the first gradient further includes: Receive the message sent by the second node Data, the By the second node The result is obtained after decryption.

4. The federated learning method according to claim 3, characterized in that, The method further includes: Receive the encrypted second gradient related data sent by the second node. , where g B The second gradient is the gradient of the second logistic regression model of the federated learning model. The second gradient after secondary encryption by the second node Decrypt and add a random number R B The result was obtained after calculation; The second gradient correlation data after the first encryption Decryption once to obtain ; Will Send to the second node.

5. The federated learning method according to claim 1, characterized in that, The optimization and iteration of the parameters of the first logistic regression model includes: The parameters of the first node are optimized iteratively based on the following formula: w A = w A - g A Where w A Let be the coefficient vector of the user features of the first node. It is the learning rate parameter.

6. A federated learning method, characterized in that, The method, applied to the second node of the joint modeling, includes: Based on user feature data and tags, construct a second-node logistic regression model; Generate a second public key and a second private key for homomorphic encryption, and send the second public key to the first node; Receive the weighted data of the first public key and the second-encrypted first node user feature data; Calculate the weighted data of the user feature data of the second node, and then encrypt the weighted data of the user feature data of the second node twice; wherein, the second encryption is to first encrypt it with the first public key and then encrypt it with the second public key. The weighted data of the first node user feature data after secondary encryption and the weighted data of the second node user feature data after secondary encryption are used to calculate the loss function and the second gradient after secondary encryption. The second gradient is the gradient of the second logistic regression model of the federated learning model. The loss function, which has undergone secondary encryption, is decrypted using the second private key and then sent to the first node. Calculate the second gradient based on the user feature data and tags; Based on the second gradient, the parameters of the second logistic regression model are optimized iteratively. When the value of the loss function meets the preset conditions, the iteration stops, and the coefficient vector of the user features of the second node is obtained at this time.

7. The federated learning method according to claim 6, characterized in that, The calculation of the loss function and the second gradient after secondary encryption includes: Based on the weighted data of the first node user feature data after secondary encryption, and the weighted data of the second node user feature data after secondary encryption, calculate: The weighted data u of the merged user feature data is then encrypted again. yw T x Data after secondary encryption The data after secondary encryption of residual d and (w T x) 2 Data after secondary encryption ; Where x is user feature data, y is label, and w is the coefficient vector of user features; Through the above , , and Calculate the loss function after double encryption, and calculate the second gradient data after double encryption. .

8. The federated learning method according to claim 7, characterized in that, The calculation of the second gradient includes: The second gradient data after secondary encryption Perform a decryption to obtain the second gradient after encryption. ; calculate And Transmit to the first node; Among them, R B These are random numbers used to preserve gradient information. This indicates encryption using the first public key. This indicates encryption using a second public key. This means that encryption is performed first using the first public key, and then encryption is performed again using the second public key.

9. The federated learning method according to claim 6, characterized in that, The optimization and iteration of the parameters of the second logistic regression model includes: The parameters of the second node are optimized iteratively based on the following formula: w B = w B - g B Where w B Let be the coefficient vector of the user features of the second node. It is the learning rate parameter.

10. A federated learning modeling system, characterized in that, Applied to joint modeling, including: The building module is used to construct federated learning models based on user feature data and labels; The processing module is used to generate a first public key and a first private key for homomorphic encryption, and send the first public key to the second node of the federated learning model; The receiving module is used to receive the second public key for homomorphic encryption sent by the second node; The processing module is further configured to calculate the weighted data of the user feature data of the first node based on the user feature data, and to send the weighted data of the user feature data of the first node to the second node after performing secondary encryption, wherein the secondary encryption is to first encrypt using the first public key and then encrypt using the second public key. The receiving module is further configured to receive a first-encrypted loss function sent by the second node. The first-encrypted loss function is obtained by the second node decrypting the second-encrypted loss function using the second private key. The second-encrypted loss function is obtained by the second node training a model based on the second-encrypted weighted user feature data from the first node and the second-encrypted weighted user feature data from the second node. Herein, the first encryption refers to encryption using the first public key. The processing module is further configured to calculate a first gradient based on the user feature data and labels, wherein the first gradient is the gradient of the first logistic regression model of the federated learning model; The encryption calculation module is also used to decrypt the encrypted loss function using the first private key to obtain the loss function; The optimization module is used to optimize and iterate the parameters of the first logistic regression model based on the first gradient. When the value of the loss function meets the preset conditions, the iteration stops and the coefficient vector of the user features of the first node is obtained at this time.

11. A federated learning modeling system, characterized in that, Applied to joint modeling, including: The building module is used to construct a second-node logistic regression model based on user feature data and tags; The processing module is used to generate a second public key and a second private key for homomorphic encryption, and to send the second public key to the first node; The receiving module is used to receive the weighted data of the first public key and the second-encrypted first node user feature data; The processing module is further configured to calculate the weighted data of the second node user feature data and perform secondary encryption on the weighted data of the second node user feature data; wherein, the secondary encryption is to first encrypt using the first public key and then encrypt using the second public key; The processing module is also used to calculate the loss function and the second gradient after secondary encryption based on the weighted data of the first node user feature data after secondary encryption and the weighted data of the second node user feature data after secondary encryption. The second gradient is the gradient of the second logistic regression model of the federated learning model. The processing module is also used to decrypt the double-encrypted loss function using the second private key and then send it to the first node; The processing module is further configured to calculate a second gradient based on the user feature data and tags; The optimization module is used to optimize and iterate the parameters of the second logistic regression model according to the second gradient. When the value of the loss function meets the preset conditions, the iteration stops and the coefficient vector of the user features of the second node is obtained at this time.

12. A network device, characterized in that, include: A processor, a memory, and a program stored in the memory and executable on the processor, wherein the program, when executed by the processor, implements the steps of the federated learning method as claimed in any one of claims 1 to 5 or 6 to 9.

13. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, implements the steps of the federated learning method as described in any one of claims 1 to 5 or 6 to 9.

Images