Training methods and systems for logistic regression models in longitudinal federated learning
By using homomorphic encryption and obfuscation in longitudinal federated learning, the problem of third-party access to privacy data is solved, and secure training and prediction of logistic regression models are achieved.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- JINGDONG TECH HLDG CO LTD
- Filing Date
- 2021-11-02
- Publication Date
- 2026-07-17
AI Technical Summary
In the context of vertical federated learning, third parties can obtain private data such as loss values and gradients, leading to privacy leaks, and existing technologies are not secure enough.
Homomorphic encryption technology is used to train models in the vertical federated learning system through encryption and obfuscation mechanisms, ensuring that data privacy is not leaked and enabling the updating of model parameters.
Without relying on third parties and while protecting the privacy of both parties' data, the training and prediction of the logistic regression model were completed, achieving secure model training and updates.
Smart Images
Figure CN116070709B_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of machine learning technology, and in particular to a training method, prediction method, longitudinal federated learning system, and storage medium for a logistic regression model in longitudinal federated learning. Background Technology
[0002] With the development of artificial intelligence, people have proposed the concept of "federated learning" to solve the problem of data silos. This allows both parties in a federation to train models and obtain model parameters without having to provide their own data, and it can also avoid the problem of data privacy leakage.
[0003] Vertical federated learning, when participants' data features have little overlap but users have significant overlap, extracts the user and data segments where participants share similar users but have different data features, and uses these segments to jointly train a machine learning model. For example, consider two participants, A and B, located in the same region. Participant A is a bank, and participant B is an e-commerce platform. Participants A and B have many identical users in the same region, but their businesses differ, resulting in different user data features. Specifically, the user data features recorded by A and B may be complementary. In such scenarios, vertical federated learning can be used to help A and B build a joint machine learning predictive model, enabling them to provide better services to their customers.
[0004] Logistic regression is a classic machine learning model suitable for classification problems. Due to its simplicity, efficiency, and strong interpretability, it is widely used in finance, recommendation systems, and advertising. Many real-world scenarios require the training and use of logistic regression models within a federated learning framework.
[0005] However, in the logistic regression algorithm in the longitudinal federated learning scenario, the third party needs to obtain privacy data such as loss values and gradients. Moreover, the third party holds the private key, which means that it can record and obtain privacy data in plaintext. Taking gradient as an example, the third party can easily deduce the model of the participants based on sum (gradient), which is not secure and privacy-preserving enough. Summary of the Invention
[0006] In view of this, the present invention provides a training method, a prediction method, a vertical federated learning system, and a storage medium for a logistic regression model in vertical federated learning, so as to complete the training of a logistic regression model in vertical federated learning without relying on a third party and while protecting the privacy data of both parties.
[0007] Specifically, the present invention is achieved through the following technical solution:
[0008] According to a first aspect of the present invention, a method for training a logistic regression model in longitudinal federated learning is provided, the method comprising:
[0009] For any party in the vertical federated learning system, perform the following operation:
[0010] Receive a second public key and a second encrypted intermediate result sent by the other party; the second public key is a homomorphic encryption public key generated by the other party, and the second encrypted intermediate result is obtained by the other party using the second public key to encrypt its generated second intermediate result for calculating gradients;
[0011] Based on the existing first encryption intermediate result, the second encryption intermediate result, and the local first sample data, the encryption gradient is obtained; the first encryption intermediate result is obtained by encrypting the locally generated first intermediate result used to calculate the gradient using the second public key.
[0012] The encryption gradient is obfuscated to obtain an encrypted obfuscated gradient, which is then sent to the other party.
[0013] Receive a plaintext obfuscation gradient sent by another party; the plaintext obfuscation gradient is obtained by the other party decrypting the cryptographic obfuscation gradient using its generated homomorphic encryption private key;
[0014] The plaintext obfuscation gradient is deobfuscated to obtain the gradient, and a portion of the local logistic regression model is updated based on the gradient; the process is iterated until a preset stopping condition is met to obtain the trained logistic regression model.
[0015] In some embodiments, obtaining the encryption gradient based on the existing first encryption intermediate result, the second encryption intermediate result, and the local first sample data includes:
[0016] The encryption gradient is obtained by adding the first and second encryption intermediate results together and then multiplying them by the first sample data.
[0017] In some embodiments, the local entity is the initiator, the initiator possesses the tag data, and the other party is the participant;
[0018] The first intermediate result is generated based on the model parameters of a portion of the logistic regression model owned by the initiator, the sample data of the initiator, and the label data;
[0019] The second intermediate result is generated based on the model parameters of a portion of the logistic regression model owned by the participating party, as well as the sample data of the participating party.
[0020] In some embodiments, before updating a portion of the locally owned logistic regression model based on the gradient, the method further includes:
[0021] Receive the second encrypted squared intermediate result sent by the participant; the second encrypted squared intermediate result is obtained by the participant encrypting the square of the second intermediate result using the second public key;
[0022] Based on the existing first encrypted squared intermediate result, the first encrypted intermediate result, the second encrypted intermediate result, the second encrypted squared intermediate result, and the tag data, the encryption loss value is calculated; the first encrypted squared intermediate result is obtained by the initiator encrypting the square of the first intermediate result using the second public key;
[0023] The encryption loss value is obfuscated to obtain an encryption obfuscation loss value, which is then sent to the participating party.
[0024] Receive the plaintext obfuscation loss value sent by the participant; the plaintext obfuscation loss value is obtained by the participant decrypting the cryptographic obfuscation loss value using the private key of the homomorphic encryption it generated;
[0025] The plaintext obfuscation loss value is deobfuscated to obtain the loss value;
[0026] The loss value is used to determine whether the logistic regression model has converged. If it has converged, the logistic regression model is determined to have completed training, and a stop training instruction is sent to the participants.
[0027] If convergence is not achieved, after updating a portion of the logistic regression model owned by the initiator based on the gradient, the process returns to the step of receiving the second encrypted intermediate result sent by the participant.
[0028] In some embodiments, the local party is a participant, the other party is an initiator, and the initiator possesses the tag data;
[0029] The first intermediate result is generated based on the model parameters of a portion of the logistic regression model owned by the participating party, as well as the sample data of the participating party;
[0030] The second intermediate result is generated based on the model parameters of a portion of the logistic regression model owned by the initiator, the sample data of the initiator, and the label data.
[0031] In some embodiments, before updating a portion of the locally owned logistic regression model based on the gradient, the method further includes:
[0032] The system receives the tag data and the second encrypted squared intermediate result sent by the initiator; the second encrypted squared intermediate result is obtained by the initiator encrypting the square of the second intermediate result using the second public key.
[0033] Based on the existing first encrypted squared intermediate result, the second encrypted intermediate result, the second encrypted squared intermediate result, and the tag data, the encryption loss value is calculated; the first encrypted squared intermediate result is obtained by the participant encrypting the square of the first intermediate result using the second public key;
[0034] The encryption loss value is obfuscated to obtain an encryption obfuscation loss value, which is then sent to the initiator.
[0035] Receive the plaintext obfuscation loss value sent by the initiator; the plaintext obfuscation loss value is obtained by the initiator decrypting the cryptographic obfuscation loss value using the private key of the homomorphic encryption it generates;
[0036] The plaintext obfuscation loss value is deobfuscated to obtain the loss value;
[0037] Based on the loss value, determine whether the logistic regression model has converged. If it has converged, determine that the logistic regression model training is complete and send a stop training instruction to the initiator.
[0038] If convergence is not achieved, after updating a portion of the logistic regression model held by the participants based on the gradient, the process returns to the step of receiving the second encrypted intermediate result sent by the initiator.
[0039] According to a second aspect of the present invention, a prediction method based on a logistic regression model in longitudinal federated learning is provided, the method being applied to a longitudinal federated learning system, the longitudinal federated learning system comprising: an initiator and participants;
[0040] The method includes:
[0041] The initiator inputs the first data of the user to be predicted into a portion of the logistic regression model owned by the initiator to obtain the prediction result of the user to be predicted; and / or
[0042] The participating party inputs the second data of the user to be predicted into a portion of the logistic regression model owned by the participating party to obtain the prediction result of the user to be predicted;
[0043] The logistic regression model is trained according to the training method for logistic regression models in longitudinal federated learning described in any of the above embodiments.
[0044] According to a third aspect of the present invention, a vertical federated learning system is provided, the system comprising: an initiator and participants;
[0045] For any party in the vertical federated learning system, perform the following operation:
[0046] Receive a second public key and a second encrypted intermediate result sent by the other party; the second public key is a homomorphic encryption public key generated by the other party, and the second encrypted intermediate result is obtained by the other party using the second public key to encrypt its generated second intermediate result for calculating gradients;
[0047] Based on the existing first encryption intermediate result, the second encryption intermediate result, and the local first sample data, the encryption gradient is obtained; the first encryption intermediate result is obtained by encrypting the locally generated first intermediate result used to calculate the gradient using the second public key.
[0048] The encryption gradient is obfuscated to obtain an encrypted obfuscated gradient, which is then sent to the other party.
[0049] Receive a plaintext obfuscation gradient sent by another party; the plaintext obfuscation gradient is obtained by the other party decrypting the cryptographic obfuscation gradient using its generated homomorphic encryption private key;
[0050] The plaintext obfuscation gradient is deobfuscated to obtain the gradient, and a portion of the local logistic regression model is updated based on the gradient; the process is iterated until a preset stopping condition is met to obtain the trained logistic regression model.
[0051] In some embodiments, the local entity is the initiator, the initiator possesses the tag data, and the other party is the participant;
[0052] The initiator is also configured as follows:
[0053] Receive the second encrypted squared intermediate result sent by the participant; the second encrypted squared intermediate result is obtained by the participant encrypting the square of the second intermediate result using the second public key;
[0054] Based on the existing first encrypted squared intermediate result, the first encrypted intermediate result, the second encrypted intermediate result, the second encrypted squared intermediate result, and the tag data, the encryption loss value is calculated; the first encrypted squared intermediate result is obtained by the initiator encrypting the square of the first intermediate result using the second public key;
[0055] The encryption loss value is obfuscated to obtain an encryption obfuscation loss value, which is then sent to the participating party.
[0056] Receive the plaintext obfuscation loss value sent by the participant; the plaintext obfuscation loss value is obtained by the participant decrypting the cryptographic obfuscation loss value using the private key of the homomorphic encryption it generated;
[0057] The plaintext obfuscation loss value is deobfuscated to obtain the loss value;
[0058] The loss value is used to determine whether the logistic regression model has converged. If it has converged, the logistic regression model is determined to have completed training, and a stop training instruction is sent to the participants.
[0059] If convergence is not achieved, after updating a portion of the logistic regression model owned by the initiator based on the gradient, the process returns to the step of receiving the second encrypted intermediate result sent by the participant.
[0060] In some embodiments, the local party is a participant, the other party is an initiator, and the initiator possesses the tag data;
[0061] The participating parties are also configured as follows:
[0062] The system receives the tag data and the second encrypted squared intermediate result sent by the initiator; the second encrypted squared intermediate result is obtained by the initiator encrypting the square of the second intermediate result using the second public key.
[0063] Based on the existing first encrypted squared intermediate result, the second encrypted intermediate result, the second encrypted squared intermediate result, and the tag data, the encryption loss value is calculated; the first encrypted squared intermediate result is obtained by the participant encrypting the square of the first intermediate result using the second public key;
[0064] The encryption loss value is obfuscated to obtain an encryption obfuscation loss value, which is then sent to the initiator.
[0065] Receive the plaintext obfuscation loss value sent by the initiator; the plaintext obfuscation loss value is obtained by the initiator decrypting the cryptographic obfuscation loss value using the private key of the homomorphic encryption it generates;
[0066] The plaintext obfuscation loss value is deobfuscated to obtain the loss value;
[0067] Based on the loss value, determine whether the logistic regression model has converged. If it has converged, determine that the logistic regression model training is complete and send a stop training instruction to the initiator.
[0068] If convergence is not achieved, after updating a portion of the logistic regression model held by the participants based on the gradient, the process returns to the step of receiving the second encrypted intermediate result sent by the initiator.
[0069] In some embodiments, the initiator is further configured to:
[0070] The first data of the user to be predicted is input into a portion of the logistic regression model owned by the initiator to obtain the prediction result of the user to be predicted;
[0071] The participating parties are also configured as follows:
[0072] The second data of the user to be predicted is input into a portion of the logistic regression model owned by the participating party to obtain the prediction result of the user to be predicted.
[0073] According to a fourth aspect of the invention, a storage medium is provided having a computer program stored thereon, which, when executed by a processor, implements the steps of a training method for a logistic regression model in longitudinal federated learning, as described in the first aspect or any possible implementation thereof.
[0074] According to a fifth aspect of the present invention, a computer program product is provided, comprising a computer program that, when executed by a processor, implements the steps of a training method for a logistic regression model in longitudinal federated learning, as described in the first aspect or any possible implementation thereof.
[0075] According to a sixth aspect of the invention, a storage medium is provided having a computer program stored thereon, which, when executed by a processor, implements the steps of the prediction method based on a logistic regression model in longitudinal federated learning, as described in the second aspect or any possible implementation thereof.
[0076] According to a seventh aspect of the present invention, a computer program product is provided, comprising a computer program that, when executed by a processor, implements the steps of a prediction method based on a logistic regression model in longitudinal federated learning, as described in the second aspect or any possible implementation thereof.
[0077] The technical solution provided by this invention brings at least the following beneficial effects:
[0078] The technical solution provided by this invention, during the training of a logistic regression model, employs obfuscation and encryption mechanisms to prevent either party in a vertical federated learning system from directly obtaining the plaintext data of the other party. However, model-related calculations can be performed under encryption to update model parameters. This achieves the training of a logistic regression model in vertical federated learning without relying on a third party and while protecting the privacy of both parties' data. Attached Figure Description
[0079] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with the invention and, together with the description, serve to explain the principles of the invention.
[0080] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or related technologies will be briefly introduced below. Obviously, those skilled in the art can obtain other drawings based on these drawings without creative effort.
[0081] Figure 1 An architecture diagram of a vertical federated learning system provided in an embodiment of the present invention;
[0082] Figure 2 A flowchart illustrating a training method for a logistic regression model in longitudinal federated learning, provided in an embodiment of the present invention;
[0083] Figure 3 An interaction diagram of a vertical federated learning system provided in an embodiment of the present invention. Detailed Implementation
[0084] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.
[0085] Reference Figure 1 The diagram illustrates the architecture of a vertical federated learning system provided in an embodiment of the present invention. The vertical federated learning system 100 includes at least two data nodes, one of which acts as an initiator 101 and the other as a participant 102.
[0086] Perform the following operation on any party in the vertical federated learning system 100:
[0087] Receive a second public key and a second encrypted intermediate result sent by the other party; the second public key is a homomorphic encryption public key generated by the other party, and the second encrypted intermediate result is obtained by the other party using the second public key to encrypt its generated second intermediate result for calculating gradients;
[0088] Based on the existing first encryption intermediate result, the second encryption intermediate result, and the local first sample data, the encryption gradient is obtained; the first encryption intermediate result is obtained by encrypting the locally generated first intermediate result used to calculate the gradient using the second public key.
[0089] The encryption gradient is obfuscated to obtain an encrypted obfuscated gradient, which is then sent to the other party.
[0090] Receive a plaintext obfuscation gradient sent by another party; the plaintext obfuscation gradient is obtained by the other party decrypting the cryptographic obfuscation gradient using its generated homomorphic encryption private key;
[0091] The plaintext obfuscation gradient is deobfuscated to obtain the gradient, and a portion of the local logistic regression model is updated based on the gradient; the process is iterated until a preset stopping condition is met to obtain the trained logistic regression model.
[0092] In this embodiment of the invention, the preset stopping condition can be a condition that the loss value is less than the set convergence value (e.g., 1e-4), or it can be a condition that the number of iterations reaches the maximum number of iterations.
[0093] Scenario 1: The initiator 101 performs the above operation. In this case, the local party is the initiator 101, which has the tag data, and the other party is the participant 102.
[0094] The aforementioned first intermediate result was generated based on the model parameters of a portion of the logistic regression model owned by the initiator 101, the sample data of the initiator 101, and the label data.
[0095] Therefore, the first intermediate encryption result mentioned above is obtained by initiator 101 encrypting the first intermediate result using the public key of homomorphic encryption generated by participant 102;
[0096] The aforementioned second intermediate result was generated based on the model parameters of a portion of the logistic regression model owned by participant 102, as well as the sample data of participant 102;
[0097] Therefore, the above-mentioned second intermediate encryption result is obtained by participant 102 encrypting the second intermediate result using the public key of homomorphic encryption generated by participant 102;
[0098] The encryption gradient is obtained based on the existing first encryption intermediate result, the second encryption intermediate result, and the local first sample data, including:
[0099] Add the first encryption intermediate result and the second encryption intermediate result together, and then multiply them by the sample data of the initiator 101 to obtain the encryption gradient of the initiator 101.
[0100] Initiator 101 was also configured as:
[0101] Receive the second encrypted squared intermediate result sent by participant 102; the second encrypted squared intermediate result is obtained by participant 102 encrypting the square of the second intermediate result using the public key of homomorphic encryption generated by participant 102;
[0102] Based on the existing first encrypted squared intermediate result, the second encrypted intermediate result, the second encrypted squared intermediate result, and the tag data, the encryption loss value is calculated; the first encrypted squared intermediate result is obtained by initiator 101 encrypting the square of the first intermediate result using the public key of participant 102;
[0103] The encryption loss value is obfuscated to obtain an encryption obfuscation loss value, which is then sent to participant 102.
[0104] The plaintext obfuscation loss value is received by participant 102; the plaintext obfuscation loss value is obtained by participant 102 decrypting the cryptographic obfuscation loss value using the private key of the homomorphic encryption it generates;
[0105] The plaintext obfuscation loss value is deobfuscated to obtain the loss value;
[0106] Based on the loss value, determine whether the logistic regression model has converged. If it has converged, determine that the logistic regression model training is complete and send a stop training command to participant 102.
[0107] If convergence is not achieved, after updating a portion of the logistic regression model owned by the initiator 101 based on the gradient, the process returns to the step of receiving the second encrypted intermediate result sent by the participant 102.
[0108] Scenario 2: Participant 102 performs the above operation. In this case, the local party is participant 102, and the other party is initiator 101. Initiator 101 has the tag data.
[0109] The aforementioned first intermediate result was generated based on the model parameters of a portion of the logistic regression model owned by participant 102, as well as the sample data of participant 102;
[0110] Therefore, the first intermediate encryption result mentioned above is obtained by participant 102 encrypting the first intermediate result using the public key of homomorphic encryption generated by initiator 101;
[0111] The aforementioned second intermediate result was generated based on the model parameters of a portion of the logistic regression model owned by the initiator 101, the sample data of the initiator 101, and the label data.
[0112] Therefore, the above-mentioned second encrypted intermediate result is obtained by initiator 101 using initiator 101's public key to encrypt the second intermediate result;
[0113] The encryption gradient is obtained based on the existing first encryption intermediate result, the second encryption intermediate result, and the local first sample data, including:
[0114] The first and second encryption intermediate results are added together, and then multiplied by the sample data of participant 102 to obtain the encryption gradient of participant 102.
[0115] Participant 102 is also configured as:
[0116] The system receives the tag data and the second encrypted squared intermediate result sent by the initiator 101; the second encrypted squared intermediate result is obtained by the initiator 101 using the initiator 101's public key to encrypt the square of the second intermediate result.
[0117] Based on the existing first encrypted square intermediate result, the second encrypted intermediate result, the second encrypted square intermediate result, and the tag data, the encryption loss value is calculated; the first encrypted square intermediate result is obtained by participant 102 encrypting the square of the first intermediate result using the public key of initiator 101;
[0118] The encryption loss value is obfuscated to obtain an encryption obfuscation loss value, which is then sent to the initiator 101.
[0119] Receive the plaintext obfuscation loss value sent by the initiator 101; the plaintext obfuscation loss value is obtained by the initiator 101 using its generated homomorphic encryption private key to decrypt the encryption obfuscation loss value;
[0120] The plaintext obfuscation loss value is deobfuscated to obtain the loss value;
[0121] Based on the loss value, determine whether the logistic regression model has converged. If it has converged, determine that the logistic regression model training is complete and send a stop training command to the initiator 101.
[0122] If convergence is not achieved, after updating a portion of the logistic regression model owned by participant 102 based on the gradient, the process returns to the step of receiving the second encrypted intermediate result sent by initiator 101.
[0123] In some embodiments, the above-described longitudinal federated learning system may make predictions based on a trained logistic regression model.
[0124] At this point, initiator 101 is also configured as:
[0125] The first data of the user to be predicted is input into a portion of the logistic regression model owned by the initiator 101 to obtain the prediction result of the user to be predicted;
[0126] Participant 102 is also configured as:
[0127] The second data of the user to be predicted is input into a portion of the logistic regression model owned by participant 102 to obtain the prediction result of the user to be predicted.
[0128] For example, in the financial sector, banks collaborate with e-commerce companies to identify financial fraud. The e-commerce company is one participant, and the bank is the initiator. The sample users are shared users of both the e-commerce company and the bank, such as small and micro-enterprises. The e-commerce company can hold transaction data from the sample users, such as order volume, and extract secondary data based on this transaction data. The bank can hold loan data from the sample users, such as loan amounts, or business data, such as establishment year and company size, and extract primary data from these data. Furthermore, as the initiator, the bank also holds the sample users' authentic labels, which can indicate whether the sample user has engaged in financial fraud.
[0129] Based on the same inventive concept, see [link to inventive concept] Figure 2 This invention provides a training method for a logistic regression model in vertical federated learning. This method can be applied to the aforementioned vertical federated learning system and may include the following steps:
[0130] For any party in a vertical federated learning system, the following operations are performed:
[0131] S101, Receive the second public key and the second encrypted intermediate result sent by the other party;
[0132] Wherein, the second public key is a homomorphic encryption public key generated by the other party, and the second encrypted intermediate result is obtained by the other party using the second public key to encrypt its generated second intermediate result for calculating the gradient.
[0133] S102. Based on the existing first encryption intermediate result, the second encryption intermediate result, and the local first sample data, obtain the encryption gradient;
[0134] The first encrypted intermediate result is obtained by locally encrypting the first intermediate result used to calculate the gradient using the second public key.
[0135] S103. Obfuscate the encryption gradient to obtain an encrypted obfuscated gradient and send it to the other party;
[0136] S104. Receive the plaintext obfuscation gradient sent by the other party;
[0137] The plaintext obfuscation gradient is obtained by the other party decrypting the cryptographic obfuscation gradient using the private key of the homomorphic encryption it generates.
[0138] S105. Deobfuscate the plaintext obfuscation gradient to obtain the gradient, and update a portion of the local logistic regression model based on the gradient; iterate until a preset stopping condition is met to obtain the trained logistic regression model.
[0139] In some embodiments, obtaining the encryption gradient based on the existing first encryption intermediate result, the second encryption intermediate result, and the local first sample data includes:
[0140] The encryption gradient is obtained by adding the first and second encryption intermediate results together and then multiplying them by the first sample data.
[0141] In some embodiments, the local entity is the initiator, the initiator possesses the tag data, and the other party is the participant;
[0142] The first intermediate result is generated based on the model parameters of a portion of the logistic regression model owned by the initiator, the sample data of the initiator, and the label data;
[0143] The second intermediate result is generated based on the model parameters of a portion of the logistic regression model owned by the participating party, as well as the sample data of the participating party.
[0144] In some embodiments, before updating a portion of the locally owned logistic regression model based on the gradient, the method further includes:
[0145] Receive the second encrypted squared intermediate result sent by the participant; the second encrypted squared intermediate result is obtained by the participant encrypting the square of the second intermediate result using the second public key;
[0146] Based on the existing first encrypted squared intermediate result, the first encrypted intermediate result, the second encrypted intermediate result, the second encrypted squared intermediate result, and the tag data, the encryption loss value is calculated; the first encrypted squared intermediate result is obtained by the initiator encrypting the square of the first intermediate result using the second public key;
[0147] The encryption loss value is obfuscated to obtain an encryption obfuscation loss value, which is then sent to the participating party.
[0148] Receive the plaintext obfuscation loss value sent by the participant; the plaintext obfuscation loss value is obtained by the participant decrypting the cryptographic obfuscation loss value using the private key of the homomorphic encryption it generated;
[0149] The plaintext obfuscation loss value is deobfuscated to obtain the loss value;
[0150] The loss value is used to determine whether the logistic regression model has converged. If it has converged, the logistic regression model is determined to have completed training, and a stop training instruction is sent to the participants.
[0151] If convergence is not achieved, after updating a portion of the logistic regression model owned by the initiator based on the gradient, the process returns to the step of receiving the second encrypted intermediate result sent by the participant.
[0152] In some embodiments, the local party is a participant, the other party is an initiator, and the initiator possesses the tag data;
[0153] The first intermediate result is generated based on the model parameters of a portion of the logistic regression model owned by the participating party, as well as the sample data of the participating party;
[0154] The second intermediate result is generated based on the model parameters of a portion of the logistic regression model owned by the initiator, the sample data of the initiator, and the label data.
[0155] In some embodiments, before updating a portion of the locally owned logistic regression model based on the gradient, the method further includes:
[0156] The system receives the tag data and the second encrypted squared intermediate result sent by the initiator; the second encrypted squared intermediate result is obtained by the initiator encrypting the square of the second intermediate result using the second public key.
[0157] Based on the existing first encrypted squared intermediate result, the second encrypted intermediate result, the second encrypted squared intermediate result, and the tag data, the encryption loss value is calculated; the first encrypted squared intermediate result is obtained by the participant encrypting the square of the first intermediate result using the second public key;
[0158] The encryption loss value is obfuscated to obtain an encryption obfuscation loss value, which is then sent to the initiator.
[0159] Receive the plaintext obfuscation loss value sent by the initiator; the plaintext obfuscation loss value is obtained by the initiator decrypting the cryptographic obfuscation loss value using the private key of the homomorphic encryption it generates;
[0160] The plaintext obfuscation loss value is deobfuscated to obtain the loss value;
[0161] Based on the loss value, determine whether the logistic regression model has converged. If it has converged, determine that the logistic regression model training is complete and send a stop training instruction to the initiator.
[0162] If convergence is not achieved, after updating a portion of the logistic regression model held by the participants based on the gradient, the process returns to the step of receiving the second encrypted intermediate result sent by the initiator.
[0163] like Figure 3 As shown below, taking A as the initiator and B as the participant as an example, the process of training the logistic regression model in vertical federated learning provided by the embodiments of the present invention is explained.
[0164] Initiator A and participant B each possess a portion of the logistic regression model, with initiator A possessing the labeled data y.
[0165] S201, Initiator A generates a homomorphic encryption public key A1 and a private key A2;
[0166] S202, Initiator A sends public key A1 to participant B;
[0167] S203, Participant B generates a homomorphic encryption public key B1 and a private key B2;
[0168] S204. Participant B sends public key B1 to initiator A;
[0169] S205, Initiator A, based on the model parameters w of a portion of its logistic regression model... a Sample data x of initiator A a Using the tag data y, generate the intermediate result U of the initiator A. a Initiator A uses public key A1 to encrypt intermediate result U. a Obtain the encrypted intermediate result ENa(U) from the initiator A a The intermediate result U is encrypted using the public key B1. a Obtain the encrypted intermediate result ENb(U) from the initiator A a ), and use public key B1 to encrypt intermediate result U a The square of is obtained as the encrypted square intermediate result ENb(U) of the initiator A. a 2 );
[0170] In this embodiment of the invention, the intermediate result U of the initiator A a It can be generated using the following formula (1).
[0171]
[0172] S206, Initiator A will encrypt the intermediate result ENa(U a Send to participant B;
[0173] S207, Participant B's model parameters w based on a portion of the logistic regression model it possesses. band sample data x of participant B. b Generate intermediate result U for participant B. b Participant B uses public key B1 to encrypt intermediate result U. b Obtain the encrypted intermediate result ENb(U) from participant B. b The intermediate result U is encrypted using the public key A1. b Obtain the encrypted intermediate result ENa(U) from participant B. b ), and use public key B to encrypt intermediate result U. b The square of is used to obtain the encrypted squared intermediate result ENb(U) of participant B. b 2 );
[0174] In this embodiment of the invention, the intermediate result U of participant B b It can be generated using the following formula (2).
[0175]
[0176] S208, Participant B will encrypt the intermediate result ENb(U b ) and encrypted squared intermediate result ENb(U b 2 Send to initiator A;
[0177] S209, Initiator A based on the encrypted squared intermediate result ENb(U a 2 ), Encrypted intermediate result ENb(U a ), Encrypted intermediate result ENb(U b ), encrypted squared intermediate result ENb(U b 2 Given the data y and the label data y, calculate the encryption loss value ENb(loss), and then obfuscate the encryption loss value ENb(loss) to obtain the encryption obfuscation loss value ENb(loss').
[0178] In this embodiment of the invention, the encryption loss value ENb(loss) can be calculated using the following formula (3).
[0179]
[0180] Among them, [U a ] represents the encrypted intermediate result ENb(U a ), [U b ] represents the encrypted intermediate result ENb(U b ), Represents the intermediate result of encrypted squared ENb(U) a 2 ), Represents the intermediate result of encrypted squared ENb(U) b 2 ), S=(X A ,X B ) represents the training dataset, X A Let X represent the dataset of common sample users owned by initiator A. B This represents the dataset of common sample users owned by participant B.
[0181] S210, Initiator A sends the encryption obfuscation loss value ENb(loss') to participant B;
[0182] S211. Participant B uses private key B2 to decrypt the encrypted obfuscation loss value ENb(loss') to obtain the plaintext obfuscation loss value loss'.
[0183] S212. Participant B sends the plaintext obfuscation loss value 'loss' to initiator A;
[0184] S213. Initiator A deobfuscates the plaintext obfuscation loss value loss' to obtain the loss value loss. Based on the loss value loss, it is determined whether the logistic regression model has converged. If it has converged, the logistic regression model training is completed, and step S214 is executed. If it has not converged, after executing step S223, it returns to execute step S205.
[0185] S214. Initiator A sends a stop training instruction to participant B;
[0186] S215, Initiator A will encrypt the intermediate result ENb(U a ) and encrypted intermediate result ENb(U b After adding them together, then add them to the sample data x of the initiator A. a Multiplying them together yields the encryption gradient ENb(G) of initiator A. a ), for the encryption gradient ENb(G a The encryption is obfuscated to obtain the encryption obfuscation gradient ENb(G) of the initiator A. a ');
[0187] In this embodiment of the invention, the encryption gradient ENb(G) of the initiator A a ) can be obtained through the following formula (4).
[0188] ENb(G a )=ENb(U a )+ENb(U b ))*x a (4)
[0189] S216, Initiator A will encrypt and obfuscate the gradient ENb(G) a ')Send to participant B;
[0190] S217, Participant B will encrypt the intermediate result ENa(U a ) and encrypted intermediate result ENa(U b After adding them together, then add them to the sample data x of participant B. b Multiplying them together yields the cryptographic gradient ENa(G) of participant B. b ), for the encryption gradient ENa(G) b Obfuscation is performed to obtain the encryption obfuscation gradient ENa(G) of participant B. b ');
[0191] In this embodiment of the invention, the encryption gradient ENa(G) of participant B b It can be obtained through the following formula (5).
[0192] ENa(G b )=(ENa(U a )+ENa(U b ))*x b (5)
[0193] S218, Participant B will encrypt and obfuscate the gradient ENa(G) b ')Sent to initiator A;
[0194] S219, Initiator A uses private key A2 to decrypt the encrypted obfuscated gradient ENa(G) b '), thus obtaining the plaintext obfuscation gradient G of participant B. b ';
[0195] S220, Initiator A obfuscates plaintext gradient G b 'Sent to participant B;'
[0196] S221, Participant B uses private key B2 to decrypt the encrypted obfuscated gradient ENb(G) a '), thus obtaining the plaintext obfuscation gradient G of the initiator A. a ';
[0197] S222, Participant B obfuscates plaintext gradient G a 'Sent to initiator A;'
[0198] S223, Initiator A's plaintext obfuscation gradient G a 'Perform deobfuscation to obtain the gradient G of the initiator A.' a Based on gradient G a Update a portion of the logistic regression model owned by initiator A;
[0199] S224, Participant B's plaintext obfuscation gradient G b 'Perform deobfuscation to obtain the gradient G of participant B.' bBased on gradient G b Update a portion of the logistic regression model owned by participant B;
[0200] The training process is repeated until a preset stopping condition is met, at which point the logistic regression model is obtained.
[0201] Based on the same inventive concept, this embodiment of the invention also provides a prediction method based on a logistic regression model in vertical federated learning. The method is applied to a vertical federated learning system, which includes an initiator and participants.
[0202] The method includes:
[0203] The initiator inputs the first data of the user to be predicted into a portion of the logistic regression model owned by the initiator to obtain the prediction result of the user to be predicted; and / or
[0204] The participating party inputs the second data of the user to be predicted into a portion of the logistic regression model owned by the participating party to obtain the prediction result of the user to be predicted;
[0205] The logistic regression model is trained according to the training method for logistic regression models in longitudinal federated learning described in any of the above embodiments.
[0206] Based on the same inventive concept, embodiments of the present invention also provide a storage medium storing a computer program thereon, wherein when the program is executed by a processor, it implements the steps of the training method for the logistic regression model in longitudinal federated learning in any of the above possible implementations.
[0207] Optionally, the storage medium may be a non-transitory computer-readable storage medium, such as a ROM, random access memory (RAM), CD-ROM, magnetic tape, floppy disk, and optical data storage device.
[0208] Based on the same inventive concept, embodiments of the present invention also provide a computer program product, including a computer program, which, when executed by a processor, implements the steps of the training method for the logistic regression model in longitudinal federated learning in any of the above possible implementations.
[0209] Based on the same inventive concept, embodiments of the present invention also provide a storage medium storing a computer program thereon, which, when executed by a processor, implements the steps of the prediction method based on the logistic regression model in longitudinal federated learning in any of the above possible implementations.
[0210] Optionally, the storage medium may be a non-transitory computer-readable storage medium, such as a ROM, random access memory (RAM), CD-ROM, magnetic tape, floppy disk, and optical data storage device.
[0211] Based on the same inventive concept, embodiments of the present invention also provide a computer program product, including a computer program, which, when executed by a processor, implements the steps of the prediction method based on the logistic regression model in longitudinal federated learning in any of the above possible implementations.
[0212] The embodiments of the subject matter and functional operation described in this specification can be implemented in the following ways: digital electronic circuits, tangibly embodied computer software or firmware, computer hardware including the structures disclosed in this specification and their structural equivalents, or combinations thereof. Embodiments of the subject matter described in this specification can be implemented as one or more computer programs, i.e., one or more modules of computer program instructions encoded on a tangible, non-transitory program carrier for execution by a data processing apparatus or for controlling the operation of a data processing apparatus. Alternatively or additionally, the program instructions may be encoded on artificially generated propagation signals, such as machine-generated electrical, optical, or electromagnetic signals, which are generated to encode information and transmit it to a suitable receiving device for execution by the data processing apparatus. The computer storage medium may be a machine-readable storage device, a machine-readable storage substrate, a random or serial access memory device, or combinations thereof.
[0213] The processing and logic flow described in this specification can be executed by one or more programmable computers that execute one or more computer programs to perform corresponding functions by operating on input data and generating output. The processing and logic flow can also be executed by dedicated logic circuitry—such as FPGAs (Field-Programmable Gate Arrays) or ASICs (Application-Specific Integrated Circuits), and the device can also be implemented as dedicated logic circuitry.
[0214] Suitable computers for executing computer programs include, for example, general-purpose and / or special-purpose microprocessors, or any other type of central processing unit. Typically, the central processing unit receives instructions and data from read-only memory and / or random access memory. The basic components of a computer include a central processing unit for implementing or executing instructions and one or more memory devices for storing instructions and data. Typically, a computer will also include one or more mass storage devices for storing data, such as disks, magneto-optical disks, or optical disks, or the computer will be operatively coupled to such mass storage devices to receive data from or transfer data to them, or both. However, a computer is not required to have such devices. Furthermore, a computer can be embedded in another device, such as a mobile phone, a personal digital assistant (PDA), a mobile audio or video player, a game console, a global positioning system (GPS) receiver, or a portable storage device such as a universal serial bus (USB) flash drive, to name a few.
[0215] Computer-readable media suitable for storing computer program instructions and data include all forms of non-volatile memory, media, and memory devices, such as semiconductor memory devices (e.g., EPROM, EEPROM, and flash memory devices), magnetic disks (e.g., internal hard disks or removable disks), magneto-optical disks, and CD-ROM and DVD-ROM disks. Processors and memory may be supplemented by or incorporated into dedicated logic circuitry.
[0216] While this specification contains numerous specific implementation details, these should not be construed as limiting the scope of any invention or the scope of the claims, but rather are primarily intended to describe features of specific embodiments of a particular invention. Certain features described in the various embodiments herein may also be implemented in combination in a single embodiment. Conversely, various features described in a single embodiment may also be implemented separately in various embodiments or in any suitable sub-combination. Furthermore, while features may function in certain combinations as described above and even initially claimed in this way, one or more features from a claimed combination may be removed from that combination in some cases, and a claimed combination may refer to a sub-combination or a variation thereof.
[0217] Similarly, although the operations are depicted in a specific order in the accompanying drawings, this should not be construed as requiring these operations to be performed in the specific order shown or sequentially, or requiring all illustrated operations to be performed to achieve the desired result. In some cases, multitasking and parallel processing may be advantageous. Furthermore, the separation of various system modules and components in the above embodiments should not be construed as requiring such separation in all embodiments, and it should be understood that the described program components and systems can generally be integrated together in a single software product or packaged into multiple software products.
[0218] Thus, specific embodiments of the subject matter have been described. Other embodiments are within the scope of the appended claims. In some cases, the actions recited in the claims may be performed in a different order and still achieve the desired result. Furthermore, the processes depicted in the drawings are not necessarily shown in a specific order or sequence to achieve the desired result. In some implementations, multitasking and parallel processing may be advantageous.
[0219] It should be noted that, in this document, relational terms such as "first" and "second" are used merely to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.
[0220] The above description is merely a specific embodiment of the present invention, enabling those skilled in the art to understand or implement the invention. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the invention. Therefore, the present invention is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features claimed herein.
Claims
1. A training method for a logistic regression model in longitudinal federated learning, characterized in that, The method includes: For any party in the vertical federated learning system, perform the following operation: Receive a second public key and a second encrypted intermediate result sent by the other party; the second public key is a homomorphic encryption public key generated by the other party, and the second encrypted intermediate result is obtained by the other party using the second public key to encrypt its generated second intermediate result for calculating gradients; Based on the existing first encryption intermediate result, the second encryption intermediate result, and the local first sample data, the encryption gradient is obtained; the first encryption intermediate result is obtained by encrypting the locally generated first intermediate result used to calculate the gradient using the second public key. The encryption gradient is obfuscated to obtain an encrypted obfuscated gradient, which is then sent to the other party. Receive a plaintext obfuscation gradient sent by another party; the plaintext obfuscation gradient is obtained by the other party decrypting the cryptographic obfuscation gradient using its generated homomorphic encryption private key; The plaintext obfuscation gradient is deobfuscated to obtain the gradient, and a portion of the local logistic regression model is updated based on the gradient; the process is iterated until a preset stopping condition is met to obtain the trained logistic regression model.
2. The method according to claim 1, characterized in that, The step of obtaining the encryption gradient based on the existing first encryption intermediate result, the second encryption intermediate result, and the local first sample data includes: The encryption gradient is obtained by adding the first and second encryption intermediate results together and then multiplying them by the first sample data.
3. The method according to claim 1 or 2, characterized in that, The local party is the initiator, which possesses the tag data; the other party is the participant. The first intermediate result is generated based on the model parameters of a portion of the logistic regression model owned by the initiator, the sample data of the initiator, and the label data; The second intermediate result is generated based on the model parameters of a portion of the logistic regression model owned by the participating party, as well as the sample data of the participating party.
4. The method according to claim 3, characterized in that, Before updating a portion of the locally owned logistic regression model based on the gradient, the method further includes: Receive the second encrypted squared intermediate result sent by the participant; the second encrypted squared intermediate result is obtained by the participant encrypting the square of the second intermediate result using the second public key; Based on the existing first encrypted squared intermediate result, the first encrypted intermediate result, the second encrypted intermediate result, the second encrypted squared intermediate result, and the tag data, the encryption loss value is calculated; the first encrypted squared intermediate result is obtained by the initiator encrypting the square of the first intermediate result using the second public key; The encryption loss value is obfuscated to obtain an encryption obfuscation loss value, which is then sent to the participating party. Receive the plaintext obfuscation loss value sent by the participant; the plaintext obfuscation loss value is obtained by the participant decrypting the cryptographic obfuscation loss value using the private key of the homomorphic encryption it generated; The plaintext obfuscation loss value is deobfuscated to obtain the loss value; The loss value is used to determine whether the logistic regression model has converged. If it has converged, the logistic regression model is determined to have completed training, and a stop training instruction is sent to the participants. If convergence is not achieved, after updating a portion of the logistic regression model owned by the initiator based on the gradient, the process returns to the step of receiving the second encrypted intermediate result sent by the participant.
5. The method according to claim 1 or 2, characterized in that, The local party is a participant, and the other party is an initiator, who possesses the tag data; The first intermediate result is generated based on the model parameters of a portion of the logistic regression model owned by the participating party, as well as the sample data of the participating party; The second intermediate result is generated based on the model parameters of a portion of the logistic regression model owned by the initiator, the sample data of the initiator, and the label data.
6. The method according to claim 5, characterized in that, Before updating a portion of the locally owned logistic regression model based on the gradient, the method further includes: The system receives the tag data and the second encrypted squared intermediate result sent by the initiator; the second encrypted squared intermediate result is obtained by the initiator encrypting the square of the second intermediate result using the second public key. Based on the existing first encrypted squared intermediate result, the second encrypted intermediate result, the second encrypted squared intermediate result, and the tag data, the encryption loss value is calculated; the first encrypted squared intermediate result is obtained by the participant encrypting the square of the first intermediate result using the second public key; The encryption loss value is obfuscated to obtain an encryption obfuscation loss value, which is then sent to the initiator. Receive the plaintext obfuscation loss value sent by the initiator; the plaintext obfuscation loss value is obtained by the initiator decrypting the cryptographic obfuscation loss value using the private key of the homomorphic encryption it generates; The plaintext obfuscation loss value is deobfuscated to obtain the loss value; Based on the loss value, determine whether the logistic regression model has converged. If it has converged, determine that the logistic regression model training is complete and send a stop training instruction to the initiator. If convergence is not achieved, after updating a portion of the logistic regression model held by the participants based on the gradient, the process returns to the step of receiving the second encrypted intermediate result sent by the initiator.
7. A prediction method based on a logistic regression model in longitudinal federated learning, characterized in that, The method is applied to a vertical federated learning system, which includes an initiator and participants. The method includes: The initiator inputs the first data of the user to be predicted into a portion of the logistic regression model owned by the initiator to obtain the prediction result of the user to be predicted; and / or The participating party inputs the second data of the user to be predicted into a portion of the logistic regression model owned by the participating party to obtain the prediction result of the user to be predicted; The logistic regression model is trained using the method described in any one of claims 1-6.
8. A vertical federated learning system, characterized in that, The system includes: initiators and participants; For any party in the vertical federated learning system, perform the following operation: Receive a second public key and a second encrypted intermediate result sent by the other party; the second public key is a homomorphic encryption public key generated by the other party, and the second encrypted intermediate result is obtained by the other party using the second public key to encrypt its generated second intermediate result for calculating gradients; Based on the existing first encryption intermediate result, the second encryption intermediate result, and the local first sample data, the encryption gradient is obtained; the first encryption intermediate result is obtained by encrypting the locally generated first intermediate result used to calculate the gradient using the second public key. The encryption gradient is obfuscated to obtain an encrypted obfuscated gradient, which is then sent to the other party. Receive a plaintext obfuscation gradient sent by another party; the plaintext obfuscation gradient is obtained by the other party decrypting the cryptographic obfuscation gradient using its generated homomorphic encryption private key; The plaintext obfuscation gradient is deobfuscated to obtain the gradient, and a portion of the local logistic regression model is updated based on the gradient; the process is iterated until a preset stopping condition is met to obtain the trained logistic regression model.
9. The system according to claim 8, characterized in that, The local party is the initiator, which possesses the tag data; the other party is the participant. The initiator is also configured as follows: Receive the second encrypted squared intermediate result sent by the participant; the second encrypted squared intermediate result is obtained by the participant encrypting the square of the second intermediate result using the second public key; Based on the existing first encrypted squared intermediate result, the first encrypted intermediate result, the second encrypted intermediate result, the second encrypted squared intermediate result, and the tag data, the encryption loss value is calculated; the first encrypted squared intermediate result is obtained by the initiator encrypting the square of the first intermediate result using the second public key; The encryption loss value is obfuscated to obtain an encryption obfuscation loss value, which is then sent to the participating party. Receive the plaintext obfuscation loss value sent by the participant; the plaintext obfuscation loss value is obtained by the participant decrypting the cryptographic obfuscation loss value using the private key of the homomorphic encryption it generated; The plaintext obfuscation loss value is deobfuscated to obtain the loss value; The loss value is used to determine whether the logistic regression model has converged. If it has converged, the logistic regression model is determined to have completed training, and a stop training instruction is sent to the participants. If convergence is not achieved, after updating a portion of the logistic regression model owned by the initiator based on the gradient, the process returns to the step of receiving the second encrypted intermediate result sent by the participant.
10. The system according to claim 8, characterized in that, The local party is a participant, and the other party is an initiator, who possesses the tag data; The participating parties are also configured as follows: The system receives the tag data and the second encrypted squared intermediate result sent by the initiator; the second encrypted squared intermediate result is obtained by the initiator encrypting the square of the second intermediate result using the second public key. Based on the existing first encrypted squared intermediate result, the second encrypted intermediate result, the second encrypted squared intermediate result, and the tag data, the encryption loss value is calculated; the first encrypted squared intermediate result is obtained by the participant encrypting the square of the first intermediate result using the second public key; The encryption loss value is obfuscated to obtain an encryption obfuscation loss value, which is then sent to the initiator. Receive the plaintext obfuscation loss value sent by the initiator; the plaintext obfuscation loss value is obtained by the initiator decrypting the cryptographic obfuscation loss value using the private key of the homomorphic encryption it generates; The plaintext obfuscation loss value is deobfuscated to obtain the loss value; Based on the loss value, determine whether the logistic regression model has converged. If it has converged, determine that the logistic regression model training is complete and send a stop training instruction to the initiator. If convergence is not achieved, after updating a portion of the logistic regression model held by the participants based on the gradient, the process returns to the step of receiving the second encrypted intermediate result sent by the initiator.
11. The system according to claim 8, characterized in that, The initiator is also configured as follows: The first data of the user to be predicted is input into a portion of the logistic regression model owned by the initiator to obtain the prediction result of the user to be predicted; The participating parties are also configured as follows: The second data of the user to be predicted is input into a portion of the logistic regression model owned by the participating party to obtain the prediction result of the user to be predicted.
12. A storage medium having a computer program stored thereon, characterized in that, When the program is executed by a processor, it implements the steps of the method according to any one of claims 1-7.