Hybrid federated logistic regression method based on homomorphic encryption

Through homomorphic encryption technology, vertical and horizontal federal logistic regression training is carried out in data mixed slicing scenarios, the efficiency and accuracy of federated learning in the data mixed slicing scenarios in the existing technology are solved, and the secure joint construction of local data is realized Modal and efficient calculation.

WO2024239593A1PCT designated stage expired Publication Date: 2024-11-28CHINA TELECOM BESTPAY CO LTD

Patent Information

Application Number
PCT/CN2023/136359
Authority / Receiving Office
WO · WO
Patent Type
Applications
Current Assignee / Owner
Priority Date
2023-05-25
Filing Date
2023-12-05
Publication Date
2024-11-28

AI Technical Summary

Technical Problem

When facing data mixing and slicing scenarios, the existing technology needs to use horizontal federated learning and vertical federated learning to meet the needs, but the operation steps are complicated, the data interaction is large, the network and computing resources are occupied, which affects efficiency, and cannot be achieved in the Local computing out of the library limits the flexibility and accuracy of data modeling.

Method used

Using a hybrid federal logistic regression method based on homomorphic encryption, the data is divided by modules, vertical and horizontal federal logistic regression training is carried out, and homomorphic public-private key pairs are generated using the key exchange protocol, and the encryption parameters are securely aggregated to realize the data. Joint modeling training breaks through the bottleneck of data modeling in a single institution.

Benefits of technology

It simplifies operation steps, reduces the amount of data interaction and resource usage, improves computing efficiency, realizes federated learning model training without local data being out of the database, improves the accuracy of federated learning model, and supports secure multi-party computing for complex computing tasks.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN2023136359_28112024_PF_FP_ABST
    Figure CN2023136359_28112024_PF_FP_ABST
Patent Text Reader

Abstract

The present invention relates to the technical field of computers, and provides a hybrid federated logistic regression method based on homomorphic encryption. The method comprises: performing module segmentation on data to form data data1 and data data2; performing longitudinal federated logistic regression training on the data data1 and the data data2; for the data data1 and the data data2, using a random number r as a seed to generate a homomorphic public-private key pair; encrypting w1, intercept1, w2, and intercept2, and performing security aggregation on encrypted parameters; and sending an aggregation intermediate result parameter to each participant, so that each participant decrypts the aggregation intermediate result parameter, and performs parameter iteration until a requirement is met to obtain a final training model. The bottleneck of modeling and intercommunication of internal and external data of a single institution is effectively broken through, local data does not leave a database, and federated learning model training can be performed in combination with multi-party data, thereby improving the accuracy of a federated learning model.
Need to check novelty before this filing date? Find Prior Art

Description

A hybrid federated logistic regression method based on homomorphic encryption Technical Field

[0001] The present invention relates to the field of computer technology, and in particular to a hybrid federated logistic regression method based on homomorphic encryption. Background Art

[0002] Privacy computing, short for privacy computing, refers to a collection of technologies that enable data analysis and computation while protecting the data itself from external disclosure. Compared to traditional data usage methods, the encryption mechanisms of privacy computing can enhance data protection and reduce the risk of data leakage. Therefore, it is considered a method for "data minimization." Traditional data security methods, such as desensitization or anonymization, sacrifice some data dimensions, resulting in ineffective utilization of data information. Privacy computing, however, offers an alternative solution, ensuring that the value of data is maximized while maintaining security.

[0003] Secure Multi-Party Computation (MPC), commonly abbreviated as Secure Multi-Party Computation (MSPC), refers to the ability to perform multi-party data fusion computations while protecting data security and privacy. In a networked environment, participants in a task each possess their own data. Multiple participants utilize a communication protocol to perform distributed computations on a function to complete the computation. Each participant provides input to the function and receives the correct output. This process also protects user privacy. Participants receive their own output but cannot access any input from other users. Secure MPC enables privacy-preserving user data sharing, which is crucial for the effective use of data. In particular, information security laws impose strict user data protection requirements, making traditional direct data sharing inadequate.

[0004] Traditional secure multi-party computation (SMC) is implemented through complex interactive cryptographic protocols. Participants encrypt their input data and pass it to other participants according to the protocol. Participants then perform a series of calculations and transformations on the ciphertext to obtain the output of the original computation task. Since participants cannot directly perform computations on the original data, computational efficiency and functionality are significantly limited. SMC implemented using traditional SMC protocols involves a direct trade-off between computational functionality and efficiency. The first option supports only specific, relatively simple computational functions and does not support complex or flexible computational tasks. The second option supports general computational tasks, but due to its lower efficiency, it only supports computations on small amounts of data.

[0005] Federated Learning (FL), also known as federated machine learning, is a method of completing multi-party joint machine learning training by circulating and processing intermediate encrypted data without releasing local raw data from the database. Depending on the distribution of the data involved in the calculation among the data parties, it can be divided into horizontal federated learning, vertical federated learning, and federated transfer learning. Horizontal federated learning refers to the situation where the data of different participants have a large overlap in features, but the overlap of data samples, that is, the samples to which the features belong, is not high. Vertical federated learning refers to the situation where the data samples of different participants have a large overlap, but the overlap of sample features is not high. Currently, horizontal federated learning and vertical federated learning have been implemented in many scenarios and have been applied. At the same time, as more and more application scenarios emerge, the need for both horizontal and vertical federated learning has emerged. Based on current one-way federated learning, when faced with mixed data segmentation scenarios (data includes both horizontal and vertical segmentation, for example, A and B have the same samples but different features, and C and D have the same features but different samples), it is necessary to build multiple federated learning models in multiple stages to implement horizontal and vertical federated learning respectively to meet the needs. However, this method is relatively cumbersome, the amount of data interaction is usually large, and it requires a large amount of network and computing resources. It is not only time-consuming and labor-intensive, but also affects efficiency.

[0006] Summary of the Invention

[0007] The purpose of the present invention is to provide a hybrid federated logistic regression method based on homomorphic encryption. It can use the data of multiple institutions in the same industry or different institutions in different industries with the help of privacy-preserving distributed hybrid federated machine learning technology, effectively breaking through the bottleneck of internal and external data modeling interoperability of a single institution, and realizing the federated learning model training with the combined data of multiple parties without leaving the warehouse, so that all participants can achieve equal and mutually beneficial joint modeling, thereby improving the accuracy of the federated learning model.

[0008] The embodiment of the present invention is achieved as follows:

[0009] In a first aspect, an embodiment of the present application provides a hybrid federated logistic regression method based on homomorphic encryption, which includes the following steps:

[0010] Perform module segmentation on the data of the first data cube to obtain a first data block and a second data block, and form data data1 from the first data block and the second data cube, and form data data2 from the second data block and the third data cube;

[0011] Perform longitudinal federated logistic regression training on data data1, merge the parameters to obtain w1, intercept1 and loss1, and perform longitudinal federated logistic regression training on data data2, merge the parameters to obtain w2, intercept2 and loss2;

[0012] According to the merge parameters, perform horizontal federated logistic regression calculations on data1 and data2 respectively;

[0013] For data data1 or data data2, each participant calls the key exchange protocol, obtains the same random number r, and uses the random number r as a seed to generate a homomorphic public-private key pair;

[0014] According to the aggregation interval parameter, encrypt w1 and intercept1, and simultaneously encrypt w2 and intercept2 to obtain the encrypted parameters. The encrypted parameters are then sent to the collaborating party for secure aggregation to obtain the aggregated intermediate result parameters.

[0015] The collaborating party sends the aggregated intermediate result parameters to each participant. Each participant decrypts the aggregated intermediate result parameters and performs the next round of parameter iteration based on the decrypted aggregated intermediate result parameters until the requirements are met and the final training model is obtained.

[0016] In some embodiments of the present invention, the step of performing longitudinal federated logistic regression training on the data data1 includes:

[0017] Based on the homomorphic encryption algorithm, the first data block generates a homomorphic encryption public key, and sends the homomorphic encryption public key to the second data party, and the second data party receives the homomorphic encryption public key;

[0018] The first data block and the second data block each have initial fitting parameters, where the first data block initializes the weight wa1 and the bias intercept1, and the second data block initializes the weight wb;

[0019] The first data block calculates the dot product of the features and parameters to obtain ua1, and the second data block calculates the dot product of the features and parameters to obtain ub;

[0020] The second data block passes ub to the first data block. The first data block receives ub and uses the formula ue1 = ua1 + ub to calculate the total fitting value ue1. Based on the total fitting value ue1 and the true value, the total loss loss1 is calculated using the loss function:

[0021] The first data block calculates the gradient factor gdf based on the total fitting value ue1 and the true value, and encrypts the gradient factor gdf to obtain the encrypted gdf;

[0022] The first data block sends the encrypted gdf to the second data party, and the second data party calculates the corresponding encryption gradient gdhb based on the encrypted gdf;

[0023] Based on the homomorphic encryption algorithm, the second data party generates a random number Rb and encrypts Rb to obtain encrypted Rb. The second data party sends the sum of the encrypted gradient gdhb and the encrypted Rb to the first data block;

[0024] The first data block decrypts the sum into gdhd=gdhb+Rb and sends gdhd to the second data party;

[0025] The second data party receives gdhd, calculates the gradient gdhb according to the formula gdhb = gdhd - Rb, and updates the weight wb according to the gradient gdhb;

[0026] The first data block obtains the gradient gdha1 by multiplying the feature and gdf matrix, and updates the weight wa1 and bias intercept1 according to the gradient gdha1.

[0027] In some embodiments of the present invention, the step of calculating the gradient factor gdf based on the total fitted value ue1 and the true value of the first data block includes:

[0028] The gradient factor gdf is calculated using the formula gdf=xwT-y, where xwT represents the fitted value and y represents the true value.

[0029] In some embodiments of the present invention, the step of calculating the corresponding encrypted gradient gdhb from the encrypted gdf by the second data cube includes:

[0030] The encrypted gradient gdhb is calculated according to the formula encrypted gradient gdhb = gradient factor gdf*Xb, where Xb represents the feature column matrix.

[0031] In some embodiments of the present invention, the step of performing longitudinal federated logistic regression training on the data data2 includes:

[0032] Based on the homomorphic encryption algorithm, the second data block sends the homomorphic encryption public key to the third data party, and the third data party receives the homomorphic encryption public key;

[0033] The second data block and the third data cube each fit the initial values ​​of the parameters, where the second data block initializes the weight wa2 and bias intercept2, and the third data cube initializes the weight wc;

[0034] The second data block calculates the dot product of the feature and the parameter to obtain ua2, and the third data block calculates the dot product of the feature and the parameter to obtain uc;

[0035] The third data party transmits uc to the second data block. The second data block receives uc and calculates the total fitting value ue2 using the formula ue2=ua2+uc. Based on the total fitting value ue2 and the true value, the total loss loss2 is calculated using the loss function.

[0036] The second data block calculates the gradient factor gdf based on the total fitting value ue2 and the true value, and encrypts the gradient factor gdf to obtain the encrypted gdf;

[0037] The second data block sends the encrypted gdf to the third data party, and the third data party calculates the encryption gradient gdhc based on the encrypted gdf;

[0038] Based on the homomorphic encryption algorithm, the third data party generates a random number Rc and encrypts Rc to obtain an encrypted Rb. The third data party sends the sum of the encrypted gradient gdhc and the encrypted Rc to the second data block;

[0039] The second data block decrypts the sum of the encrypted gradient gdhc and the encrypted Rc into gdhd=gdhc+Rc, and sends gdhd to the third data party;

[0040] The third data party receives and updates the weight wc based on gdhd minus the random number Rc;

[0041] The second data block obtains the gradient gdha2 by multiplying the feature and gdf matrix, and updates the weight wa2 and bias intercept2 according to the gradient gdha2.

[0042] In a second aspect, an embodiment of the present application provides an electronic device comprising a memory for storing one or more programs and a processor. When the one or more programs are executed by the processor, the method according to any one of the first aspects is implemented.

[0043] In a third aspect, an embodiment of the present application provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the method as described in any one of the first aspects above.

[0044] Compared with the prior art, the embodiments of the present invention have at least the following advantages or beneficial effects:

[0045] The present invention proposes a hybrid federated logistic regression method based on homomorphic encryption, which includes the following steps: performing module segmentation on the data of the first data party to obtain a first data block and a second data block, and forming data data1 from the first data block and the second data party, and forming data data2 from the second data block and the third data party. Performing longitudinal federated logistic regression training on data data1, merging parameters to obtain w1, intercept1 and loss1, and performing longitudinal federated logistic regression training on data data2, merging parameters to obtain w2, intercept2 and loss2. According to the merged parameters, performing horizontal federated logistic regression calculation on data data1 and data data2 respectively. For data data1 or data data2, each participant calls a key exchange protocol to obtain the same random number r, and uses the random number r as a seed to generate a homomorphic public-private key pair. According to the aggregation interval parameter, encrypt w1 and intercept1, and encrypt w2 and intercept2 at the same time to obtain encryption parameters, and send the encryption parameters to the collaborating parties for secure aggregation to obtain aggregation intermediate result parameters. The collaborating party sends the aggregated intermediate result parameters to all participating parties. Each party decrypts the aggregated intermediate result parameters and performs the next round of parameter iteration based on the decrypted aggregated intermediate result parameters until the requirements are met, resulting in the final trained model. This completes the training process of the hybrid federated logistic regression model for the mixed data partitioning scenario. This addresses scenarios where both horizontal and vertical federated learning are required to meet requirements, while also improving computational efficiency. It simplifies the operational steps, avoids excessive data exchange, and reduces network and computing resource usage. It also overcomes the computational limitations of large-scale data scenarios, resolving the issue of local computational resources and enabling federated learning model training to be completed, enabling data modeling tasks within and outside a single organization. This enables secure multi-party computation of arbitrary computation functions, breaking the limitations of traditional machine learning, which is limited to centralized local modeling, and enabling distributed federated machine learning modeling. The current distributed hybrid federated logistic regression algorithm is primarily used in mixed data partitioning scenarios where both horizontal and vertical federated learning are required. This addresses the issue of one-way federated learning requiring multiple stages to build multiple federated learning models to implement both horizontal and vertical federated learning in mixed data partitioning scenarios. It effectively breaks through the bottleneck of data modeling interoperability within and outside a single institution, and achieves the goal of not leaving the local data warehouse, but combining data from multiple parties to train federated learning models, allowing all participants to conduct equal and mutually beneficial joint modeling to improve the accuracy of the federated learning model. BRIEF DESCRIPTION OF THE DRAWINGS

[0046] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the following briefly introduces the drawings required for use in the embodiments. It should be understood that the following drawings only illustrate certain embodiments of the present invention and therefore should not be regarded as limiting the scope. For ordinary technicians in this field, other relevant drawings can be obtained based on these drawings without paying any creative work.

[0047] FIG1 is a flowchart of a hybrid federated logistic regression method based on homomorphic encryption provided by an embodiment of the present invention;

[0048] FIG2 is a schematic diagram of data segmentation provided by an embodiment of the present invention;

[0049] FIG3 is a flowchart of an interaction between a first data block and a second data block provided by an embodiment of the present invention;

[0050] FIG4 is a flowchart of interaction between a second data block and a third data block according to an embodiment of the present invention;

[0051] FIG5 is a flowchart of a horizontal federation provided by an embodiment of the present invention;

[0052] FIG6 is a schematic structural block diagram of an electronic device provided by an embodiment of the present invention.

[0053] Icon: 101 - memory; 102 - processor; 103 - communication interface. DETAILED DESCRIPTION

[0054] To make the objectives, technical solutions, and advantages of the embodiments of the present application more clear, the technical solutions in the embodiments of the present application will be clearly and completely described below in conjunction with the accompanying drawings of the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Generally, the components of the embodiments of the present application described and shown in the drawings herein can be arranged and designed in various different configurations.

[0055] Example

[0056] Please refer to Figure 1, which shows a flow chart of a hybrid federated logistic regression method based on homomorphic encryption provided by an embodiment of the present invention. The present application embodiment provides a hybrid federated logistic regression method based on homomorphic encryption, which includes the following steps:

[0057] S110: Divide the data of the first data block into modules to obtain a first data block and a second data block, and form data data1 from the first data block and the second data block, and form data data2 from the second data block and the third data block;

[0058] For example, please refer to Figure 2, which shows a data segmentation diagram provided by an embodiment of the present invention. Taking data from parties A, B, and C as an example, party A contains a label. First, the data is segmented into modules. Data data1 consists of data blocks A1 and B of A, and data data2 consists of data blocks A2 and C of A.

[0059] S120: Perform longitudinal federated logistic regression training on data data1, merge parameters to obtain w1, intercept1, and loss1; perform longitudinal federated logistic regression training on data data2, merge parameters to obtain w2, intercept2, and loss2;

[0060] For example, longitudinal federated logistic regression training is performed on data data1 and data data2 respectively, that is, A1 and B are vertically federated, and A2 and C are vertically federated.

[0061] S130: Perform horizontal federated logistic regression calculations on data data1 and data data2 respectively according to the merge parameters;

[0062] For example, refer to Figure 5, which shows a flowchart of a horizontal federation method provided by an embodiment of the present invention. Parameters are merged: A1 and B merge their parameters to obtain w1, intercept1, and loss1. A2 and C merge their parameters to obtain w2, intercept2, and loss2. Based on the merged parameters, a horizontal federated logistic regression calculation is performed on data1 and data2.

[0063] S140: For data data1 or data data2, each participant invokes a key exchange protocol, obtains the same random number r, and uses the random number r as a seed to generate a homomorphic public-private key pair;

[0064] Specifically, in the horizontal federation part, taking Party 1, Party 2, and Arbiter as an example, each participant calls the key exchange protocol, obtains the same random number r, and uses the random number as a seed to generate a homomorphic public-private key pair.

[0065] S150: Encrypt w1 and intercept1 according to the aggregation interval parameter, and simultaneously encrypt w2 and intercept2 to obtain encryption parameters, and send the encrypted parameters to the collaborating party for secure aggregation to obtain aggregation intermediate result parameters;

[0066] Specifically, each participant encrypts [w1] and [intercept1] according to the aggregation interval parameter. Party 1 encrypts [w2] and [intercept2], and Party 2 encrypts [w2] and [intercept2], and sends the encrypted parameters to the collaborating party for secure aggregation.

[0067] S160: The collaborating party sends the aggregated intermediate result parameters to each participant. Each participant decrypts the aggregated intermediate result parameters and performs the next round of parameter iteration based on the decrypted aggregated intermediate result parameters until the requirements are met and the final training model is obtained.

[0068] Specifically, the collaborating party sends the aggregated intermediate results [w] and [intercept] to each participant. Each participant decrypts w and intercept and then performs the next round of parameter iteration based on the aggregated and decrypted parameters w and intercept. The final training model is obtained until the requirements are met.

[0069] During the implementation process described above, this method uses homomorphic encryption to further enhance data security. First, the data is partitioned into multiple vertical partitions. Then, a vertical logistic regression model is trained on the multiple vertical partitions, and a horizontal federated logistic regression model is trained on the multiple vertical data. This ultimately implements the hybrid federated logistic regression model training process for mixed data partitioning scenarios. This addresses scenarios where both horizontal and vertical federated learning are required to meet requirements, while also improving computational efficiency. It simplifies operational steps, avoids excessive data interaction, and reduces the use of network and computing resources. It also overcomes the computational limitations of real-world scenarios involving large amounts of data, resolving the issue of local computations being unable to be stored by each party. Federated learning model training can also be completed, enabling data modeling tasks within and outside a single organization. This enables secure multi-party computation of arbitrary computational functions, breaking the limitations of centralized local modeling in traditional machine learning and enabling distributed federated machine learning modeling. The current distributed hybrid federated logistic regression algorithm is primarily applicable to mixed data partitioning scenarios where both horizontal and vertical federated learning are required to meet requirements. This solves the problem of one-way federated learning requiring multiple stages of building multiple federated learning models to achieve both horizontal and vertical federated learning in mixed and fragmented data scenarios. This effectively breaks through the bottleneck of data modeling interoperability within a single institution, enabling federated learning model training with multi-party data without requiring local data to be stored. This allows for equal and mutually beneficial joint modeling across all participants, ultimately improving the accuracy of federated learning models.

[0070] For example, this method can be applied to operators and commercial banks to solve the communication + financial anti-fraud scenario to realize data hybrid segmentation scenario and provide feasibility for federated model training. The types of commercial business almost overlap, and the data dimensions are basically the same. If only the data of one bank is used, it will inevitably cause the sample distribution to be relatively single, and it will not be able to identify fraudulent behavior using other bank products well. If the user behavior data of multiple commercial banks and telecom operators are combined, the user group will be enriched to a great extent. However, the user data of each commercial bank and the communication data of the telecom operator are all private data of the user. Without the permission of the user and the regulatory agency, it is impossible to directly flow to a third party to realize data sharing. At this time, the telecom operator has a full amount of samples and some features, and other participants such as commercial banks have the same features (different from the characteristics of the operator), but each has some samples. In the face of this hybrid data segmentation scenario, if the hybrid federation idea based on homomorphic encryption proposed by the present invention is adopted, the data of all parties can be easily utilized to realize the data hybrid segmentation scenario for federated model training. Among them, each platform node authorizes each other and uploads local resources at the same time. The task creator obtains the data set resources of each participating node, locks the selected resources, and initiates the hybrid federated logistic regression algorithm calculation task. Each node collaborates to calculate and realize the calculation process of the hybrid federated logistic regression algorithm, thereby realizing the joint modeling and training task of multi-party data.

[0071] In some implementations of this embodiment, the step of performing longitudinal federated logistic regression training on the data data1 includes:

[0072] Based on the homomorphic encryption algorithm, the first data block generates a homomorphic encryption public key, and sends the homomorphic encryption public key to the second data party, and the second data party receives the homomorphic encryption public key;

[0073] The first data block and the second data block each have initial fitting parameters, where the first data block initializes the weight wa1 and the bias intercept1, and the second data block initializes the weight wb;

[0074] The first data block calculates the dot product of the features and parameters to obtain ua1, and the second data block calculates the dot product of the features and parameters to obtain ub;

[0075] The second data block passes ub to the first data block. The first data block receives ub and uses the formula ue1 = ua1 + ub to calculate the total fitting value ue1. Based on the total fitting value ue1 and the true value, the total loss loss1 is calculated using the loss function:

[0076] The first data block calculates the gradient factor gdf based on the total fitting value ue1 and the true value, and encrypts the gradient factor gdf to obtain the encrypted gdf;

[0077] The first data block sends the encrypted gdf to the second data party, and the second data party calculates the corresponding encryption gradient gdhb based on the encrypted gdf;

[0078] Based on the homomorphic encryption algorithm, the second data party generates a random number Rb and encrypts Rb to obtain encrypted Rb. The second data party sends the sum of the encrypted gradient gdhb and the encrypted Rb to the first data block;

[0079] The first data block decrypts the sum into gdhd=gdhd+Rb and sends gdhd to the second data party;

[0080] The second data party receives gdhd, calculates the gradient gdhb according to the formula gdhd = gdhd - Rb, and updates the weight wb according to the gradient gdhb;

[0081] The first data block obtains the gradient gdha1 by multiplying the feature and gdf matrix, and updates the weight wa1 and bias intercept1 according to the gradient gdha1.

[0082] For example, please refer to Figure 3, which shows an interaction flow chart of a first data block and a second data party provided by an embodiment of the present invention. The specific process of A1 and B performing vertical federation is as follows: Party A1 sends the homomorphic encryption public key to Party B. Party A1 and Party B each fit the initial values ​​of the parameters, where Party A1 initializes the weight wa1 and the bias intercept1; Party B initializes the weight wb. A1 and B each calculate the dot product result of the features and parameters. Party B passes the dot product result ub of the features and parameters to Party A1, and Party A1 calculates the total fitting value ue 1 of A 1 and B. The total loss is calculated using the loss function based on the total fitting value and the true value: A1 calculates the gradient factor based on both the total fitted value and the true value: gdf = xwT - y = y` - y. A1 encrypts [gdf] and sends it to B. B calculates its encrypted gradient [gdhb] based on the encrypted [gdf]. Gradient = (feature column) matrix multiplication (gradient factor). B generates a random number and encrypts it [Rb]. B then sums the encrypted gradient and the encrypted random number [gdhb] + [Rb] and sends it to A1. A1 decrypts the sum of the encrypted gradient and the encrypted random number to gdhd = gdhb + Rb. A1 sends gdhd to B. A1 multiplies the features by the gdf matrix to obtain A1's gradient gdha1. A1 updates A1's weights wa1 and intercept1 based on gdha1. B updates B's weight wb based on gdhd minus the added random number Rb, thus completing one round of vertical federated parameter update.

[0083] In some implementations of this embodiment, the step of calculating the gradient factor gdf based on the total fitting value ue1 and the true value of the first data block includes:

[0084] The gradient factor gdf is calculated using the formula gdf=xwT-y, where xwT represents the fitted value and y represents the true value.

[0085] In some implementations of this embodiment, the step of calculating the corresponding encrypted gradient gdhb by the second data cube according to the encrypted gdf includes:

[0086] The encrypted gradient gdhb is calculated according to the formula encrypted gradient gdhb = gradient factor gdf*Xb, where Xb represents the feature column matrix.

[0087] In some implementations of this embodiment, the step of performing longitudinal federated logistic regression training on the data data2 includes:

[0088] Based on the homomorphic encryption algorithm, the second data block sends the homomorphic encryption public key to the third data party, and the third data party receives the homomorphic encryption public key;

[0089] The second data block and the third data cube each fit the initial values ​​of the parameters, where the second data block initializes the weight wa2 and bias intercept2, and the third data cube initializes the weight wc;

[0090] The second data block calculates the dot product of the feature and the parameter to obtain ua2, and the third data block calculates the dot product of the feature and the parameter to obtain uc;

[0091] The third data party transmits uc to the second data block. The second data block receives uc and calculates the total fitting value ue2 using the formula ue2=ua2+uc. Based on the total fitting value ue2 and the true value, the total loss loss2 is calculated using the loss function.

[0092] The second data block calculates the gradient factor gdf based on the total fitting value ue2 and the true value, and encrypts the gradient factor gdf to obtain the encrypted gdf;

[0093] The second data block sends the encrypted gdf to the third data party, and the third data party calculates the encryption gradient gdhc based on the encrypted gdf;

[0094] Based on the homomorphic encryption algorithm, the third data party generates a random number Rc and encrypts Rc to obtain an encrypted Rb. The third data party sends the sum of the encrypted gradient gdhc and the encrypted Rc to the second data block;

[0095] The second data block decrypts the sum of the encrypted gradient gdhc and the encrypted Rc into gdhd=gdhc+Rc, and sends gdhd to the third data party;

[0096] The third data party receives and updates the weight wc based on gdhd minus the random number Rc;

[0097] The second data block obtains the gradient gdha2 by multiplying the feature and gdf matrix, and updates the weight wa2 and bias intercept2 according to the gradient gdha2.

[0098] For example, please refer to Figure 4, which shows a flow chart of the interaction between a second data block and a third data party provided by an embodiment of the present invention. The data interaction process between A2 and C is consistent with the data interaction process between A1 and B. The process of vertical federation between A2 and C is as follows: Party A2 sends the homomorphic encryption public key to Party C. Party A2 and C each fit the initial values ​​of the parameters, where Party A2 initializes the weight wa2 and the bias intercept2; Party C initializes the weight wc. A2 and C each calculate the dot product of the features and parameters. Party A2 passes the dot product result uc of the features and parameters to Party A2, and Party A2 calculates the total fitting value ue2 of A2 and C. The total loss is calculated using the loss function based on the total fitting value and the true value: Party A2 calculates the gradient factor based on both the total fitted value and the true value: gdf = xwT - y = y` - y. Party A2 encrypts [gdf] and sends it to Party C. Party C calculates its encrypted gradient [gdhc] ​​based on the encrypted [gdf]. Gradient = (feature column) matrix multiplication (gradient factor). Party C generates a random number and encrypts it [Rc]. Party B then sums the encrypted gradient and the encrypted random number [gdhc] ​​+ [Rc] and sends it to Party A2. Party A2 decrypts the sum of the encrypted gradient and the encrypted random number to gdhd = gdhc + Rc. Party A2 sends gdhd to Party C. Party A2 multiplies the features by the gdf matrix to obtain its gradient gdha2. Party A2 updates its weights wa2 and intercept2 based on gdha2. Party C updates its weight wc based on gdhd minus the added random number Rc, thus completing one round of vertical federated parameter update.

[0099] Please refer to Figure 6, which is a schematic structural block diagram of an electronic device provided in an embodiment of the present application. The electronic device includes a memory 101, a processor 102 and a communication interface 103. The memory 101, the processor 102 and the communication interface 103 are electrically connected to each other directly or indirectly to realize data transmission or interaction. For example, these components can be electrically connected to each other through one or more communication buses or signal lines. The memory 101 can be used to store software programs and modules, and the processor 102 executes various functional applications and data processing by executing the software programs and modules stored in the memory 101. The communication interface 103 can be used to communicate signaling or data with other node devices.

[0100] Among them, the memory 101 can be, but is not limited to, random access memory (RAM), read only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), etc.

[0101] The processor 102 may be an integrated circuit chip with signal processing capabilities. The processor 102 may be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it may also be a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components.

[0102] It is understood that the structure shown in Figure 6 is merely illustrative, and the electronic device may include more or fewer components than shown in Figure 6, or have a configuration different from that shown in Figure 6. Each component shown in Figure 6 may be implemented using hardware, software, or a combination thereof.

[0103] In the embodiments provided in this application, it should be understood that the disclosed devices and methods can also be implemented in other ways. The device embodiments described above are merely schematic. For example, the flowcharts and block diagrams in the accompanying drawings show the possible architectures, functions and operations of the devices, methods and computer program products according to multiple embodiments of the present application. In this regard, each box in the flowchart or block diagram can represent a module, a program segment or a part of the code, and the module, program segment or a part of the code contains one or more executable instructions for implementing the specified logical functions. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in an order different from that marked in the accompanying drawings. For example, two consecutive boxes can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram and / or flowchart, and the combination of boxes in the block diagram and / or flowchart, can be implemented using a dedicated hardware-based system that performs the specified function or action, or can be implemented using a combination of dedicated hardware and computer instructions.

[0104] In addition, the functional modules in each embodiment of the present application can be integrated together to form an independent part, or each module can exist independently, or two or more modules can be integrated to form an independent part.

[0105] If the functions are implemented in the form of software function modules and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art or the part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes several instructions for enabling a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.

[0106] It will be apparent to those skilled in the art that the present application is not limited to the details of the exemplary embodiments described above and that the present application can be implemented in other specific forms without departing from the spirit or essential characteristics of the present application. Therefore, the embodiments should be considered in all respects as illustrative and non-restrictive, and the scope of the present application is defined by the appended claims, not the foregoing description, and all variations within the meaning and range of equivalents of the claims are intended to be included therein. Any reference sign in a claim should not be construed as limiting the claim to which it relates.

Claims

1. A hybrid federated logistic regression method based on homomorphic encryption, characterized in that: The steps include: Perform module segmentation on the data of the first data cube to obtain a first data block and a second data block, and form data data1 with the first data block and the second data cube, and form data data2 with the second data block and the third data cube; Performing longitudinal federated logistic regression training on the data data1, merging parameters to obtain w1, intercept1, and loss1; performing longitudinal federated logistic regression training on the data data2, merging parameters to obtain w2, intercept2, and loss2; According to the merge parameters, perform horizontal federated logistic regression calculations on data1 and data2 respectively; For data data1 or data data2, each participant calls the key exchange protocol, obtains the same random number r, and uses the random number r as a seed to generate a homomorphic public-private key pair; According to the aggregation interval parameter, encrypt w1 and intercept1, and simultaneously encrypt w2 and intercept2 to obtain encryption parameters, and send the encryption parameters to the collaborating party for secure aggregation to obtain aggregation intermediate result parameters; The collaborating party sends the aggregated intermediate result parameters to each participating party, and each participating party decrypts the aggregated intermediate result parameters and performs the next round of parameter iteration based on the decrypted aggregated intermediate result parameters until the requirements are met and the final training model is obtained.

2. The hybrid federated logistic regression method based on homomorphic encryption according to claim 1 is characterized in that: The steps of performing longitudinal federated logistic regression training on the data data1 include: Based on a homomorphic encryption algorithm, the first data block generates a homomorphic encryption public key, and sends the homomorphic encryption public key to the second data party, and the second data party receives the homomorphic encryption public key; The first data block and the second data block each have initial fitting parameters, wherein the first data block initializes the weight wa1 and the bias intercept1, and the second data block initializes the weight wb; The first data block calculates the dot product of the feature and the parameter to obtain ua1, and the second data block calculates the dot product of the feature and the parameter to obtain ub; The second data block transmits ub to the first data block, and the first data block receives ub, calculates the total fitting value ue1 using the formula ue1=ua1+ub, and calculates the total loss loss1 based on the total fitting value ue1 and the true value using the loss function: The first data block calculates a gradient factor gdf based on the total fitting value ue1 and the true value, and encrypts the gradient factor gdf to obtain an encrypted gdf; The first data block sends the encrypted gdf to the second data party, and the second data party calculates the corresponding encryption gradient gdhb according to the encrypted gdf; Based on the homomorphic encryption algorithm, the second data party generates a random number Rb and encrypts Rb to obtain encrypted Rb, and the second data party sends the sum of the encrypted gradient gdhb and the encrypted Rb to the first data block; The first data block decrypts the sum into gdhd=gdhb+Rb, and sends gdhd to the second data party; The second data processor receives gdhd, calculates a gradient gdhb according to the formula gdhb=gdhd-Rb, and updates a weight wb according to the gradient gdhb; The first data block obtains the gradient gdha1 by multiplying the feature and the gdf matrix, and updates the weight wa1 and bias intercept1 according to the gradient gdha1.

3. The hybrid federated logistic regression method based on homomorphic encryption according to claim 2 is characterized in that: The step of calculating the gradient factor gdf based on the total fitting value ue1 and the true value of the first data block includes: The gradient factor gdf is calculated using the formula gdf=xwT-y, where xwT represents the fitted value and y represents the true value.

4. The hybrid federated logistic regression method based on homomorphic encryption according to claim 2 is characterized in that: The step of the second data entity calculating the corresponding encrypted gradient gdhb according to the encrypted gdf includes: The encrypted gradient gdhb is calculated according to the formula encrypted gradient gdhb = gradient factor gdf*Xb, where Xb represents the feature column matrix.

5. The hybrid federated logistic regression method based on homomorphic encryption according to claim 1 is characterized in that: The steps of performing longitudinal federated logistic regression training on the data data2 include: Based on the homomorphic encryption algorithm, the second data block sends the homomorphic encryption public key to the third data party, and the third data party receives the homomorphic encryption public key; The second data block and the third data block each have initial fitting parameters, wherein the second data block initializes the weight wa2 and the bias intercept2, and the third data block initializes the weight wc; The second data block calculates the dot product of the feature and the parameter to obtain ua2, and the third data block calculates the dot product of the feature and the parameter to obtain uc; The third data block transmits uc to the second data block, the second data block receives uc, calculates the total fitting value ue2 using the formula ue2=ua2+uc, and calculates the total loss loss2 based on the total fitting value ue2 and the true value using the loss function; The second data block calculates a gradient factor gdf based on the total fitting value ue2 and the true value, and encrypts the gradient factor gdf to obtain an encrypted gdf; The second data block sends the encrypted gdf to the third data party, and the third data party calculates the encryption gradient gdhc according to the encrypted gdf; Based on the homomorphic encryption algorithm, the third data party generates a random number Rc and encrypts Rc to obtain an encrypted Rb, and the third data party sends the sum of the encrypted gradient gdhc and the encrypted Rc to the second data block; The second data block decrypts the sum of the encrypted gradient gdhc and the encrypted Rc into gdhd=gdhc-Rc, and sends gdhd to the third data party; The third data party receives and updates the weight wc according to gdhd minus the random number Rc; The second data block obtains the gradient gdha2 by multiplying the feature and the gdf matrix, and updates the weight wa2 and the bias intercept2 according to the gradient gdha2.

6. An electronic device, characterized in that: include: a memory for storing one or more programs; processor; When the one or more programs are executed by the processor, the method according to any one of claims 1 to 5 is implemented.

7. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the method according to any one of claims 1 to 5 is implemented.

Citation Information

Patent Citations

  • Business data processing method, device and apparatus in federated learning

    CN112200713A

  • Longitudinal federated learning linear regression and logistic regression model training method and device

    CN113505894A

  • Longitudinal federated logistic regression training method and device

    CN115983407A

  • Hybrid federated logistic regression method based on homomorphic encryption

    CN116776293A

  • Training method and apparatus for a distributed machine learning model, device and medium

    US20230078726A1

Cited By

  • Federal analysis method for non-independent gene data

    CN119741976A

  • Privacy protection longitudinal federated learning method and system based on function encryption

    CN119830334A

  • Robust privacy federal learning method based on homomorphic evaluation

    CN119940481A

  • Data processing system of medical model federal learning framework

    CN120633889A