Longitudinal federated learning framework based on secret sharing and public key encryption

Abstract

The invention discloses a longitudinal federated learning framework based on secret sharing and public key encryption, and belongs to the technical field of federated learning. The longitudinal federated learning framework is applied to federated training and contribution degree calculation, and participating entities are an aggregation server and two or more participants. According to the method, privacy data of the aggregation server and the participants are protected through secret sharing, and public key encryption is used on a transmission channel to enhance privacy protection.

Description

Technical Field

[0001] The present invention designs a vertical federated learning framework, specifically a vertical federated learning framework based on secret sharing and public key encryption. Background Art

[0002] Vertical federated learning is a distributed machine learning framework suitable for scenarios where different participants possess distinct feature spaces but share the same user set. In this scenario, each participant's dataset differs in the feature dimension but is identical in the sample dimension. Vertical federated learning collaboratively computes gradients and loss functions on each participant's features, enabling the final model to leverage the complementarity of each participant's features and improve model performance.

[0003] Vertical federated learning has broad application prospects in multiple fields, particularly in finance, healthcare, and the internet. For example, in finance, different banks possess data on the same customers with different financial attributes. Through vertical federated learning, they can jointly train a credit assessment model, improving the accuracy of credit assessments. In healthcare, different hospitals possess different types of medical data (such as genetic data and medical records) on the same patients. Through vertical federated learning, a comprehensive disease prediction model can be constructed to improve the quality of medical services. In the internet sector, different internet companies possess behavioral data on the same users across different platforms. Through vertical federated learning, more accurate user profiling and recommendation algorithms can be developed.

[0004] Current vertical federated learning approaches primarily rely on technologies such as secure multi-party computation and homomorphic encryption to ensure data privacy. However, these technologies face numerous challenges and limitations in practical applications. First, secure multi-party computation and homomorphic encryption typically introduce high computational overhead, significantly slowing model training. Second, to ensure privacy, vertical federated learning requires frequent exchange of encrypted data between participating parties, which incurs significant communication overhead. Furthermore, privacy protection measures may compromise model accuracy, particularly in situations where data distribution is uneven or data quality is poor. Summary of the Invention

[0005] The purpose of this invention is to provide a more secure vertical federated learning framework to address the above technical problems.

[0006] The technical solutions of the present invention are as follows:

[0007] (1) The aggregation server and each participant perform the training task of vertical federated learning based on the communication framework;

[0008] (2) The aggregation server and each participant calculate the contribution of each participant based on the communication framework.

[0009] Furthermore, the training task of the vertical federated learning described in step (1) includes the following steps:

[0010] (1.1) Initialization: The aggregator and other participants each have their own datasets. The sample spaces are aligned under a private intersection, and the aggregator uniquely possesses the true labels. The aggregator and other participants each generate a public-private key pair, make the public key public, and establish a routing table between the data sender and the recipient.

[0011] (1.2) Each participant i uses local data to obtain its own partial prediction value P according to the specified task i ;

[0012] (1.3) Each participant transmits part of the predicted value to the aggregation server based on the communication transmission framework;

[0013] (1.4) The aggregation server decrypts the received partial prediction values, performs aggregation training, updates the model parameters of the aggregation server, and sends the model error to each participant based on the communication framework;

[0014] (1.5) After receiving the error sent by the aggregation server, each participant performs gradient descent based on the error and updates the local model.

[0015] Furthermore, the calculation of the contribution of each participant in step (2) includes the following steps:

[0016] (2.1) The aggregation server uses a random number r to randomly mask the aggregate value Z: Z = Z + r, and sends it to each participant;

[0017] (2.2) Calculation for each participant i: Average value Distance Weight m is the number of participants;

[0018] (2.3) Each participant transmits the calculation result dist based on the communication transmission framework i Transmit to the aggregation server;

[0019] (2.4) The aggregation server calculates the contribution of each participant:

[0020] Furthermore, each participant in step (1.1) establishes a routing table, including the following steps:

[0021] (1.1.1) When the number of participants is 2, the participants are divided into two categories: first hop and last hop; when the number of participants is greater than 2, the participants are divided into three categories: first hop, middle hop, and last hop;

[0022] (1.1.2) Each participant establishes a routing table between the participants: if there is an intermediate hop participant, the next hop of the first hop participant is the intermediate hop participant, otherwise the next hop is the last hop participant; the next hop of the intermediate hop participant is the intermediate hop participant, and the next hop of the last intermediate hop participant is the last hop participant.

[0023] Furthermore, the communication framework described in step (1.3) includes the following steps:

[0024] (1.3.1) The first-hop participant slices the predicted value P1. Slice a Send to the aggregation server, and send another slice Send to the next hop;

[0025] (1.3.2) If there is an intermediate hop server, the intermediate hop participant receives the slice of the previous hop participant Overlay the slice with its own partial predictions: Slice the superimposed result. Slice a Send to the aggregation server, and send another slice Send to the next hop participant. Where k is the intermediate hop server;

[0026] (1.3.3) If there is an intermediate hop participant, the previous hop of the tail hop participant is the intermediate hop participant, otherwise, it is the first hop participant. The tail hop participant receives the slice of the previous hop participant, superimposes the slice with its own partial prediction value, and slices the superimposed result. The tail jump participant will slice two and All are sent to the aggregation server;

[0027] (1.3.4) In all communication processes, the sender encrypts the data using the receiver's public key. After receiving the data, the receiver decrypts the data using its own private key.

[0028] The vertical federated learning framework of the present invention has the following advantages:

[0029] (1) The present invention is based on secret sharing and public key encryption to protect the privacy of data.

[0030] (2) The vertical federated learning communication framework of the present invention can be applied to the training process of vertical federated learning. The aggregation server and each participant perform the training task locally and transmit the model to the aggregation server based on the communication framework.

[0031] (3) The vertical federated learning communication framework of the present invention can be applied to the calculation process of the contribution of each participant. Based on the communication framework, each participant assists the aggregation server in calculating the contribution of each participant.

[0032] (4) The vertical federated learning framework of the present invention is based on secret sharing technology and combines public key encryption schemes to protect the privacy data of the aggregation server and participants. The vertical federated learning communication framework of the present invention can be applied to the training process of vertical federated learning and the calculation process of the contribution of each participant. BRIEF DESCRIPTION OF THE DRAWINGS

[0033] Figure 1 This is a diagram of the vertical federated learning communication framework of the present invention;

[0034] Figure 2 This is the flowchart of the vertical federated learning training of the present invention;

[0035] Figure 3 This is a flowchart of the vertical federated learning contribution calculation process of the present invention. DETAILED DESCRIPTION

[0036] In order to clearly illustrate the vertical federated learning framework based on secret sharing and public key encryption of the present invention, the present invention is further described below with reference to examples and drawings, but this should not limit the scope of protection of the present invention.

[0037] See also Figure 1 、 Figure 2 and Figure 3 This embodiment discloses a vertical federated learning framework based on secret sharing and public key encryption, and the method includes the following steps:

[0038] (1) The aggregation server and each participant perform the training task of vertical federated learning based on the communication framework;

[0039] (2) The aggregation server and each participant calculate the contribution of each participant based on the communication framework.

[0040] The proposed vertical federated learning framework is based on secret sharing and incorporates public key encryption during communication to protect the privacy of the aggregation server and each participant. This invention can be applied to both training and contribution calculation in vertical federated learning.

[0041] Specifically, the training task of the vertical federated learning described in step (1) is as follows: Figure 2 As shown, the following steps are included:

[0042] (1.1) Initialization: The aggregator and other participants each have their own datasets. The sample spaces are aligned under a private intersection, and the aggregator uniquely possesses the true labels. The aggregator and other participants each generate a public-private key pair, make the public key public, and establish a routing table between the data sender and the recipient.

[0043] (1.2) Each participant i uses local data to obtain its own partial prediction value P according to the specified task i ;

[0044] (1.3) Each participant transmits part of the predicted value to the aggregation server based on the communication transmission framework;

[0045] (1.4) The aggregation server decrypts the received partial prediction values, performs aggregation training, updates the model parameters of the aggregation server, and sends the model error to each participant based on the communication framework;

[0046] (1.5) After receiving the error sent by the aggregation server, each participant performs gradient descent based on the error and updates the local model.

[0047] Specifically, the contribution of each participant in step (2) is calculated as follows: Figure 3 As shown, the following steps are included:

[0048] (2.1) The aggregation server uses a random number r to randomly mask the aggregate value Z: Z = Z + r, and sends it to each participant;

[0049] (2.2) Calculation for each participant i: Average value Distance Weight m is the number of participants;

[0050] (2.3) Each participant transmits the calculation result dist based on the communication transmission framework i Transmit to the aggregation server;

[0051] (2.4) The aggregation server calculates the contribution of each participant:

[0052] Specifically, each participant in step (1.1) establishes a routing table, including the following steps:

[0053] (1.1.1) When the number of participants is 2, the participants are divided into two categories: first hop and last hop; when the number of participants is greater than 2, the participants are divided into three categories: first hop, middle hop, and last hop;

[0054] (1.1.2) Each participant establishes a routing table between the participants: if there is an intermediate hop participant, the next hop of the first hop participant is the intermediate hop participant, otherwise the next hop is the last hop participant; the next hop of the intermediate hop participant is the intermediate hop participant, and the next hop of the last intermediate hop participant is the last hop participant.

[0055] Specifically, the communication framework described in step (1.3) is as follows: Figure 1 As shown, the following steps are included:

[0056] (1.3.1) The first-hop participant slices the predicted value P1. Slice a Send to the aggregation server, and send another slice Send to the next hop;

[0057] (1.3.2) If there is an intermediate hop server, the intermediate hop participant receives the slice of the previous hop participant Overlay the slice with its own partial predictions: Slice the superimposed result. Slice a Send to the aggregation server, and send another slice Send to the next hop participant. Where k is the intermediate hop server;

[0058] (1.3.3) If there is an intermediate hop participant, the previous hop of the tail hop participant is the intermediate hop participant, otherwise, it is the first hop participant. The tail hop participant receives the slice of the previous hop participant, superimposes the slice with its own partial prediction value, and slices the superimposed result. The tail jump participant will slice two and All are sent to the aggregation server;

[0059] (1.3.4) In all communication processes, the sender encrypts the data using the receiver's public key. After receiving the data, the receiver decrypts the data using its own private key.

[0060] This invention uses secret sharing to transmit users' private data. The first-hop participant slices the model and sends it to the aggregation server and the next-hop participant. Other participants receive the data from the previous-hop participant, aggregate it with their own model, and then slice it again. The two slices are sent to the aggregation server and the next-hop participant, respectively. The last-hop participant sends both slices to the aggregation server. Through this secret sharing scheme, combined with public key encryption during the transmission process, neither the aggregation server nor any participant can obtain the private data of other participants.

[0061] The above embodiments are preferred implementation modes of the present invention, but the implementation modes of the present invention are not limited to the above embodiments. Any other changes, modifications, substitutions, combinations, and simplifications that do not deviate from the spirit and principles of the present invention should be considered as equivalent replacement methods and are included in the scope of protection of the present invention.

Claims

1. A vertical federated learning framework based on secret sharing and public key encryption, characterized in that: The method comprises the following steps: (1) The aggregation server and each participant perform the training task of vertical federated learning based on the communication framework; (2) The aggregation server and each participant calculate the contribution of each participant based on the communication framework.

2. A vertical federated learning framework based on secret sharing and public key encryption according to claim 1, characterized in that: The training task of the vertical federated learning described in step (1) includes the following steps: (1.1) Initialization: The aggregator and other participants have their own data sets. The sample spaces are aligned under the privacy intersection. The aggregator is the only one that has the real labels. The aggregator and other participants each generate a public-private key pair, make the public key public, and establish a routing table between the data sender and the recipient. (1.2) Each participant i uses local data to obtain its own partial prediction value P according to the specified task i ; (1.3) Each participant transmits part of the predicted value to the aggregation server based on the communication transmission framework; (1.4) The aggregation server decrypts the received partial prediction values, performs aggregation training, updates the model parameters of the aggregation server, and sends the model error to each participant based on the communication framework; (1.5) After each participant receives the error sent by the aggregation server, it performs gradient descent based on the error and updates the local model.

3. A vertical federated learning framework based on secret sharing and public key encryption according to claim 1, characterized in that: The calculation of the contribution of each participant in step (2) includes the following steps: (2.1) The aggregation server uses a random number r to randomly mask the aggregation value Z: Z = Z + r, and sends it to each participant; (2.2) Calculation for each participant i: Average value Distance Weight m is the number of participants; (2.3) Each participant transmits the calculation result dist based on the communication transmission framework i Transmit to the aggregation server; (2.4) The aggregation server calculates the contribution of each participant:

4. A vertical federated learning framework based on secret sharing and public key encryption according to claim 2, characterized in that: Each participant in step (1.1) establishes a routing table, including the following steps: (1.1.1) When the number of participants is 2, the participants are divided into two categories: first hop and last hop; when the number of participants is greater than 2, the participants are divided into three categories: first hop, middle hop and last hop; (1.1.2) Each participant establishes a routing table between the participants: if there is an intermediate hop participant, the next hop of the first hop participant is the intermediate hop participant, otherwise the next hop is the last hop participant; The next hop of the intermediate hop participant is the intermediate hop participant, and the next hop of the last intermediate hop participant is the last hop participant.

5. A vertical federated learning framework based on secret sharing and public key encryption according to claim 2, characterized in that: The communication framework described in step (1.3) includes the following steps: (1.3.1) The first-hop participant slices the partial prediction value P1, P1 = P1 1 +P1 2 , a slice P1 1 Send it to the aggregation server and send another slice P1 2 Send to the next hop; (1.3.2) If there is an intermediate hop server, the intermediate hop participant receives the slice of the previous hop participant Overlay a slice with its own partial predictions: Slice the superimposed result. Slice a Send to the aggregation server, and send another slice Send to the next hop participant. Where k is the intermediate hop server; (1.3.3) If there is an intermediate hop participant, the previous hop of the tail hop participant is the intermediate hop participant, otherwise, it is the first hop participant. The tail hop participant receives the slice of the previous hop participant, superimposes the slice with its own partial prediction value, and slices the superimposed result. The tail jump participant will slice two and All are sent to the aggregation server; (1.3.4) In all communication processes, the sender uses the receiver's public key to encrypt the data, and after the receiver receives the data, he uses his own private key to decrypt the data.

Images