Longitudinal federal learning framework based on secret sharing

Abstract

The invention discloses a longitudinal federated learning framework based on secret sharing, and belongs to the technical field of federated learning. The longitudinal federated learning framework is applied to federated training and contribution degree calculation, and participating entities are an aggregation server and two or more participants. The privacy data of the aggregation server and the participants are protected through secret sharing.

Description

Technical Field

[0001] The present invention designs a vertical federated learning framework, specifically a vertical federated learning framework based on secret sharing. Background Art

[0002] Vertical federated learning is a distributed machine learning framework that is suitable for scenarios where different participants have different feature spaces but share the same user set. In this case, the datasets of each participant are different in the feature dimension but the same in the sample dimension. Vertical federated learning collaboratively calculates the gradients and loss functions on each feature, so that the final model can take advantage of the complementarity of the features of each party and improve the performance of the model.

[0003] Vertical federated learning has broad application prospects in many fields, especially in finance, medicine and the Internet. For example, in the financial field, different banks have different financial attribute data of the same customers. Through vertical federated learning, they can jointly train a credit assessment model to improve the accuracy of credit assessment. In the medical field, different hospitals have different types of medical data (such as genetic data, medical records, etc.) of the same patients. Through vertical federated learning, a comprehensive disease prediction model can be built to improve the quality of medical services. In the Internet field, different Internet companies have the behavioral data of the same user on different platforms. Through vertical federated learning, more accurate user portraits and recommendation algorithms can be performed.

[0004] Current vertical federated learning methods mainly rely on technologies such as secure multi-party computation and homomorphic encryption to ensure data privacy. However, these technologies face many challenges and limitations in practical applications. First, secure multi-party computation and homomorphic encryption usually introduce high computational overhead, resulting in a significant decrease in model training speed. Second, in order to ensure privacy protection, vertical federated learning requires frequent exchange of encrypted data between participants, which will incur huge communication overhead. In addition, privacy protection measures may affect the accuracy of the model, especially when the data distribution is uneven or the data quality is poor. Summary of the invention

[0005] The purpose of the present invention is to provide a more secure vertical federated learning framework in response to the above technical problems.

[0006] The technical solution of the present invention is as follows:

[0007] (1) The aggregation server and each participant perform the training task of vertical federated learning based on the communication framework;

[0008] (2) The aggregation server and each participant calculate the contribution of each participant based on the communication framework.

[0009] Furthermore, the training task of the vertical federated learning described in step (1) includes the following steps:

[0010] (1.1) Initialization: The aggregator and other participants have their own data sets. The sample spaces are aligned under the privacy intersection. The aggregator is the only one that has the real labels. The aggregator and other participants each generate a public-private key pair, make the public key public, and establish a routing table between the data sender and the recipient.

[0011] (1.2) Each participant i uses local data to obtain its own partial prediction value P according to the specified task i ;

[0012] (1.3) Each participant transmits part of the predicted value to the aggregation server based on the communication transmission framework;

[0013] (1.4) The aggregation server decrypts the received partial prediction values, performs aggregation training, and updates the model parameters of the aggregation server;

[0014] (1.5) The aggregation server and each participant transmit the model error of the aggregation server based on the communication transmission framework;

[0015] (1.6) After each participant receives the error sent by the aggregation server, it performs gradient descent based on the error and updates the local model.

[0016] Furthermore, the calculation of the contribution of each participant in step (2) includes the following steps:

[0017] (2.1) The aggregation server uses a random number r to randomly mask the aggregation value Z: Z = Z + r, and transmits it to each participant based on the communication transmission framework;

[0018] (2.2) Calculation for each participant i: Average value Distance Weight m is the number of participants;

[0019] (2.3) Each participant transmits the calculation result dist based on the communication transmission framework i Transmit to the aggregation server;

[0020] (2.4) The aggregation server calculates the contribution of each participant:

[0021] Furthermore, each participant in step (1.1) establishes a routing table, including the following steps:

[0022] (1.1.1) When the number of participants is 2, the participants are divided into two categories: first hop and last hop; when the number of participants is greater than 2, the participants are divided into three categories: first hop, middle hop and last hop;

[0023] (1.1.2) Each participant establishes a routing table between the participants: if there is an intermediate hop participant, the next hop of the first hop participant is the intermediate hop participant, otherwise the next hop is the last hop participant; the next hop of the intermediate hop participant is the intermediate hop participant, and the next hop of the last intermediate hop participant is the last hop participant.

[0024] Furthermore, the communication framework described in step (1.3) includes the following steps:

[0025] (1.3.1) The first-hop participant slices the partial prediction value P1 Slice a Send to the aggregation server, and send another slice Send to the next hop;

[0026] (1.3.2) If there is an intermediate hop server, the intermediate hop participant receives the slice of the previous hop participant Overlay a slice with its own partial predictions: Slice the superimposed result. Slice a Send to the aggregation server, and send another slice Send to the next hop participant. Where k is the intermediate hop server;

[0027] (1.3.3) If there is an intermediate hop participant, the previous hop of the tail hop participant is the intermediate hop participant, otherwise, it is the first hop participant. The tail hop participant receives the slice of the previous hop participant, superimposes the slice with its own partial prediction value, and slices the superimposed result. The tail jump participant will slice two and All are sent to the aggregation server.

[0028] Furthermore, the communication framework described in step (1.5) includes the following steps:

[0029] (1.5.1) The aggregation server slices the model gradient, sends slice 1 to the first-hop participant, and sends slice 2 to the tail-hop server;

[0030] (1.5.2) Slice 1 is transmitted from the first-hop participant to the last-hop participant, and slice 2 is transmitted from the last-hop participant to the first-hop participant;

[0031] (1.5.3) Each participant superimposes the two slices to obtain the model slice of the aggregation server.

[0032] The vertical federated learning framework of the present invention has the following advantages:

[0033] (1) The present invention is based on secret sharing to protect the privacy of data.

[0034] (2) The vertical federated learning communication framework of the present invention can be applied to the training process of vertical federated learning. The aggregation server and each participant performs the training task locally and transmits the model to the aggregation server based on the communication framework.

[0035] (3) The vertical federated learning communication framework of the present invention can be applied to the calculation process of the contribution of each participant. Based on the communication framework, each participant assists the aggregation server in calculating the contribution of each participant.

[0036] (4) The vertical federated learning framework of the present invention is based on secret sharing technology to protect the privacy data of the aggregation server and the participants. The vertical federated learning communication framework of the present invention can be applied to the training process of vertical federated learning and the calculation process of the contribution of each participant. BRIEF DESCRIPTION OF THE DRAWINGS

[0037] Figure 1 A communication framework diagram for transmission between parties of vertical federated learning of the present invention;

[0038] Figure 2 This is a flowchart of the vertical federated learning training of the present invention;

[0039] Figure 3 This is a flow chart of the vertical federated learning contribution calculation process of the present invention. DETAILED DESCRIPTION

[0040] In order to clearly illustrate a vertical federated learning framework based on secret sharing of the present invention, the present invention is further described below with reference to examples and drawings, but this should not limit the protection scope of the present invention.

[0041] See also Figure 1 , Figure 2 and Figure 3 , this embodiment discloses a vertical federated learning framework based on secret sharing, including the following steps:

[0042] (1) The aggregation server and each participant perform the training task of vertical federated learning based on the communication framework;

[0043] (2) The aggregation server and each participant calculate the contribution of each participant based on the communication framework.

[0044] The vertical federated learning framework of the present invention is based on secret sharing to protect the privacy data of the aggregation server and each participant. The invention can be applied to the training and contribution calculation of vertical federated learning.

[0045] Specifically, the training task of the vertical federated learning described in step (1) is as follows: Figure 2 As shown, the following steps are included:

[0046] (1.1) Initialization: The aggregator and other participants have their own data sets. The sample spaces are aligned under the privacy intersection. The aggregator is the only one that has the real labels. The aggregator and other participants each generate a public-private key pair, make the public key public, and establish a routing table between the data sender and the recipient.

[0047] (1.2) Each participant i uses local data to obtain its own partial prediction value P according to the specified task i ;

[0048] (1.3) Each participant transmits part of the predicted value to the aggregation server based on the communication transmission framework;

[0049] (1.4) The aggregation server decrypts the received partial prediction values, performs aggregation training, and updates the model parameters of the aggregation server;

[0050] (1.5) The aggregation server and each participant transmit the model error of the aggregation server based on the communication transmission framework;

[0051] (1.6) After each participant receives the error sent by the aggregation server, it performs gradient descent based on the error and updates the local model.

[0052] Specifically, the contribution of each participant in step (2) is calculated as follows: Figure 3 As shown, the following steps are included:

[0053] (2.1) The aggregation server uses a random number r to randomly mask the aggregation value Z: Z = Z + r, and transmits it to each participant based on the communication transmission framework;

[0054] (2.2) Calculation for each participant i: Average value Distance Weight m is the number of participants;

[0055] (2.3) Each participant transmits the calculation result dist based on the communication transmission framework i Transmit to the aggregation server;

[0056] (2.4) The aggregation server calculates the contribution of each participant:

[0057] Specifically, each participant in step (1.1) establishes a routing table, including the following steps:

[0058] (1.1.1) When the number of participants is 2, the participants are divided into two categories: first hop and last hop; when the number of participants is greater than 2, the participants are divided into three categories: first hop, middle hop and last hop;

[0059] (1.1.2) Each participant establishes a routing table between the participants: if there is an intermediate hop participant, the next hop of the first hop participant is the intermediate hop participant, otherwise the next hop is the last hop participant; the next hop of the intermediate hop participant is the intermediate hop participant, and the next hop of the last intermediate hop participant is the last hop participant.

[0060] Specifically, the communication framework described in step (1.3) is as follows: Figure 1 As shown, the following steps are included:

[0061] (1.3.1) The first-hop participant slices the partial prediction value P1, Slice a Send to the aggregation server, and send another slice Send to the next hop;

[0062] (1.3.2) If there is an intermediate hop server, the intermediate hop participant receives the slice of the previous hop participant Overlay a slice with its own partial predictions: Slice the superimposed result. Slice a Send to the aggregation server, and send another slice Send to the next hop participant. Where k is the intermediate hop server;

[0063] (1.3.3) If there is an intermediate hop participant, the previous hop of the tail hop participant is the intermediate hop participant, otherwise, it is the first hop participant. The tail hop participant receives the slice of the previous hop participant, superimposes the slice with its own partial prediction value, and slices the superimposed result. The tail jump participant will slice two and All are sent to the aggregation server.

[0064] Specifically, the communication framework described in step (1.5) is as follows: Figure 1 As shown, the following steps are included:

[0065] (1.5.1) The aggregation server slices the model gradient, sends slice 1 to the first-hop participant, and sends slice 2 to the tail-hop server;

[0066] (1.5.2) Slice 1 is transmitted from the first-hop participant to the last-hop participant, and slice 2 is transmitted from the last-hop participant to the first-hop participant;

[0067] (1.5.3) Each participant superimposes the two slices to obtain the model slice of the aggregation server.

[0068] The present invention uses secret sharing to transmit the user's private data, and does not transmit its own complete model on the channel. Through the secret sharing scheme of the present invention, the aggregation server and any participant cannot obtain the private data of other participants.

[0069] The above embodiments are preferred implementation modes of the present invention, but the implementation modes of the present invention are not limited to the above embodiments. Any other changes, modifications, substitutions, combinations, and simplifications that do not deviate from the spirit and principles of the present invention should be equivalent replacement methods and are included in the protection scope of the present invention.

Claims

1. A vertical federated learning framework based on secret sharing, characterized in that: The method comprises the following steps: (1) The aggregation server and each participant perform the training task of vertical federated learning based on the communication framework; (2) The aggregation server and each participant calculate the contribution of each participant based on the communication framework.

2. A vertical federated learning framework based on secret sharing according to claim 1, characterized in that: The training task of the vertical federated learning described in step (1) includes the following steps: (1.1) Initialization: The aggregator and other participants have their own data sets. The sample spaces are aligned under the privacy intersection. The aggregator is the only one that has the real labels. The aggregator and other participants each generate a public-private key pair, make the public key public, and establish a routing table between the data sender and the recipient. (1.2) Each participant i uses local data to obtain its own partial prediction value P according to the specified task i ; (1.3) Each participant transmits part of the predicted value to the aggregation server based on the communication transmission framework; (1.4) The aggregation server decrypts the received partial prediction values, performs aggregation training, and updates the model parameters of the aggregation server; (1.5) The aggregation server and each participant transmit the model error of the aggregation server based on the communication transmission framework; (1.6) After each participant receives the error sent by the aggregation server, it performs gradient descent based on the error and updates the local model.

3. According to a vertical federated learning framework based on secret sharing according to claim 1, it is characterized in that: The calculation of the contribution of each participant in step (2) includes the following steps: (2.1) The aggregation server uses a random number r to randomly mask the aggregation value Z: Z = Z + r, and transmits it to each participant based on the communication transmission framework; (2.2) Calculation for each participant i: Average value Distance Weight m is the number of participants; (2.3) Each participant transmits the calculation result dist based on the communication transmission framework i Transmit to the aggregation server; (2.4) The aggregation server calculates the contribution of each participant:

4. A vertical federated learning framework based on secret sharing according to claim 2, characterized in that: Each participant in step (1.1) establishes a routing table, including the following steps: (1.1.1) When the number of participants is 2, the participants are divided into two categories: first hop and last hop; when the number of participants is greater than 2, the participants are divided into three categories: first hop, middle hop and last hop; (1.1.2) Each participant establishes a routing table between the participants: if there is an intermediate hop participant, the next hop of the first hop participant is the intermediate hop participant, otherwise the next hop is the last hop participant; The next hop of the intermediate hop participant is the intermediate hop participant, and the next hop of the last intermediate hop participant is the last hop participant.

5. According to a vertical federated learning framework based on secret sharing according to claim 2, it is characterized in that: The communication framework described in step (1.3) includes the following steps: (1.3.1) The first-hop participant slices the partial prediction value P1, P1 = P1 1 +P1 2 , a slice P1 1 Send it to the aggregation server and send another slice P1 2 Send to the next hop; (1.3.2) If there is an intermediate hop server, the intermediate hop participant receives the slice of the previous hop participant Overlay a slice with its own partial predictions: Slice the superimposed result. Slice a Send to the aggregation server, and send another slice Send to the next hop participant. Where k is the intermediate hop server; (1.3.3) If there is an intermediate hop participant, the previous hop of the tail hop participant is the intermediate hop participant, otherwise, it is the first hop participant. The tail hop participant receives the slice of the previous hop participant, superimposes the slice with its own partial prediction value, and slices the superimposed result. The tail jump participant will slice two and All are sent to the aggregation server.

6. A vertical federated learning framework based on secret sharing according to claim 2, characterized in that: The communication framework described in step (1.5) includes the following steps: (1.5.1) The aggregation server slices the model gradient, sends slice 1 to the first-hop participant, and sends slice 2 to the tail-hop server; (1.5.2) Slice 1 is transmitted from the first-hop participant to the last-hop participant, and slice 2 is transmitted from the last-hop participant to the first-hop participant; (1.5.3) Each participant superimposes the two slices to obtain the model slice of the aggregation server.

Images