Longitudinal federated learning methods, devices, and media with different sample identifiers

Abstract

This invention relates to the field of federated learning technology, specifically to a method, apparatus, and medium for longitudinal federated learning with different sample identifiers. The method includes: in the alignment phase of federated learning, performing sample alignment for participants holding different sample identifiers using an unintentional, programmable pseudo-random function (OPPRF), Cuckoo Hashing, and Simple Hashing, adding noise to the sample identifiers during the alignment process; and completing the joint training of the model by participants using Paillier homomorphic encryption, ensuring the privacy of participants during the training process through encryption. This invention can achieve sample alignment when participants have different sample identifiers, achieving the goal of joint model training while protecting the privacy of each participant.

Description

Technical Field

[0001] This invention relates to the field of federated learning technology, and more specifically to longitudinal federated learning methods, devices, and media for samples with different identifiers. Background Technology

[0002] Vertical federated learning is suitable for datasets with significant overlap in user samples but limited overlap in user features, such as banks and e-commerce companies. These companies provide different services to users, resulting in users exhibiting diverse characteristics, but their user bases have considerable overlap. Vertical federated learning increases the feature dimension of the training dataset. Before jointly training the model, samples must first be aligned. Current vertical federated learning schemes primarily employ Privacy Set Intersection (PSI) techniques for sample alignment. For example, WeBank's federated learning framework Fate mainly uses RSA and hash function-based PSI for sample alignment. Liu et al., in their paper "Asymmetrical vertical federated learning," used a Pohlig-Hellman-based PSI for sample alignment in asymmetric federated learning environments. However, these schemes all assume that the participants have the same sample identifiers, failing to consider alignment issues when sample identifiers differ. After sample alignment, the participants can jointly train the model. Current vertical federated learning schemes mainly employ cryptographic methods such as homomorphic encryption, secret sharing, and differential privacy to protect privacy information during training. In their paper "Private federated learning on vertically partitioned data via entity resolution and additively homomorphic encryption", Hardy et al. used an additive homomorphic encryption scheme to protect privacy information during the interaction process. In order to further protect the gradient information, the sender needs to add a random number to the gradient before sending it to the receiver. This scheme requires the participation of a coordinator.

[0003] With the advent of the big data era, data, as a new factor of production, has become a research hotspot, and extracting valuable information from massive amounts of data has gradually become a key research area. Traditional data processing methods involve aggregating all data for analysis and modeling. However, this aggregation can expose sensitive user information. If those who possess the data maliciously use it during analysis and modeling—for example, by privately packaging and selling collected data—it can lead to the leakage of user privacy. Data circulation between enterprises faces challenges such as reluctance, inability, and unwillingness, severely restricting data flow and causing problems like data silos. Therefore, how to both protect data privacy and fully realize data value has become a research focus. Federated learning emerged to address this need. It aims to help participating parties collaboratively complete training without leaking their local private data. Its key feature is that private data does not leave the local machine. Due to this excellent data privacy protection characteristic, federated learning is widely used in various fields. Based on data distribution, federated learning can be divided into vertical federated learning, horizontal federated learning, and federated transfer learning. In vertical federated learning, sample alignment is required before participants jointly train the model. Existing vertical federated learning methods either neglect sample alignment or only address alignment when participants' unique identifiers (IDs) are identical. However, in reality, the same user may use different information when registering on different platforms. For example, a user might register with an email address on platform A and a phone number on platform B, resulting in different unique identifiers for the same user in the two databases. If these two platforms need to jointly train the model, sample alignment is essential, rendering traditional vertical federated learning methods inapplicable. Therefore, designing a vertical federated learning method that can handle situations where participants' sample identifiers are different is crucial and has significant application value. Summary of the Invention

[0004] To address the technical problems existing in the prior art, this invention provides a vertical federated learning method, device, and medium for cases where sample identifiers are different. When the sample identifiers in the participant databases are different, the participants' samples are first aligned, and then the participants can use their data to jointly train the model. This solves the alignment problem when the sample identifiers in the participant databases are different, and further, combined with the training phase, a complete vertical federated learning process is realized.

[0005] The first objective of this invention is to provide a longitudinal federated learning method for samples with different identifiers.

[0006] A second objective of this invention is to provide a computer device.

[0007] A third objective of this invention is to provide a storage medium.

[0008] The first objective of this invention can be achieved by adopting the following technical solution:

[0009] A longitudinal federated learning method for samples with different identifiers, the method comprising:

[0010] S1. Participants A and B, together with third party C, run the unintentional and programmable pseudo-random function OPPRF to obtain the corresponding unintentional and programmable pseudo-random function OPPRF output. Participants A and B add the corresponding unintentional and programmable pseudo-random function OPPRF output as noise to their own datasets.

[0011] S2. Participant B sends the dataset with added noise to Participant A. Participant A obtains the intersection of the datasets with added noise from both parties based on Participant B's dataset with added noise, and sorts Participant A's dataset based on this intersection. Participant A then sends the intersection of these datasets to Participant B, and Participant B sorts Participant B's dataset based on this intersection.

[0012] S3. Participants A and B jointly train a logistic regression model using gradient descent. They exchange ciphertext of intermediate results, calculate the complete encrypted gradient, add noise to the encrypted gradient, and send it to third party C. Third party C decrypts the received ciphertext to obtain plaintext, which is then sent to both participants A and B. Participants A and B remove noise from the received plaintext and update their gradients. Participant A then obtains the trained logistic regression model θ. A Participant B receives the trained logistic regression model θ B .

[0013] In a preferred embodiment, step S1 includes:

[0014] S11. Participant A, Participant B, and Third Party C each hold a set of all samples in their own database under an identifier. Participant A, Participant B, and Third Party C negotiate three hash functions.

[0015] A third party C holds two identifiers, which form two sets. The elements in the two sets correspond one-to-one. The third party C adds noise to each element in the two sets to obtain a new value. The same noise is added to the same position in the two sets. The elements in one set and the noise-added values ​​in the other set are combined into a point value pair by row, forming two point value pair sets. The point value pairs in the point value pair sets are used as the input for C when running OPPRF.

[0016] S12. Participants A and B store their own sets in bins using the Cuckoo Hash Function, with each bin storing at most one element; Participant C maps the elements from its two sets to bins using a simple hash function, with each bin storing multiple elements; if a bin of Participants A and B has no element stored, then Participants A and B store an invalid element in that bin.

[0017] S13. For each bin, participant A and participant C run the pseudo-random function OPPRF. Participant A is the receiver of the unintentional and programmable pseudo-random function OPPRF and inputs the elements in its bin. Participant C is the sender of the unintentional and programmable pseudo-random function OPPRF and inputs the elements in the bin and their corresponding values ​​after adding noise. Participant A obtains the corresponding unintentional and programmable pseudo-random function OPPRF output.

[0018] S14. For each bin, participant B and participant C run the pseudo-random function OPPRF. Participant B is the receiver and participant C is the sender. Participant B receives the corresponding unintentional and programmable pseudo-random function OPPRF output.

[0019] The second objective of this invention can be achieved by adopting the following technical solution:

[0020] A computer device includes a processor and a memory for storing a processor-executable program, wherein when the processor executes the program stored in the memory, it implements the above-described longitudinal federated learning method under different sample identifiers.

[0021] The third objective of this invention can be achieved by adopting the following technical solution:

[0022] A storage medium storing a program that, when executed by a processor, implements the aforementioned longitudinal federated learning method for samples with different identifiers.

[0023] Compared with the prior art, the present invention has the following advantages and beneficial effects:

[0024] The present invention provides a longitudinal federated learning method, device, and medium for use with different sample identifiers. During the alignment phase of federated learning, it aligns samples held by participants using techniques such as the unintentional, programmable pseudo-random function OPPRF, Cuckoo Hashing, and Simple Hashing, adding noise to the sample identifiers during alignment. Paillier homomorphic encryption is used to complete the joint training of the model by the participants, ensuring their privacy during the training process. This invention can achieve sample alignment even when participants have different sample identifiers, achieving the goal of joint model training while protecting the privacy of each participant. Attached Figure Description

[0025] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on the structures shown in these drawings without creative effort.

[0026] Figure 1 This is a flowchart of the longitudinal federated learning method under different sample identifiers in this embodiment of the invention;

[0027] Figure 2 This is a schematic diagram of the process of two medical institutions jointly training a logistic regression model in an embodiment of the present invention. Detailed Implementation

[0028] The technical solution of the present invention will be further described in detail below with reference to the accompanying drawings and embodiments. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments, and the implementation of the present invention is not limited thereto. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0029] Example 1:

[0030] This invention addresses the problem of alignment when sample identifiers are different, which is a challenge of existing solutions. It proposes a longitudinal federated learning method for this purpose. This method achieves sample alignment when the sample identifiers of participating parties are different, and then uses homomorphic encryption to train the model after alignment. This effectively ensures the privacy and security of sample information and gradients for each participating party throughout the entire training process.

[0031] This invention proposes a longitudinal federated learning method for samples with different identifiers. It primarily utilizes four techniques—OPPRF, SimpleHashing, Cuckoo Hashing, and Paillier homomorphic encryption—to construct the scheme. The four techniques are described below:

[0032] I. Oblivious Programmable PRF (Oplivious Programmable PRF)

[0033] OPPRF is a combination of the inadvertent pseudo-random function OPRF and the programmable pseudo-random function PPRF. It mainly involves a sender and a receiver, with the sender being A and the input of A being {(x... i ,y i ),i∈[n]}, the receiver is B, and the input of B is x. This invention adopts an OPPRF based on table construction, which mainly includes the following 5 steps:

[0034] (1) A and B run the OPRF protocol. A's input is k and B's input is x. After the protocol runs, B receives F(k,x).

[0035] (2) For {x i Let A compute {F(k,x1),F(k,x2),...,F(k,x)}, i∈[n]}. n A samples the value v until all {H(F(k,x)} are reached. i The expressions )||v),∈[n]} are different.

[0036] (3) For i∈[n], A first calculates h i =H(F(k,x) i )||v), then the hth digit of table T i The element at each position is set as

[0037] (4) A stores random numbers at positions in table T where no elements are stored, and then sends table T and v to B.

[0038] (5) B calculates h = H(F(k,x)||v), and then outputs it.

[0039] If x = x i B will get x i Corresponding y i Otherwise, a random number will be obtained.

[0040] II. Simple Hashing

[0041] Simple Hashing contains b bins {B1, B2, ..., B} b} and k hash functions {h1, h2, ..., h k Each bin can store multiple elements. When storing element x into a bin, first calculate the hash function values ​​of x under k hash functions {h1(x),h2(x),…,h...}. k (x)}, then put x into the corresponding k bins.

[0042] III. Cuckoo Hashing (Cuckoo Hash Function)

[0043] Cuckoo Hashing contains k hash functions {h1, h2, ..., hk}. k} and b bins {B1, B2, ..., B b Each bin can hold at most one element. Element x is stored in... In one of the bins.

[0044] IV. Paillier Homomorphic Encryption

[0045] This invention utilizes the Paillier homomorphic encryption scheme to ensure the security of privacy information during the interaction phase of training. It typically consists of the KeyGen, Enc, and Den algorithms. Specifically, the KeyGen algorithm takes security parameters as input and outputs pk and sk; the Enc algorithm takes plaintext m and pk as input and outputs ciphertext m. Dec algorithm input ciphertext And sk, output the decrypted plaintext m. The Paillier homomorphic encryption scheme supports the following two operations:

[0046] Add the two ciphertexts together:

[0047]

[0048] Constant multiplication ciphertext:

[0049]

[0050] The federated learning described in this invention is a vertical federated learning model, which includes participant A, participant B, and a trusted third party C. Participants A and B, with the help of third party C, perform sample alignment and model training, ultimately obtaining their own trained models. The vertical federated learning method includes a sample alignment phase and a training phase.

[0051] Participant A and Participant B each hold their own databases. Each sample in the database consists of an identifier and a feature value. Each participant holds a portion of the features, and the sample identifiers of Participant A and Participant B are different. For example, the sample identifier of Participant A is ID1, and the sample identifier of Participant B is ID2. Third party C has massive amounts of data, and each sample in the database contains two identifiers (ID1, ID2).

[0052] like Figure 1 As shown, the present invention describes a vertical federated learning method for samples with different identifiers. The vertical federated learning method includes participant A, participant B, and a third party C. The method is characterized by the following steps:

[0053] S1. Participants A and B, together with third party C, run the unintentional and programmable pseudo-random function OPPRF to obtain the corresponding OPPRF output. Participants A and B then add the corresponding OPPRF output as noise to their own datasets.

[0054] S11. Participant A, Participant B, and Third Party C each hold a set of all samples in their own database under a sample identifier. Participant A, Participant B, and Third Party C negotiate three hash functions.

[0055] If participants A and B want to train a model together, they first need to align the samples. Since the sample IDs of A and B are different, this invention introduces a trusted third party C with massive amounts of data to assist A and B in aligning the samples.

[0056] A third party, C, possesses two identifiers, which constitute two sets. The elements in these sets correspond one-to-one. Third party C adds noise to each element in both sets, adding the same noise to the same positions in both sets to obtain new values. This step is to protect the privacy of third party C's identifiers during the processes described below.

[0057] S12. Participants A and B store their respective sets into bins using the Cuckoo hash function, with each bin storing at most one element. Third party C maps the elements from the two sets into bins using a simple hash function, with each bin storing multiple elements. After mapping to bins, if a bin does not contain an element, participants A and B store an invalid element ⊥ in that bin. This mapping to bins step allows participants to perform corresponding operations by bin in the following steps; that is, participants only need to operate on bins with the same label, without needing to operate on bins with different labels, thus reducing the number of operations between bins.

[0058] S13. For each bin, participant A and third party C run an unintentional, programmable pseudo-random function (OPPRF). The OPPRF involves two participants, a sender and a receiver. The sender inputs several pairs of values ​​(x... i ,y i If the receiver inputs x, then if x equals some x i Then the receiver receives the corresponding y i For each bin, participant A acts as the receiver of the OPPRF, inputting the elements in its bin, while third party C acts as the sender of the OPPRF, inputting the elements stored in that bin and their corresponding values ​​after adding noise. Participant A then obtains the corresponding OPPRF output.

[0059] S14. Similarly, for each bin, participant B and third party C run an unintentional, programmable pseudo-random function (OPPRF). Participant B acts as the receiver and third party C acts as the sender. Participant B can obtain the corresponding OPPRF output.

[0060] S2. Participant B sends the dataset with added noise to Participant A. Participant A obtains the intersection of the datasets with added noise from both parties based on Participant B's dataset with added noise, and sorts Participant A's dataset based on the intersection of the datasets. Participant A then sends the intersection of the datasets to Participant B, and Participant B sorts Participant B's dataset based on the intersection of the datasets.

[0061] S21. Loop through each bin. If the bin does not store invalid elements, participant B will XOR the output of the corresponding unintentional, programmable pseudo-random function OPPRF with the elements in its own bin, forming a set K. B Set K B Send to participant A; participant A then XORs the corresponding unintentional, programmable pseudo-random function OPPRF output with elements in its own bin to form set K. A By matching K A and K B According to set K A and set K B The intersection of the two sets of points yields the intersection K. A ', will intersect with K A 'Corresponding element e i Insert into intersection S A , intersect K A Send to participant B.

[0062] S22, Participant B according to K A 'and K B The intersection of the elements p will be used to select the corresponding elements p. iInsert into intersection S B .

[0063] After the alignment phase described above, if the same user is registered in both participant A and participant B, then the user's two identifiers will be located in the intersection S. A Intersection S B Same line.

[0064] Specifically, in this embodiment, it is assumed that the ID of the sample in participant A's database is an email address, and the set E = {e1, e2, ..., e...} m}; The IDs of samples in participant B's database are phone numbers, and the set P = {p1, p2, ..., p...} m}; while the sample identifiers in the trusted third-party C database include both email addresses and phone numbers, the set EP={(e′1,p′1),(e′2,p′2),…,(e′ n ,p′ n Participants A, B, and third party C agree on three hash functions {h1, h2, h3} and the number of bins b. The specific steps for sample alignment are as follows:

[0065] Third-party C compares each pair of elements {(e′) in set EP j ,p′ j ),j∈[n, select a random number {r j ,j∈[n]}. Then calculate: and

[0066] Participant A uses Cuckoo Hashing to map each element in set E to b bins {B} a [1],B a [2]…,B a [b]}, where B a [i] represents the element in the i-th bin of A. Participant B uses Cuckoo Hashing to map each element in set P to b bins {B}. b [1],B b [2]…,B b [b]}, where B b [i] represents the element in the i-th bin of B. After A and B complete the Cuckoo Hashing operation, if any bin of A or B is empty, an invalid element ⊥ is filled into that bin. C has two sets of bins. and C uses Simple Hashing to transform {e′1,e′2,…,e′} n}Mapped to the first set of bin middle Similarly, C uses Simple Hashing to transform {p′1,p′2,…,p′} n}Mapped to the second set of bin middle in This represents all elements in the i-th bin.

[0067] For each bin, third parties C and A run the OPPRF protocol once. In the i-th bin, participant A acts as the receiver and inputs element B from the i-th bin. a [i], Third-party C acts as the sender and inputs the element in the i-th bin. After the protocol is executed, A receives y′ i If the i-th bin of participant A does not store an invalid element ⊥, participant A calculates...

[0068] Similarly, for each bin, third parties C and B run the OPPRF protocol once. In the i-th bin, B inputs B b [i], C input If the i-th bin of B does not store an invalid element ⊥, then B calculates...

[0069] Participant B Order Then K B Send to A.

[0070] Participant A Order Initialize set S a and K′ A For an empty set, for {K} A [j]∈K A ,j∈m},A checks K A [j]∈K B Is this valid? If so, party A will give B... a [j] Place it into the intersection S A In the middle, put K A [j]Place K′ A In the middle, finally put K′ A Send to B.

[0071] For {K B [k]∈K B ,k∈m}, participant B checks K B [k]∈K′ A Whether it is true, if K B [k]=K′ A [j], then B b [k] is placed in set S bIn the j-th row, the final participant B has an intersection S. B .

[0072] After the above protocol is completed, if email address e i and phone number p i If they belong to the same user, then e i and p i They will appear in set S respectively A and set S B The same row is used to complete sample alignment.

[0073] S3. Participant A and Participant B jointly train the logistic regression model using the gradient descent method. Participant A and Participant B exchange encrypted intermediate results, calculate the complete encrypted gradient, add noise to the encrypted gradient, and send it to third party C. Third party C decrypts the ciphertext, removes the noise, and updates the gradient.

[0074] After aligning samples using the alignment method of this invention, the participants can jointly train various models, such as linear regression models, logistic regression models, and neural networks. To achieve a complete longitudinal federated learning process, this invention utilizes homomorphic encryption to construct a secure model training process. With the assistance of a third party C, participants A and B jointly train the logistic regression model using gradient descent.

[0075] During the training phase, following the alignment phase described above, participant A and participant B rearranged their datasets based on the intersection results. Participant B's dataset also included the labels for each data point. The training process included the following steps:

[0076] S31. Third party C generates a public and private key for homomorphic encryption and sends the public key to participant A and participant B. Participant A and participant B initialize their own weights respectively.

[0077] S32. For each data point in the dataset, participant A calculates the ciphertext of its local intermediate result based on the weight and the data point itself. Participant B calculates the ciphertext of its local intermediate result based on the weight, the data point itself, and the label value. Participants A and B send their ciphertexts of intermediate results to each other. Participants A and B then homomorphically add their own ciphertext and the received ciphertext to obtain a new ciphertext. Participants A and B multiply each data point in their own dataset by the new ciphertext. After iterating through each data point in the dataset, they accumulate the results to obtain their complete encrypted gradient. The ciphertext of the intermediate result is used to calculate the complete gradient values ​​for participants A and B.

[0078] S33. Participant A and Participant B each add noise to the complete encrypted gradient and send it to third party C. Third party C decrypts the two ciphertexts and sends them to Participant A and Participant B respectively.

[0079] S34. Participants A and B remove noise from the received values ​​and update the gradients of Participant A and Participant B.

[0080] S35. Repeat steps S32-S34 until the model converges or the maximum number of iterations is reached. Participant A and participant B then obtain the trained logistic regression model.

[0081] In this embodiment, after aligning the samples of participant A and participant B, participant A and participant B have n intersection elements. A and B rearrange their own datasets, so that both datasets contain n samples, where participant A's dataset is X. A =(x A,1 ,x A,2 ,…,x A,n Each sample x A,i There is d A There are 10 feature values, and participant A initializes the model's weights as a column vector θ. A The length is d A Participant B, Dataset X B =((x) B,1 ,y1),(x B,2 ,y2),…,(x B,n ,y n )), where each sample x B,i There is d B There are eigenvalues ​​and their corresponding labels are y. i B initializes the model's weights as column vectors θ. B , length d B .

[0082] Participant A and Participant B jointly negotiate the learning rate η and the loss function for logistic regression:

[0083]

[0084] in Then the loss function L(θ) is related to θ A The gradient of participant A is obtained by taking the partial derivative. The formula for calculating the gradient of participant A is as follows:

[0085]

[0086] Where i represents the i-th data point, and each participant has n data points for training; θ is the weight column vector. Because there are multiple features, each feature has a weight value. Participants A and B have different features, so the weight of participant A is θ. A The weight of participant B is θ B θ contains θ A and θ B , This represents the vertical concatenation of vectors.

[0087] The loss function L(θ) with respect to θ B The gradient of participant B is obtained by taking the partial derivative. The formula for calculating the gradient of participant B is as follows:

[0088]

[0089] Participant A and Participant B can update the gradient using the following formula:

[0090]

[0091] Specifically, the steps for participant A and participant B to jointly train a logistic regression model with the help of third party C are as follows:

[0092] (1) Third party C generates public and private keys (pk, sk) using the KeyGen algorithm for homomorphic encryption, and then sends the public key pk to participants A and B.

[0093] For i = 1, ..., n, execute steps (2) to (4).

[0094] (2) Local calculation by participant A Then encrypt u using the public key pk. A,i Receive ciphertext And send it to participant B.

[0095] (3) Local calculation by participant B Then encrypt u using the public key pk. B,i Received the ciphertext And send it to participant A.

[0096] (4) Local calculation by participant A And calculate Participant B's local calculation And calculate

[0097] (5) A calculates the gradient Then select a random number R. A To blind the local gradient, calculate And send the calculation results to C.

[0098] (6) Similarly, B calculates the gradient. Then select a random number R. B To blind the local gradient, calculate And send the calculation results to C.

[0099] (7) C updates the gradient after receiving the gradients from A and B. First, it decrypts the gradient using its own private key sk to obtain the gradient. and Then put Send to A, Send to B.

[0100] (8) A is obtained after removing the random number. Then update the gradient locally.

[0101] (9) B removes the random number and obtains Then update the gradient locally.

[0102] Repeat steps (2) through (9) until the model converges.

[0103] In this embodiment, we take the example of two medical institutions, one domestic and one international, jointly training a logistic regression model. Figure 2 As shown, this application example mainly includes a foreign medical institution A (sample identifier is an email address), a domestic medical institution B (sample identifier is a phone number), and a trusted third party C (sample identifier includes both an email address and a phone number). The steps for training the logistic regression model for the foreign medical institution A and the domestic medical institution B with the help of the trusted third party C are as follows:

[0104] (1) Running OPPRF: Foreign medical institution A and domestic medical institution B respectively use Cuckoo Hashing to store the elements in the email set and the phone number set into bin. After C uses Simple Hashing to store the elements in the email set and the phone number set into bin, for each bin, foreign medical institution A and domestic medical institution B respectively perform an OPPRF operation with a trusted third party C. Foreign medical institution A and domestic medical institution B respectively obtain the OPPRF output value corresponding to each element in the set.

[0105] (2) Interacting information to achieve sample alignment: Foreign medical institution A and domestic medical institution B respectively XOR the corresponding OPPRF values ​​of the elements in the set and interact to obtain the intersection elements, thereby achieving sample alignment.

[0106] (3) Sending public key: C sends the public key to medical institution A abroad and medical institution B in China.

[0107] (4) Sending the encrypted intermediate results: For each data item, foreign medical institution A and domestic medical institution B respectively calculate u A,i and u B,i Then, the corresponding ciphertext is sent to the other party to assist them in calculating the gradient.

[0108] (5) Calculate the gradient: Both parties use the information obtained from the interaction to calculate the ciphertext of their own gradient.

[0109] (6) Sending encrypted and noisy gradients: Foreign medical institution A and domestic medical institution B send their own gradients to a trusted third party C for decryption after the ciphertext is blinded.

[0110] (7) Gradient with noise: Trusted third party C decrypts the ciphertext sent by foreign medical institution A and domestic medical institution B, and returns it to A and B respectively.

[0111] (8) Update gradient: After receiving the plaintext, foreign medical institution A and domestic medical institution B remove noise to obtain the gradient, and then update the gradient locally.

[0112] Repeat steps (4) to (8) until the model converges.

[0113] After the above steps are completed and the model converges, the foreign medical institution A holds the trained model θ. A Domestic medical institution B possesses a pre-trained model θ B When using this model for prediction, assume that one predicted data point for A is x′. A One of B's ​​predicted data points is x′. B Then A calculates And substitute it into the activation function Obtain the predicted value y′ A , where y′ A ∈[0,1]. Similarly, B can also be used to calculate the predicted data x′. B The corresponding predicted value y′ B .

[0114] In summary, the longitudinal federated learning method for different sample identifiers provided by this invention can achieve sample alignment when the sample identifiers of the participants are different, and can achieve the goal of jointly training the model while protecting the privacy information of each participant.

[0115] Example 2:

[0116] This embodiment provides a computer device, which may be a server, computer, etc., including a processor, memory, input device, display, and network interface connected via a system bus. The processor provides computing and control capabilities. The memory includes a non-volatile storage medium and internal memory. The non-volatile storage medium stores an operating system, computer programs, and a database. The internal memory provides an environment for the operation of the operating system and computer programs in the non-volatile storage medium. When the processor executes the computer programs stored in the memory, it implements the vertical federated learning method under different sample identifiers in Embodiment 1 above, as follows:

[0117] S1. Participants A and B, together with third party C, run the unintentional and programmable pseudo-random function OPPRF to obtain the corresponding unintentional and programmable pseudo-random function OPPRF output. Participants A and B add the corresponding unintentional and programmable pseudo-random function OPPRF output as noise to their own datasets.

[0118] S2. Participant B sends the dataset with added noise to Participant A. Participant A obtains the intersection of the datasets with added noise from both parties based on Participant B's dataset with added noise, and sorts Participant A's dataset based on this intersection. Participant A then sends the intersection of these datasets to Participant B, and Participant B sorts Participant B's dataset based on this intersection.

[0119] S3. Participants A and B jointly train a logistic regression model using gradient descent. Participants A and B exchange encrypted intermediate results, calculate the complete encrypted gradient, add noise to the encrypted gradient, and send it to third party C. Third party C decrypts the received ciphertext to obtain the plaintext, and sends the plaintext to both participants A and B. Participants A and B remove noise from the received plaintext and update their gradients. Participant A then obtains the trained logistic regression model θ. A Participant B receives the trained logistic regression model θ B .

[0120] Step S1 includes:

[0121] S11. Participant A, Participant B, and Third Party C each hold a set of all samples in their own database under an identifier. Participant A, Participant B, and Third Party C negotiate three hash functions.

[0122] A third party C holds two identifiers, which form two sets. The elements in the two sets correspond one-to-one. The third party C adds noise to each element in the two sets to obtain a new value. The same noise is added to the same position in the two sets. The elements in one set and the noise-added values ​​in the other set are combined into a point value pair by row, forming two point value pair sets. The point value pairs in the point value pair sets are used as the input for C when running OPPRF.

[0123] S12. Participants A and B store their own sets in bins using the Cuckoo Hash Function, with each bin storing at most one element; Participant C maps the elements from its two sets to bins using a simple hash function, with each bin storing multiple elements; if a bin of Participants A and B has no element stored, then Participants A and B store an invalid element in that bin.

[0124] S13. For each bin, participant A and participant C run the pseudo-random function OPPRF. Participant A is the receiver of the unintentional and programmable pseudo-random function OPPRF and inputs the elements in its bin. Participant C is the sender of the unintentional and programmable pseudo-random function OPPRF and inputs the elements in the bin and their corresponding values ​​after adding noise. Participant A obtains the corresponding unintentional and programmable pseudo-random function OPPRF output.

[0125] S14. For each bin, participant B and participant C run the pseudo-random function OPPRF. Participant B is the receiver and participant C is the sender. Participant B receives the corresponding unintentional and programmable pseudo-random function OPPRF output.

[0126] Example 3:

[0127] This embodiment provides a storage medium, which is a computer-readable storage medium, storing a computer program. When the program is executed by a processor, the processor executes the computer program stored in the memory to implement the vertical federated learning method under different sample identifiers in Embodiment 1 above, as follows:

[0128] S1. Participants A and B, together with third party C, run the unintentional and programmable pseudo-random function OPPRF to obtain the corresponding unintentional and programmable pseudo-random function OPPRF output. Participants A and B add the corresponding unintentional and programmable pseudo-random function OPPRF output as noise to their own datasets.

[0129] S2. Participant B sends the dataset with added noise to Participant A. Participant A obtains the intersection of the datasets with added noise from both parties based on Participant B's dataset with added noise, and sorts Participant A's dataset based on this intersection. Participant A then sends the intersection of these datasets to Participant B, and Participant B sorts Participant B's dataset based on this intersection.

[0130] S3. Participants A and B jointly train a logistic regression model using gradient descent. Participants A and B exchange encrypted intermediate results, calculate the complete encrypted gradient, add noise to the encrypted gradient, and send it to third party C. Third party C decrypts the received ciphertext to obtain the plaintext, and sends the plaintext to both participants A and B. Participants A and B remove noise from the received plaintext and update their gradients. Participant A then obtains the trained logistic regression model θ. A Participant B receives the trained logistic regression model θ B .

[0131] Step S1 includes:

[0132] S11. Participant A, Participant B, and Third Party C each hold a set of all samples in their own database under an identifier. Participant A, Participant B, and Third Party C negotiate three hash functions.

[0133] A third party C holds two identifiers, which form two sets. The elements in the two sets correspond one-to-one. The third party C adds noise to each element in the two sets to obtain a new value. The same noise is added to the same position in the two sets. The elements in one set and the noise-added values ​​in the other set are combined into a point value pair by row, forming two point value pair sets. The point value pairs in the point value pair sets are used as the input for C when running OPPRF.

[0134] S12. Participants A and B store their own sets in bins using the Cuckoo Hash Function, with each bin storing at most one element; Participant C maps the elements from its two sets to bins using a simple hash function, with each bin storing multiple elements; if a bin of Participants A and B has no element stored, then Participants A and B store an invalid element in that bin.

[0135] S13. For each bin, participant A and participant C run the pseudo-random function OPPRF. Participant A is the receiver of the unintentional and programmable pseudo-random function OPPRF and inputs the elements in its bin. Participant C is the sender of the unintentional and programmable pseudo-random function OPPRF and inputs the elements in the bin and their corresponding values ​​after adding noise. Participant A obtains the corresponding unintentional and programmable pseudo-random function OPPRF output.

[0136] S14. For each bin, participant B and participant C run the pseudo-random function OPPRF. Participant B is the receiver and participant C is the sender. Participant B receives the corresponding unintentional and programmable pseudo-random function OPPRF output.

[0137] The above embodiments are preferred embodiments of the present invention, but the embodiments of the present invention are not limited to the above embodiments. Any changes, modifications, substitutions, combinations, or simplifications made without departing from the spirit and principle of the present invention shall be considered equivalent substitutions and shall be included within the protection scope of the present invention.

Claims

1. A vertical federated learning method with different sample identifiers, wherein the vertical federated learning includes participant A, participant B, and third party C, characterized in that, The method includes the following steps: S1. Participants A and B, together with third party C, run the unintentional and programmable pseudo-random function OPPRF to obtain the corresponding unintentional and programmable pseudo-random function OPPRF output. Participants A and B add the corresponding unintentional and programmable pseudo-random function OPPRF output as noise to their own datasets. Step S1 includes: S11. Participant A, Participant B, and Third Party C each hold a set of all samples in their own database under an identifier. Participant A, Participant B, and Third Party C negotiate three hash functions. A third party C holds two identifiers, which form two sets. The elements in the two sets correspond one-to-one. The third party C adds noise to each element in the two sets to obtain a new value. Specifically, the same noise is added to the same position in the two sets. The elements in one set and the noise-added values ​​in the other set are combined into a point value pair by row, forming two point value pair sets. The point value pairs in the point value pair sets are used as the input for C when running OPPRF. S12. Participants A and B store their own sets in bins using the Cuckoo Hash Function, with each bin storing at most one element. Participant C maps the elements from its two sets to bins using a simple hash function, with each bin storing multiple elements. If a bin of participants A and B has no element stored, then participants A and B store an invalid element in that bin. S13. For each bin, participants A and C run the pseudo-random function OPPRF. Participant A is the receiver of the unintentional and programmable pseudo-random function OPPRF and inputs the elements in its bin. Participant C is the sender of the unintentional and programmable pseudo-random function OPPRF and inputs the elements in the bin and their corresponding values ​​after adding noise. Participant A obtains the corresponding unintentional and programmable pseudo-random function OPPRF output. S14. For each bin, participant B and participant C run the pseudo-random function OPPRF. Participant B is the receiver and participant C is the sender. Participant B receives the corresponding unintentional and programmable pseudo-random function OPPRF output. S2. Participant B sends the dataset with added noise to Participant A. Participant A obtains the intersection of the datasets with added noise from both parties based on Participant B's dataset with added noise, and sorts Participant A's dataset based on this intersection. Participant A then sends the intersection of these datasets to Participant B, and Participant B sorts Participant B's dataset based on this intersection. S3. Participant A and Participant B jointly train a logistic regression model using gradient descent. Participants A and B exchange encrypted intermediate results, calculate the complete encrypted gradient, add noise to the encrypted gradient, and send it to third party C. Third party C decrypts the received ciphertext to obtain the plaintext, and sends the plaintext to both Participants A and B. Participants A and B remove noise from the received plaintext and update their gradients. Participant A then obtains the trained logistic regression model. Participant B receives the trained logistic regression model. .

2. The longitudinal federated learning method for different sample identifiers according to claim 1, characterized in that, Step S2 includes: S21. Loop through each bin. If the bin does not store invalid elements, participant B will XOR the output of the corresponding unintentional, programmable pseudo-random function OPPRF with the elements in its own bin, forming a set K. B Set K B The data is sent to participant A; similarly, participant A XORs the output of the corresponding unintentional, programmable pseudo-random function OPPRF with the elements in its own bin to form set K. A By matching K A and K B According to set K A and set K B The intersection of the two sets of points yields the intersection K. A ', will intersect with K A 'Put the corresponding elements into the intersection S' A , intersect K A 'Send to participant B;' S22, Participant B, based on the intersection K A 'and set K B Find the intersection of elements and place the corresponding elements into the intersection S. B .

3. The longitudinal federated learning method for different sample identifiers according to claim 1, characterized in that, Step S3 includes: S31. Third party C generates a public and private key for homomorphic encryption and sends the public key to participant A and participant B. Participant A and participant B initialize their own weights respectively. S32. For each data point in the dataset, participant A calculates the ciphertext of its local intermediate result based on the weight and the data point. Participant B calculates the ciphertext of its local intermediate result based on the weight, the data point, and the label value. Participants A and B send their ciphertexts of intermediate results to each other. Participants A and B then homomorphically add their own ciphertext and the received ciphertext of intermediate results to obtain a new ciphertext. Participants A and B multiply each data point in their own dataset by the new ciphertext. After iterating through each data point in the dataset, they accumulate the results to obtain their complete encrypted gradient. The ciphertext of the intermediate result is used to calculate the gradient values ​​of participant A and participant B. S33. Participant A and Participant B each add noise to the complete encrypted gradient and send it to Participant C. Participant C decrypts the two ciphertexts and sends them to Participant A and Participant B respectively. S34. Participants A and B remove noise from the received values ​​and update the gradients of Participant A and Participant B. S35. Repeat steps S32-S34 until the logistic regression model converges or the maximum number of iterations is reached. Participant A and participant B then obtain the trained logistic regression model.

4. The longitudinal federated learning method for different sample identifiers according to claim 3, characterized in that, The formula for calculating the gradient value of participant A is as follows: ； in, For participant A's dataset, For participant A's dataset, the i-th sample, For participant B's dataset, For the i-th sample in participant B's dataset, y i for The corresponding tags For the weight column vector, For the weight of participant A, express Transpose express Transpose; The formula for calculating the gradient value of participant B is as follows: ； in, For participant A's dataset, For the i-th sample, For participant B's dataset, For the i-th sample in participant B's dataset, for The corresponding tags For the weight column vector, For the weight of participant B, express Transpose express Transpose.

5. A computer device, comprising a processor and a memory for storing a processor-executable program, characterized in that, When the processor executes the program stored in the memory, it implements the longitudinal federated learning method under the condition that the sample identifiers are different as described in any one of claims 1-4.

6. A storage medium storing a program, characterized in that, When the program is executed by the processor, it implements the longitudinal federated learning method for sample identifiers that are different according to any one of claims 1-4.

Images