Inter-enterprise federated learning algorithm model training method based on label encryption transmission

Abstract

The application provides an inter-enterprise federated learning algorithm model training method based on label encryption transmission, comprising the following steps: S1, a coordinator performs initialization, obtains various training parameters, senses the number of participants, and starts to make the participants perform label ID alignment operation; S2, after the label alignment is completed, the coordinator initiates model training, and each participant trains a machine learning model by using common entity data; S3, each participant predicts the label result corresponding to the ID, and the coordinator calculates a final prediction value according to the prediction results of each participant. The application solves the problems of responsibility ownership and efficiency in the communication process of the traditional federated learning algorithm under the conditions of different business formats between enterprises.

Description

Technical Field

[0001] This invention relates to the field of artificial intelligence technology, and in particular to a training method for an inter-enterprise federated learning algorithm model based on tag-encrypted transmission. Background Technology

[0002] With increasing attention being paid to the importance of privacy and data security, data privacy and security issues have become hot topics, attracting significant attention from major enterprises. Meanwhile, the application of artificial intelligence and big data in corporate operations has yielded remarkable results, triggering a new wave of AI development. Against these backdrops, federated learning has emerged.

[0003] In federated learning, participants can share training data but remain invisible to each other, enabling collaborative training. Under this mechanism, a trusted coordinator controls the process and manages the generation, distribution, and decryption of keys. Each participant encrypts their data and sends it to the coordinator and other participants to jointly train a federated model, which is then distributed to all participants. However, this process requires each participant to send their own data; if keys are leaked, the risk of data leakage is shared by all participants. Furthermore, improving the efficiency of interactions at each stage while ensuring data security is crucial for enhancing the overall efficiency of federated learning.

[0004] Given the low overlap in data feature spaces among enterprises, but the potential for sharing a large user base, a federated learning approach based on user alignment and expanding the feature space is adopted. However, in traditional federated learning systems, each participant bears the risk of data leakage that may result from sending data to other participants. This risk reduces the feasibility of federated learning to some extent and hinders the establishment of a sound data-sharing environment among enterprises.

[0005] In traditional federated learning, intermediate results need to be sent in each round of interaction, and the required data is calculated using these intermediate results to update the parameters for that round. Two main scenarios that delay computation efficiency arise during this process: first, the exchanged intermediate data is in ciphertext, and computation in ciphertext is often 1-2 orders of magnitude slower than computation in plaintext; second, if the two parties complete their intermediate data computation at different times, the party that finishes first will have to wait for the other, increasing the overall time cost. Summary of the Invention

[0006] To address the aforementioned issues, this invention provides a method for training inter-enterprise federated learning algorithm models based on encrypted tag transmission. Using a homomorphic encryption algorithm, the participant holding the target prediction tag sends the encrypted tag to the unlabeled participant. Each participant then independently completes its training, obtaining its own model. The final prediction value is calculated using the prediction results from each participant. This method better aligns with the fundamental business collaboration characteristics of inter-enterprise partnerships, where "the requesting party bears limited data transmission risk, while the collaborating party does not," making federated learning more feasible. Simultaneously, this federated learning algorithm improves computational efficiency by nearly 50% while maintaining the quality of traditional federated learning.

[0007] This invention provides a training method for an inter-enterprise federated learning algorithm model based on tag-encrypted transmission. The specific technical solution is as follows:

[0008] S1: The coordinator initializes, acquires various training parameters, senses the number of participants, and begins to have the participants perform label ID alignment operations.

[0009] S2: After label alignment is completed, the coordinator initiates model training, and all participants will use the shared entity data to train the machine learning model.

[0010] S3: Each participant predicts the label result corresponding to the ID, and the coordinator calculates the final predicted value based on the prediction results of each participant.

[0011] Furthermore, the training parameters include the training objective, the number of training rounds, and regularization parameters.

[0012] Furthermore, in step S1, the sample ID alignment method uses encrypted sample alignment performed by RSA key and hash function.

[0013] Furthermore, in step S2, all participants use the same machine learning model algorithm for model training during the training process.

[0014] Furthermore, (referencing 3) step S1, the specific process is as follows:

[0015] S101: The coordinator acquires various training parameters, perceives the number of participants, and sends label alignment messages to each participant;

[0016] S102: The tag holder generates an RSA key pair, including a public key (e, n) and a private key (d, n), and uses the private key to calculate the final signature of its own sample ID;

[0017] S103: Send the public key (e, n) to the unlabeled participant and store the private key locally;

[0018] S104: Generate a random number that is coprime to n and not equal to 1 for each sample, encrypt the ID using the public key, multiply it by the random number and add blind perturbation, and use the encrypted data as the first data;

[0019] S105: The unlabeled participant sends the first data to the label-holding sender;

[0020] S106: Use the private key to perform a preliminary calculation to sign the first data, and use the data and its own final signature as the second data;

[0021] S107: The tagged party sends the second data to the untagged participant;

[0022] S108: Perform deblinding calculation on the second data and calculate the final signature. Perform an intersection operation on the final signatures of both parties to obtain the aligned ID set.

[0023] S109: The unlabeled party sends the alignment ID set to the label-holding party, the label-holding party saves the alignment ID set, and sends a sample alignment completion signal to the coordinator;

[0024] S110: After receiving the sample alignment completion signal, the coordinator sends a signal to start model training.

[0025] Further, step S2, the specific process is as follows:

[0026] S201: The coordinator generates a homomorphic encryption public key and a private key, stores the private key locally, and sends the public key to each participant;

[0027] S202: The tag holder uses the public key to homomorphically encrypt tag y and uses this data as the first encrypted data, while starting its own model training.

[0028] S203: The tag holder sends the first encrypted data to the untagged participant;

[0029] S204: Unlabeled participants use the first encrypted data as a label and combine it with their own data to train the model, obtain the required model parameters, and add a random mask to blind the data. The blinded result is then used as the second encrypted data.

[0030] S205: Unlabeled participants send the second encrypted data to the coordinator for decryption;

[0031] S206: The coordinator decrypts the second encrypted data using the private key and uses the decryption result as the third data.

[0032] S207: The coordinator sends the decrypted third data to the sender of the second encrypted data;

[0033] S208: Unlabeled participants remove the mask attached to the third data to obtain intermediate results of model parameters, and use the intermediate results to update various model parameters;

[0034] S209: Each participant checks whether the training objective has been achieved. If not, they return to step S204 and repeat the process until all participants have completed model training.

[0035] Further, step S3, the specific process is as follows:

[0036] S301: The coordinator obtains the predicted target ID and sends the ID to each participant;

[0037] S302: Each participant uses the ID, its own data, and the model to predict the label result corresponding to the ID;

[0038] S303: Each participating party sends its prediction results to the coordinator, who then uses these results to calculate the final prediction value.

[0039] Furthermore, each participant's ID, data, and model are stored locally on their respective distributed servers, and no data other than the calculation results is sent.

[0040] Furthermore, (referencing 7), the coordinator uses the following formula to calculate the final prediction result:

[0041]

[0042] Among them, pred j This represents the final prediction result for the j-th sample, where n represents the number of participants, and pred i:j This represents the prediction result of the i-th participant for the j-th sample.

[0043] The beneficial effects of this invention are as follows:

[0044] 1. This invention enables the requesting party to bear limited data transmission risk by sending encrypted tags, while the collaborating party does not bear any data transmission risk. This solves the problem of ownership of responsibility in the communication process between enterprises in different business sectors in traditional federated learning algorithms, making federated learning more operable. While utilizing data from other parties, it ensures that most data, except for the tags, does not leave the local machine, thus maintaining the privacy protection feature of federated learning.

[0045] 2. This algorithm does not require the exchange of intermediate results between each participant in each iteration, reducing intermediate communication and avoiding the time-waiting problem caused by different training data volumes during training. It improves efficiency while achieving the quality of traditional federated learning. Attached Figure Description

[0046] Figure 1 This is a schematic diagram of the overall structure and process;

[0047] Figure 2 This is a schematic diagram of the tag ID alignment process;

[0048] Figure 3 This is a schematic diagram of the model training process;

[0049] Figure 4 This is a schematic diagram of the label prediction process. Detailed Implementation

[0050] The technical solutions in the embodiments of the present invention are clearly and completely described in the following description. Obviously, the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of the present invention.

[0051] In the description of the embodiments of the present invention, it should be noted that the indicated orientation or positional relationship is based on the orientation or positional relationship shown in the accompanying drawings, or the orientation or positional relationship in which the product of the invention is conventionally placed during use, or the orientation or positional relationship in which those skilled in the art conventionally understand it during use. This is only for the convenience of describing the present invention and simplifying the description, and is not intended to indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation. Therefore, it should not be construed as a limitation of the present invention. Furthermore, the terms "first" and "second" are only used to distinguish descriptions and should not be construed as indicating or implying relative importance.

[0052] In the description of the embodiments of the present invention, it should also be noted that, unless otherwise explicitly specified and limited, the terms "set" and "connection" should be interpreted broadly. For example, they can refer to a fixed connection, a detachable connection, or an integral connection; they can refer to a direct connection or an indirect connection through an intermediate medium. Those skilled in the art can understand the specific meaning of the above terms in the present invention based on the specific circumstances.

[0053] Example 1

[0054] Embodiment 1 of the present invention discloses a training method for an inter-enterprise federated learning algorithm model based on tag-encrypted transmission, such as... Figure 1 As shown, in this embodiment, two clients are selected to participate in the training for illustration. The client holding the label is a credit business under the corresponding group enterprise, and the client not holding the label is an e-commerce business under the corresponding group enterprise. The central server (coordinator) is deployed on the group's central server.

[0055] The trained model is used to predict the loan amount of credit enterprise users; both the local training dataset and the local validation set contain multiple customer data, and each customer data contains multiple customer features and a label indicating the user's expected loan amount.

[0056] Specifically as follows:

[0057] S1: The coordinator initializes, acquires various training parameters, senses the number of participants, and begins to have the participants perform label ID alignment operations.

[0058] In this embodiment, the training parameters include training objectives, number of training rounds, and regularization parameters, etc.

[0059] After the coordinator is initialized, it distributes the training parameters to each participant and commands the participants to start the sample alignment work. In this embodiment, in order to improve computational efficiency, the gmpy2 library is used to accelerate the entire computation of the following steps.

[0060] The RSA encryption algorithm is used to align encrypted sample IDs. Specifically, the participant holding the target training label y acts as the Server, and the participant holding only the features and not the label acts as the Client.

[0061] The server-side sample ID set {s1, s2, ..., s k The sample ID set on the Client side is {c1, c2, ..., c}. n The server generates an RSA encryption key pair, including a private key (d, n) and a public key (e, n). The private key (d, n) is stored on the server side, and the public key (e, n) is sent to the client side.

[0062] The server uses its private key to calculate the final signature of its own sample ID, as follows:

[0063]

[0064] Among them, hs j =H(s) j H(), H′() represent hash functions; K s:j Indicates to hs j The result of RSA encryption using the server's private key d, t j It is the final signature of the sample ID on the server side;

[0065] The client generates a random number R for each sample ID. c (Greater than 1 and less than n, and coprime to n), and the ID is encrypted using RSA with the public key e and multiplied by a random number R. cA blind perturbation was performed, and the first lock was added:

[0066]

[0067] After completing the calculation, the encrypted data is sent to the server as the first data.

[0068] The server receives the first data sent by the client, performs a power-d operation on the data using its private key, and then adds a second lock.

[0069]

[0070] The server will transmit the data with the second lock and its own sample ID's final signature as the second data to the client to continue the unblinding process.

[0071]

[0072] Among them, K c:j It is hc j The result of RSA encryption using the server's private key d, t′ i Indicates to hc j The final signature of the encrypted result;

[0073] The client performs a crossover operation on the encrypted hash values ​​from both sides to obtain the sample alignment ID from both sides:

[0074]

[0075] Where, {a1,a2,…,a k The Client sends the aligned ID set to the Server. At this point, both parties have obtained the aligned ID set, and the Server sends a signal to the coordinator that the ID alignment is complete.

[0076] Based on the above steps, both parties obtained the intersection of their ID sets without disclosing the sample IDs, thus achieving sample ID alignment.

[0077] S2: After completing the sample ID alignment, the coordinator initiates model training, and all participants will use the shared entity data to train the machine learning model.

[0078] Specifically as follows:

[0079] The coordinator generates homomorphic encryption key pairs, stores the private key locally, and sends the public key to each participant;

[0080] The party holding the tag y uses the public key to encrypt the tag y they hold:

[0081] y encrypted [[y]]

[0082] Where [[y]] represents the homomorphic encryption of y, denoted as the first encrypted data, R y This indicates that after the tag holder completes the encryption, the tag holder will send [[y]] to other participants who do not hold tags;

[0083] The modeling iteration begins, with the label holder using their own data to complete the training;

[0084] Unlabeled participants use their own feature data to train and predict the results of labeled participants y. The specific label content can be encrypted by engineers during data preprocessing using mapping to prevent privacy leaks caused by unlabeled trainers using model results to calculate the original value of the encrypted label.

[0085] Other participants used encrypted tags to perform computational modeling and obtained the following intermediate results:

[0086] IntermediateResult i ]]=α(X+[[y]])

[0087] Where α() represents the computation required in the modeling process, X represents the unlabeled data possessed by the participants, X+[[y]] represents joint training using unlabeled data and encrypted labeled data, and i represents the number of iteration rounds;

[0088] The intermediate results are all parameters required by the model training methods selected by both parties, including but not limited to gradients, loss, etc.

[0089] After completing the calculation, the participants add a random mask to the encrypted intermediate results for obfuscation, and the blinded result is used as the second encrypted data. After being blinded, it is sent to the coordinator.

[0090] Data i ]]=[[IntermediateResult i ]]+R i

[0091] R i Indicates an additional random mask;

[0092] After receiving the second encrypted data, the coordinator uses the private key to decrypt the intermediate result:

[0093] Data i ]] decrypted Data i =IntermediateResult i +Ri

[0094] The decryption result is used as the third data and is sent back to the second encrypted data sender, i.e. the untagged participant.

[0095] The participants receive the decrypted results from the coordinator, i.e., the third data. After removing the random mask, they use the intermediate results to update their own model parameters.

[0096] Data i -R i =IntermediateResult i

[0097] λ i =β(λ i IntermediateResult i )

[0098] Where, λ i Let represent the model parameters in the i-th iteration, and β() represent the calculation of the updated model parameters;

[0099] After each training round, check whether the loss has reached the specified target or whether the number of training rounds has reached the upper limit. If not, continue iterating until the set model training objective is achieved, and each participant obtains their respective model {θ0, θ1, ..., θ}. n};

[0100] S3: Each participant predicts the label result corresponding to the ID, and the coordinator calculates the final predicted value based on the prediction results of each participant.

[0101] In this embodiment, the tag holder initiates a prediction task to the coordinator, and the coordinator starts executing the prediction task after obtaining the prediction task ID.

[0102] Specifically as follows:

[0103] The model is used to predict the target sample. The coordinator sends the target sample ID j to each participant, and each participant uses the model to predict the target sample ID.

[0104]

[0105] Where f() represents the predicted value calculated according to the model type, X i:j θ represents the local data corresponding to sample ID j in participant i. i This represents the model corresponding to the participant, where n represents the number of participants;

[0106] Each participant sends the prediction results to the coordinator. In this embodiment, each participant's ID, its own data and model are stored locally on each party's distributed server. Except for the calculation results, no other data is sent.

[0107] The coordinator calculates the arithmetic mean of the forecasts as the final forecast result:

[0108]

[0109] Among them, the predicted average value pred j This represents the final prediction result for the j-th sample, i.e., the result of the joint prediction by the model, where n represents the number of participants, and pred i:j This represents the prediction result of the i-th participant for the j-th sample.

[0110] The model training method of this invention can model and predict data from enterprises with many common samples across different business formats. It can be applied to various scenarios and predict various types of data.

[0111] The model training method described in this embodiment can save about 50% of the time cost while maintaining almost the same learning effect as traditional federated learning.

[0112] This invention is not limited to the specific embodiments described above. The invention extends to any new feature or combination disclosed in this specification, as well as any new method or process step or combination disclosed herein.

Claims

1. A training method for an inter-enterprise federated learning algorithm model based on tag-encrypted transmission, characterized in that, include: S1: The coordinator initializes, acquires various training parameters, detects the number of participants, and begins instructs participants to perform label ID alignment. The sample ID alignment method uses encrypted sample alignment with an RSA key and a hash function. The specific process is as follows: S101: The coordinator acquires various training parameters, perceives the number of participants, and sends label alignment messages to each participant; S102: The tag holder generates an RSA key pair, including a public key (e, n) and a private key (d, n), and uses the private key to calculate the final signature of its own sample ID; S103: Send the public key (e, n) to the unlabeled participant and store the private key locally; S104: Generate a random number that is coprime to n and not equal to 1 for each sample, encrypt the ID using the public key, multiply it by the random number and add blind perturbation, and use the encrypted data as the first data; S105: The unlabeled participant sends the first data to the label-holding sender; S106: Use the private key to perform a preliminary calculation to sign the first data, and use the data and its own final signature as the second data; S107: The tagged party sends the second data to the untagged participant; S108: Perform deblinding calculation on the second data and calculate the final signature. Perform an intersection operation on the final signatures of both parties to obtain the aligned ID set. S109: The unlabeled party sends the alignment ID set to the label-holding party, the label-holding party saves the alignment ID set, and sends a sample alignment completion signal to the coordinator; S110: After receiving the sample alignment completion signal, the coordinator sends the signal to start model training; S2: After label alignment is completed, the coordinator initiates model training. All participants will use the shared entity data to train the machine learning model. The specific process is as follows: S201: The coordinator generates a homomorphic encryption public key and a private key, stores the private key locally, and sends the public key to each participant; S202: The tag holder uses the public key to homomorphically encrypt tag y and uses this data as the first encrypted data, and at the same time begins to train its own model. S203: The tag holder sends the first encrypted data to the untagged participant; S204: Unlabeled participants use the first encrypted data as a label and combine it with their own data to train the model, obtain the required model parameters, and add a random mask to blind the data. The blinded result is then used as the second encrypted data. S205: Unlabeled participants send the second encrypted data to the coordinator for decryption; S206: The coordinator decrypts the second encrypted data using the private key and uses the decryption result as the third data; S207: The coordinator sends the decrypted third data to the sender of the second encrypted data; S208: Unlabeled participants remove the mask attached to the third data to obtain intermediate results of model parameters, and use the intermediate results to update various model parameters; S209: Each participant checks whether the training objective has been achieved. If not, they return to step S204 and repeat the process until all participants have completed model training. S3: Each participant predicts the label result corresponding to the ID, and the coordinator calculates the final predicted value based on the prediction results of each participant.

2. The inter-enterprise federated learning algorithm model training method according to claim 1, characterized in that, In step S1, the training parameters include the training objective, the number of training rounds, and the regularization parameters.

3. The inter-enterprise federated learning algorithm model training method according to claim 1, characterized in that, In step S2, all participants use the same machine learning model algorithm for model training.

4. The inter-enterprise federated learning algorithm model training method according to claim 1, characterized in that, Step S3, the specific process is as follows: S301: The coordinator obtains the predicted target ID and sends the ID to each participant; S302: Each participant uses the ID, its own data, and the model to predict the label result corresponding to the ID; S303: Each participating party sends its prediction results to the coordinator, who then uses these results to calculate the final prediction value.

5. The inter-enterprise federated learning algorithm model training method according to claim 4, characterized in that, Each participant's ID, data, and model are stored locally on their respective distributed servers. Except for the calculation results, no other data is sent.

6. The inter-enterprise federated learning algorithm model training method according to claim 4, characterized in that, The coordinator uses the following formula to calculate the final prediction result: in, Let represent the final prediction result for the j-th sample, and n represent the number of participants. This represents the prediction result of the i-th participant for the j-th sample.

Images