Federated Learning Modeling Optimization Method, Device, Medium and Product

Abstract

The present application discloses a method, device, medium and product for optimizing federated learning modeling, which is applied to a first party and includes: extracting the first party's samples to be trained and their corresponding preset true labels; based on the first party's bottom feature extraction model to be trained, converting the first party's training samples and the preset first party's bottom passport data into the first party's training intermediate output; based on the first party's training intermediate output, the preset true labels, the preset first party's top passport data and the top prediction model to be trained, jointly calculating the total model loss corresponding to the top prediction model to be trained with the second party's training intermediate output generated by each second party; based on the model total loss, optimizing the first party's bottom feature extraction model to be trained and the top prediction model to be trained to obtain the first party's bottom feature extraction model and the top prediction model. The present application solves the technical problem of low computational efficiency during federated learning modeling.

Description

Technical Field

[0001] This application relates to the field of artificial intelligence technology in financial technology (Fintech), and particularly to a method, device, medium, and product for optimizing federated learning modeling. Background Art

[0002] With the continuous development of financial technology, especially Internet technology finance, more and more technologies (such as distributed, artificial intelligence, etc.) are applied in the financial field. However, the financial industry also poses higher requirements for technologies, such as higher requirements for the distribution of to-do items corresponding to the financial industry.

[0003] With the continuous development of computer technology, the application of federated learning has become more and more extensive. Currently, in the process of federated learning, homomorphic encryption or multi-party secure computing is usually used to protect data privacy. However, multi-party secure computing involves complex cryptographic operations, so both communication and computing overheads are relatively large, resulting in low computing efficiency during federated learning. Moreover, the data complexity and data volume of the ciphertext data after homomorphic encryption are much larger than those of the plaintext data, so the computing overhead of homomorphic encryption is extremely large, resulting in low computing efficiency during federated learning. In addition, the homomorphic encryption method requires a large number of encryption and decryption processes during the federated learning process, which will further reduce the computing efficiency during federated learning. Summary of the Invention

[0004] The main purpose of this application is to provide a method, device, medium, and product for optimizing federated learning modeling, aiming to solve the technical problem of low computing efficiency during federated learning in the prior art.

[0005] To achieve the above purpose, this application provides a method for optimizing federated learning modeling. The method for optimizing federated learning modeling is applied to the first party, and the method for optimizing federated learning modeling includes:

[0006] Obtain the first-party bottom feature extraction model to be trained and the top prediction model to be trained, and extract the first-party samples to be trained and the preset true labels corresponding to the first-party training samples;

[0007] Based on the first-party bottom feature extraction model to be trained, jointly convert the first-party training samples and the preset first-party bottom passport data into a first-party training intermediate output with passport embeddings;

[0008] Based on the first - party training intermediate output, the preset true label, the preset first - party top - level passport data, and the to - be - trained top - level prediction model, through federated interaction with each second - party, and combining the second - party training intermediate output with passport embeddings generated by each second - party, calculate the total model loss corresponding to the to - be - trained top - level prediction model, where the second - party training intermediate output is obtained by the to - be - trained second - party bottom - level feature extraction model obtained by the second - party, and is obtained by transforming the preset second - party bottom - level passport data and the second - party training samples corresponding to the first - party training samples;

[0009] Based on the total model loss, optimize the to - be - trained first - party bottom - level feature extraction model and the to - be - trained top - level prediction model to obtain the first - party bottom - level feature extraction model and the top - level prediction model.

[0010] To achieve the above object, the present application provides a federated learning modeling optimization method. The federated learning modeling optimization method is applied to the second - party, and the federated learning modeling optimization method includes:

[0011] Obtain the to - be - trained second - party bottom - level feature extraction model, and extract second - party training samples;

[0012] Based on the to - be - trained second - party bottom - level feature extraction model, transform the second - party training samples and the preset second - party bottom - level passport data into a second - party training intermediate output;

[0013] Send the second - party training intermediate output to the first - party for the first - party to aggregate the second - party training intermediate outputs sent by each second - party, and based on the first - party training intermediate output obtained by transforming the first - party training samples and the preset first - party bottom - level passport data, obtain an aggregated training intermediate output, and based on the output prediction label obtained by transforming the aggregated training intermediate output and the preset first - party top - level passport data, and the preset true label corresponding to the first - party training samples, calculate the total model loss;

[0014] Receive the second - party gradient of the total model loss with respect to the second - party training intermediate output sent by the first - party, and based on the second - party gradient, optimize the to - be - trained second - party bottom - level feature extraction model to obtain the second - party bottom - level feature extraction model.

[0015] The present application provides a federated prediction optimization method. The federated prediction optimization method is applied to the first - party, and the federated prediction optimization method includes:

[0016] Obtain the first - party samples to be predicted, the preset first - party bottom - level passport data, and the preset first - party top - level passport data;

[0017] The first - party bottom - end feature extraction model constructed by federated learning based on the preset first - party bottom - end passport data converts the first - party sample to be predicted and the preset first - party bottom - end passport data into a first - party intermediate output with passport embeddings;

[0018] Receive the second - party intermediate outputs with passport embeddings sent by each second - party. Among them, the second - party intermediate output is obtained by the second - party bottom - end feature extraction model constructed by the second - party based on the preset second - party bottom - end passport data, which converts the preset second - party bottom - end passport data and the second - party sample to be predicted corresponding to the first - party sample to be predicted;

[0019] Aggregate the first - party passport - embedding intermediate output and each of the second - party passport - embedding intermediate outputs to obtain an aggregated passport - embedding intermediate output;

[0020] The top - end prediction model constructed by federated learning based on the preset first - party top - end passport data converts the aggregated passport - embedding intermediate output and the preset first - party top - end passport data into a target prediction result.

[0021] This application provides a federated prediction optimization method. The federated prediction optimization method is applied to the second - party, and the federated prediction optimization method includes:

[0022] Obtain the second - party sample to be predicted and the preset second - party bottom - end passport data;

[0023] The second - party bottom - end feature extraction model constructed by federated learning based on the preset second - party bottom - end passport data converts the second - party sample to be predicted and the preset second - party bottom - end passport data into a second - party intermediate output with passport embeddings;

[0024] Send the second - party intermediate output to the first - party for the first - party to generate a target prediction result through the first - party bottom - end feature extraction model constructed by federated learning based on the preset first - party bottom - end passport data and the top - end prediction model constructed by federated learning based on the preset first - party top - end passport data, based on the second - party intermediate outputs sent by each second - party, the first - party sample to be predicted corresponding to the second - party sample to be predicted, the preset first - party bottom - end passport data, and the preset first - party top - end passport data.

[0025] This application also provides a federated learning modeling optimization device. The federated learning modeling optimization device is a virtual device and is applied to the first - party. The federated learning modeling optimization device includes:

[0026] An extraction module, configured to obtain a first-party bottom feature extraction model to be trained and a top prediction model to be trained, and extract first-party samples to be trained and preset true labels corresponding to the first-party training samples;

[0027] A conversion module, configured to convert the first-party training samples and preset first-party bottom passport data into a first-party training intermediate output with passport embeddings based on the first-party bottom feature extraction model to be trained;

[0028] A calculation module, configured to calculate the total model loss corresponding to the top prediction model to be trained based on the first-party training intermediate output, the preset true labels, preset first-party top passport data, and the top prediction model to be trained, by performing federated interaction with each second party and aggregating the second-party training intermediate outputs with passport embeddings generated by each second party, where the second-party training intermediate output is obtained by converting preset second-party bottom passport data and second-party training samples corresponding to the first-party training samples using a second-party bottom feature extraction model to be trained obtained by the second party;

[0029] An optimization module, configured to optimize the first-party bottom feature extraction model to be trained and the top prediction model to be trained based on the total model loss, to obtain a first-party bottom feature extraction model and a top prediction model.

[0030] This application further provides a federated learning modeling optimization device. The federated learning modeling optimization device is a virtual device and is applied to a second party. The federated learning modeling optimization device includes:

[0031] An extraction module, configured to obtain a second-party bottom feature extraction model to be trained and extract second-party training samples;

[0032] A conversion module, configured to convert the second-party training samples and preset second-party bottom passport data into a second-party training intermediate output based on the second-party bottom feature extraction model to be trained;

[0033] A calculation module, configured to send the second-party training intermediate output to the first party for the first party to aggregate the second-party training intermediate outputs sent by each second party, and obtain an aggregated training intermediate output based on the first-party training intermediate output obtained by converting the first-party training samples and preset first-party bottom passport data, and calculate the total model loss based on the output prediction label obtained by converting the aggregated training intermediate output and preset first-party top passport data, and the preset true labels corresponding to the first-party training samples;

[0034] An optimization module, configured to receive the second-party gradient of the total loss of the model sent by the first party with respect to the intermediate output of the second-party training, and optimize the second-party bottom feature extraction model to be trained based on the second-party gradient, so as to obtain a second-party bottom feature extraction model.

[0035] The present application further provides a federated prediction optimization device. The federated prediction optimization device is a virtual device and is applied to the first party. The federated prediction optimization device includes:

[0036] An acquisition module, configured to acquire a first-party sample to be predicted, a preset first-party bottom passport data, and a preset first-party top passport data;

[0037] A first conversion module, configured to convert the first-party sample to be predicted and the preset first-party bottom passport data into a first-party intermediate output with passport embeddings based on the first-party bottom feature extraction model constructed by federated learning using the preset first-party bottom passport data;

[0038] A receiving module, configured to receive the second-party intermediate outputs with passport embeddings sent by each second party, where the second-party intermediate output is obtained by the second party converting the preset second-party bottom passport data and the second-party sample to be predicted corresponding to the first-party sample to be predicted based on the second-party bottom feature extraction model constructed by federated learning using the preset second-party bottom passport data;

[0039] An aggregation module, configured to aggregate the first-party passport embedding intermediate output and each of the second-party passport embedding intermediate outputs to obtain an aggregated passport embedding intermediate output;

[0040] A second conversion module, configured to convert the aggregated passport embedding intermediate output and the preset first-party top passport data into a target prediction result based on the top prediction model constructed by federated learning using the preset first-party top passport data.

[0041] The present application further provides a federated prediction optimization device. The federated prediction optimization device is a virtual device and is applied to the second party. The federated prediction optimization device includes:

[0042] An acquisition module, configured to acquire a second-party sample to be predicted and a preset second-party bottom passport data;

[0043] A conversion module, configured to convert the second-party sample to be predicted and the preset second-party bottom passport data into a second-party intermediate output with passport embeddings based on the second-party bottom feature extraction model constructed by federated learning using the preset second-party bottom passport data;

[0044] A sending module, configured to send the second-party intermediate output to the first party, so that the first party can generate a target prediction result based on the second-party intermediate outputs sent by each second party, the first-party samples to be predicted corresponding to the second-party samples to be predicted, the preset first-party bottom passport data, and the preset first-party top passport data, through a first-party bottom feature extraction model constructed by federated learning based on the preset first-party bottom passport data and a top prediction model constructed by federated learning based on the preset first-party top passport data.

[0045] This application also provides a federated learning modeling optimization device, which is an entity device. The federated learning modeling optimization device includes: a memory, a processor, and a program of the federated learning modeling optimization method stored on the memory and executable on the processor. When the program of the federated learning modeling optimization method is executed by the processor, the steps of the federated learning modeling optimization method as described above can be implemented.

[0046] This application also provides a federated prediction optimization device, which is an entity device. The federated prediction optimization device includes: a memory, a processor, and a program of the federated prediction optimization method stored on the memory and executable on the processor. When the program of the federated prediction optimization method is executed by the processor, the steps of the federated prediction optimization method as described above can be implemented.

[0047] This application also provides a medium, which is a readable storage medium. A program for implementing the federated learning modeling optimization method is stored on the readable storage medium. When the program of the federated learning modeling optimization method is executed by the processor, the steps of the federated learning modeling optimization method as described above can be implemented.

[0048] This application also provides a medium, which is a readable storage medium. A program for implementing the federated prediction optimization method is stored on the readable storage medium. When the program of the federated prediction optimization method is executed by the processor, the steps of the federated prediction optimization method as described above can be implemented.

[0049] This application also provides a product, which is a computer program product, including a computer program. When the computer program is executed by the processor, the steps of the federated learning modeling optimization method as described above can be implemented.

[0050] This application also provides a product, which is a computer program product, including a computer program. When the computer program is executed by the processor, the steps of the federated prediction optimization method as described above can be implemented.

[0051] The present application provides a method for optimizing federated learning modeling. Compared with the prior art that protects data privacy by means of homomorphic encryption or multi-party secure computation during the federated learning process, the present application first obtains a first-party bottom-end feature extraction model to be trained and a top-end prediction model to be trained, and extracts the first-party samples to be trained and the preset true labels corresponding to the first-party training samples. Then, based on the first-party bottom-end feature extraction model to be trained, the first-party training samples and the preset first-party bottom-end passport data are jointly converted into a first-party training intermediate output with passport embeddings. Further, based on the first-party training intermediate output, the preset true labels, the preset first-party top-end passport data, and the top-end prediction model to be trained, through federated interaction with each second party, the second-party training intermediate output with passport embeddings generated by each second party is combined, and the total model loss corresponding to the top-end prediction model to be trained is calculated. Among them, the second-party training intermediate output is obtained by the second-party bottom-end feature extraction model to be trained by the second party, and is obtained by converting the preset second-party bottom-end passport data and the second-party training samples corresponding to the first-party training samples. It should be noted that the second-party training intermediate output with passport embeddings is still plaintext data, and its data complexity and data volume are much smaller than those of homomorphic encrypted data. And because the passport in the preset second-party bottom-end passport data is embedded in the second-party training intermediate output, and the first party does not hold the preset second-party bottom-end passport data, even if the first party receives the second-party training intermediate output belonging to plaintext data, it cannot reverse-engineer the privacy data belonging to the second party. Then, based on the total model loss, the first-party bottom-end feature extraction model to be trained and the top-end prediction model to be trained are optimized to obtain the first-party bottom-end feature extraction model and the top-end prediction model, thereby achieving the purpose of constructing a federated model based on passport embeddings, protecting data privacy during the federated learning process, and without the need to protect data privacy by means of homomorphic encryption or multi-party secure computation, achieving the purpose of performing data security computation in plaintext state during federated learning, reducing the data computation complexity during the federated learning process, and avoiding a large number of data encryption and decryption processes during the federated learning process. Therefore, it overcomes the technical defects in the prior art that since multi-party secure computation involves complex cryptographic operations, the communication and computation overheads are both large, resulting in low computational efficiency during federated learning, and the data complexity and data volume of the ciphertext data after homomorphic encryption are much larger than those of plaintext data, so the computational overhead of homomorphic encryption is extremely large, resulting in low computational efficiency during federated learning, and the homomorphic encryption method needs to perform a large number of encryption and decryption processes during the federated learning process, which will further reduce the computational efficiency during federated learning. Therefore, the computational efficiency during federated learning is improved. Brief Description of the Drawings

[0052] The accompanying drawings herein are incorporated into and constitute a part of this specification, showing embodiments consistent with the present application, and together with the specification are used to explain the principles of the present application.

[0053] To more clearly illustrate the technical solutions in the embodiments of the present application or in the prior art, the following will briefly introduce the accompanying drawings required for use in the description of the embodiments or the prior art. Obviously, for those of ordinary skill in the art, without creative efforts, other drawings can also be obtained based on these drawings.

[0054] Figure 1 Schematic flowchart of the first embodiment of the federated learning modeling optimization method of the present application;

[0055] Figure 2 Schematic diagram of the to-be-trained top passport embedding module in the federated learning modeling optimization method of the present application;

[0056] Figure 3 Schematic structural diagram of the to-be-trained top passport embedding layer in the federated learning modeling optimization method of the present application;

[0057] Figure 4 Schematic flowchart of the vertical federated learning based on passport embedding in the federated learning modeling optimization method of the present application;

[0058] Figure 5 Schematic flowchart of the second embodiment of the federated learning modeling optimization method of the present application;

[0059] Figure 6 Schematic flowchart of the third embodiment of the federated learning modeling optimization method of the present application;

[0060] Figure 7 Schematic flowchart of the fourth embodiment of the federated learning modeling optimization method of the present application;

[0061] Figure 8 Schematic diagram of the device structure of the hardware operating environment involved in the federated learning modeling optimization method in the embodiments of the present application;

[0062] Figure 9 Schematic diagram of the device structure of the hardware operating environment involved in the federated prediction optimization method in the embodiments of the present application;

[0063] Figure 10 Schematic diagram of the hardware architecture of the federated learning involved in the solution of the embodiments of the present application.

[0064] The realization of the objectives, functional features, and advantages of the present application will be further described in conjunction with the embodiments with reference to the accompanying drawings. Detailed implementation manners

[0065] It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0066] An embodiment of the present application provides a method for optimizing federated learning modeling. In the first embodiment of the method for optimizing federated learning modeling of the present application, refer to Figure 1 , the method for optimizing federated learning modeling is applied to the first party, and the method for optimizing federated learning modeling includes:

[0067] Step S10, obtain the to-be-trained first-party bottom feature extraction model and the to-be-trained top prediction model, and extract the to-be-trained first-party samples and the preset true labels corresponding to the first-party training samples;

[0068] In this embodiment, it should be noted that the method for optimizing federated learning modeling is applied to the vertical federated learning scenario. The first party and the second party are both participants in vertical federated learning. The number of the first parties can be one or more. Among them, the to-be-trained first-party bottom feature extraction model, the to-be-trained top prediction model, and the training samples with preset true labels are set in the first party. The to-be-trained second-party bottom feature extraction model and the training samples without sample labels are set in the second party. Among them, the to-be-trained first-party bottom feature extraction model is an untrained neural network model in the first party, which is used to convert the training samples in the first party into an intermediate output with passport embedding. The to-be-trained second-party bottom feature extraction model is an untrained neural network model in the second party, which is used to convert the training samples in the second party into an intermediate output with passport embedding. The to-be-trained top prediction model is an untrained neural network model in the first party, which is used to convert the aggregation result of the output of the to-be-trained first bottom feature extraction model and the outputs of the to-be-trained second bottom feature extraction models into an output prediction label. The preset true label is the identifier of the to-be-trained first-party samples, which is used to identify information such as the category or attribute of the to-be-trained first-party samples.

[0069] Step S20, based on the to-be-trained first-party bottom feature extraction model, jointly convert the first-party training samples and the preset first-party bottom passport data into a first-party training intermediate output with passport embedding;

[0070] In this embodiment, it should be noted that the preset first-party bottom passport data at least includes a first-party passport sample to be embedded. Among them, the first-party passport sample to be embedded is a pre-set sample for passport embedding in the first-party bottom feature extraction model to be trained. Among them, the first-party passport sample to be embedded can be a picture, text, or a specific coding matrix, etc. The first-party bottom feature extraction model to be trained at least includes a first-party passport embedding module to be trained, which is used to embed the first-party passport sample to be embedded into the intermediate output of the model of the first-party training sample in the first-party bottom feature extraction model to be trained, and jointly convert it with the first-party training sample into the final output of the model. Among them, the intermediate output of the model is the output of the intermediate network layer of the model, and the final output of the model is the output generated by the output layer of the model. For example, assuming that the neural network model A includes an input layer, 10 hidden layers, and an output layer, the output of the input layer and the outputs of the 10 hidden layers are all the intermediate outputs of the model.

[0071] Specifically, after inputting the first-party training sample and the preset first-party bottom passport data into the first-party bottom feature extraction model to be trained, through each first-party passport embedding module to be trained in the first-party bottom feature extraction model to be trained, respectively embed the first-party passport sample to be embedded corresponding to each first-party passport embedding module to be trained into the intermediate output of the model corresponding to the first-party training sample. After the model prediction is completed, the output layer of the first-party bottom feature extraction model to be trained outputs the first-party training intermediate output with passport embedding. For example, assuming that the first-party bottom feature extraction model to be trained includes 2 first-party passport embedding modules A and B to be trained. Among them, the first-party passport embedding module A to be trained is set between the 3rd hidden layer and the 4th hidden layer of the first-party bottom feature extraction model to be trained. Then, through the first-party passport embedding module A to be trained, embed the first-party passport sample to be embedded corresponding to A into the 3rd output of the 3rd hidden layer, and use the 3rd output after embedding the first-party passport sample to be embedded corresponding to A as the input of the 4th hidden layer. The first-party passport embedding module B to be trained is set between the 40th hidden layer and the 41st hidden layer of the first-party bottom feature extraction model to be trained. Then, through the first-party passport embedding module B to be trained, embed the first-party passport sample to be embedded corresponding to B into the 40th output of the 40th hidden layer, and use the 40th output after embedding the first-party passport sample to be embedded corresponding to B as the input of the 41st hidden layer. Finally, the output layer of the first-party bottom feature extraction model to be trained outputs the first-party training intermediate output.

[0072] Among them, the first-party bottom feature extraction model to be trained includes a first-party bottom neural network to be trained, and the first-party bottom neural network to be trained includes at least a first-party passport embedding module to be trained. The preset first-party bottom passport data includes at least a first-party passport sample to be embedded corresponding to the first-party passport embedding module to be trained.

[0073] The step of jointly converting the first-party training sample and the preset first-party bottom passport data into a first-party training intermediate output with passport embedding based on the first-party bottom feature extraction model to be trained includes:

[0074] Step S21: Based on the first part of the first-party bottom neural network to be trained in the first-party bottom neural network to be trained, which is before the first-party passport embedding module to be trained, convert the first-party training sample into an intermediate output of the first-party bottom network to be trained;

[0075] In this embodiment, it should be noted that the first part of the first-party bottom neural network to be trained is the part of the neural network corresponding to the model intermediate output that is generated as the input of the first-party passport embedding module to be trained in the first-party bottom neural network to be trained, and can be used to convert the first-party training sample into a partial input of the first-party passport embedding module to be trained.

[0076] Specifically, after the first-party training sample is input into the first-party bottom neural network to be trained, through the first part of the first-party bottom neural network to be trained in the first-party bottom neural network to be trained, which is before the first-party passport embedding module to be trained, the first-party training sample is mapped into an intermediate output of the first-party bottom network to be trained.

[0077] Step S22: Based on the first-party passport embedding module to be trained, jointly convert the intermediate output of the first-party bottom network to be trained and the first-party passport sample to be embedded into an output of the first-party bottom passport embedding module to be trained;

[0078] In this embodiment, specifically, based on the first-party passport embedding module to be trained, by embedding the first-party passport sample to be embedded into the embedding module output generated by the first-party passport embedding module to be trained for the intermediate output of the first-party bottom network to be trained, an output of the first-party bottom passport embedding module to be trained is obtained.

[0079] Furthermore, the first-party passport embedding module to be trained includes a first-party passport embedding layer and a first-party module neural network layer. Step S22 further includes:

[0080] Based on the neural network layer of the first-party module to be trained, map the intermediate output of the bottom network of the first-party to be trained to the output to be embedded by the first party, and map the passport sample to be embedded by the first party to the passport to be embedded by the first party. Then, based on the passport embedding function in the passport embedding layer of the first-party module to be trained, convert the passport to be embedded into the passport embedding parameters of the first party. Furthermore, based on the passport embedding parameters of the first party, perform passport embedding on the output to be embedded to obtain the output of the bottom passport embedding module of the first party to be trained. Among them, the process of performing passport embedding on the output to be embedded based on the passport embedding parameters of the first party to obtain the output of the bottom passport embedding module of the first party to be trained is as follows:

[0081] X = γ * X t + β

[0082] Where X is the output of the bottom passport embedding module of the first party to be trained, and X t is the output to be embedded, and γ and β are the passport embedding parameters of the first party.

[0083] Step S23: Based on the second part of the bottom neural network of the first party to be trained that is after the bottom passport embedding module of the first party to be trained, convert the output of the bottom passport embedding module of the first party to be trained into the intermediate output of the first-party training.

[0084] In this embodiment, it should be noted that the second part of the bottom neural network of the first party to be trained is the part of the neural network that takes the output of the bottom passport embedding module of the first party to be trained as the input, and can be used to convert the output of the bottom passport embedding module of the first party to be trained into the intermediate output of the first-party training.

[0085] Specifically, map the output of the bottom passport embedding module of the first party to be trained to the intermediate output of the first-party training through the second part of the bottom neural network of the first party to be trained that is after the bottom passport embedding module of the first party to be trained.

[0086] Step S30: Based on the intermediate output of the first party, the preset true label, the preset top passport data of the first party, and the top prediction model to be trained, through federated interaction with each second party, calculate the total model loss corresponding to the top prediction model to be trained by combining the intermediate output of the second-party training with passport embedding generated by each second party, where the intermediate output of the second-party training is obtained by the bottom feature extraction model of the second party to be trained for converting the preset bottom passport data of the second party and the second-party training samples corresponding to the first-party training samples;

[0087] In this embodiment, it should be noted that a second-party bottom feature extraction model to be trained is set in each of the second parties, and each second party will, based on the second-party bottom feature extraction model to be trained, convert the second-party training samples corresponding to the first-party training samples and the preset second-party bottom passport data into second-party training intermediate outputs. Among them, the second-party training samples and the first-party training samples correspond to the same common sample ID, and the common sample ID is determined when the first party aligns samples with each second party. The specific process of the second party generating the second-party training intermediate outputs can refer to the content in step S20 and its detailed steps, which will not be elaborated here.

[0088] Specifically, receive the second-party training intermediate outputs sent by each second party, and based on a preset aggregation method, aggregate the first-party training intermediate outputs and the second-party training intermediate outputs of each second party to generate an aggregated training intermediate output. Then, based on the top prediction model to be trained, jointly convert the aggregated training intermediate output and the preset first-party top passport data into output prediction labels, and further calculate the total model loss based on the output prediction labels and the preset true labels. Among them, the preset aggregation methods include weighted average and summation, etc.

[0089] Among them, the step of calculating the total model loss corresponding to the top prediction model to be trained by federally interacting with each second party based on the first-party training intermediate output, the preset true label, the preset first-party top passport data, and the top prediction model to be trained, and jointly generating the second-party training intermediate outputs with passport embeddings by each second party includes:

[0090] Step S31, receive the second-party training intermediate outputs with passport embeddings sent by each second party;

[0091] In this embodiment, it should be noted that after the second party inputs the second-party training samples and the preset second-party bottom passport data into the second-party bottom feature extraction model to be trained, through each second-party passport embedding module in the second-party bottom feature extraction model to be trained, respectively embed the to-be-embedded passport samples corresponding to each second-party passport embedding module in the preset second-party bottom passport data into the model intermediate output corresponding to the second-party training samples. After the model prediction is completed, the output layer of the second-party bottom feature extraction model to be trained outputs the second-party training intermediate outputs with passport embeddings. Among them, since the second party embeds the preset second-party bottom passport data when generating the second-party training intermediate outputs, when the first party receives the second-party training intermediate outputs, because the first party does not hold the preset second-party bottom passport data, it is impossible to reverse-infer the first party's private data, such as model parameters and samples, through the second-party training intermediate outputs, achieving the purpose of protecting data privacy.

[0092] Step S32: Aggregate the first - party training intermediate output and each of the second - party training intermediate outputs to obtain an aggregated training intermediate output;

[0093] In this embodiment, specifically, based on a preset aggregation method, the first - party training intermediate output and each of the second - party training intermediate outputs are aggregated into an aggregated training intermediate output, where the preset aggregation methods include weighted average and summation, etc.

[0094] Step S33: Through the to - be - trained top - end prediction model, jointly convert the aggregated training intermediate output and the preset first - party top - end passport data into an output prediction label;

[0095] In this embodiment, specifically, after inputting the aggregated training intermediate output and the preset first - party top - end passport data into the to - be - trained top - end prediction model, through each to - be - trained top - end passport embedding module in the to - be - trained top - end prediction model, respectively embed the top - end passport embedding samples corresponding to each to - be - trained top - end passport embedding module in the preset first - party top - end passport data into the model intermediate output corresponding to the aggregated training intermediate output, and after the model prediction is completed, the output layer of the to - be - trained top - end prediction model outputs the output prediction label.

[0096] Among them, the to - be - trained top - end prediction model includes a to - be - trained top - end neural network, the to - be - trained top - end neural network includes at least one to - be - trained top - end passport embedding module, and the preset first - party top - end passport data includes at least one top - end passport embedding sample corresponding to the to - be - trained top - end passport embedding module.

[0097] The step of jointly converting the aggregated training intermediate output and the preset first - party top - end passport data into an output prediction label through the to - be - trained top - end prediction model includes:

[0098] Step S331: Based on the first part of the to - be - trained top - end neural network in the to - be - trained top - end neural network that is before the to - be - trained top - end passport embedding module, convert the aggregated training intermediate output into a to - be - trained top - end network intermediate output;

[0099] In this embodiment, it should be noted that the first part of the to - be - trained top - end neural network is the part of the neural network corresponding to the model intermediate output that serves as the input of the to - be - trained top - end passport embedding module in the to - be - trained top - end neural network, and is used to convert the aggregated training intermediate output into part of the input of the to - be - trained top - end passport embedding module.

[0100] Specifically, after inputting the aggregated training intermediate output into the to-be-trained top neural network, the aggregated training intermediate output is mapped to the to-be-trained top network intermediate output through the first part of the to-be-trained top neural network before the to-be-trained top passport embedding module in the to-be-trained top neural network.

[0101] Step S332, based on the to-be-trained top passport embedding module, jointly convert the to-be-trained top network intermediate output and the top passport embedding sample into the to-be-trained top passport embedding module output;

[0102] In this embodiment, specifically, based on the to-be-trained top passport embedding module, the to-be-trained top passport embedding module output is obtained by embedding the top passport embedding sample into the embedding module output generated by the to-be-trained top passport embedding module for the to-be-trained top network intermediate output.

[0103] Wherein, the to-be-trained top passport embedding module includes a to-be-trained top passport embedding layer and a to-be-trained top module neural network layer,

[0104] The step of jointly converting the to-be-trained top network intermediate output and the top passport embedding sample into the to-be-trained top passport embedding module output based on the to-be-trained top passport embedding module includes:

[0105] Step A10, based on the to-be-trained top module neural network layer, linearly transform the to-be-trained network intermediate output into the to-be-embedded training network layer output, and linearly transform the top passport embedding sample into the top to-be-embedded passport;

[0106] In this embodiment, specifically, based on the to-be-trained top module neural network layer, linear transformations are performed on the to-be-trained network intermediate output and the top passport embedding sample to obtain the to-be-embedded training network layer output corresponding to the to-be-trained network intermediate output, and the top to-be-embedded passport corresponding to the top passport embedding sample.

[0107] Step A20, based on the passport function in the first-party passport embedding layer, convert the top to-be-embedded passport into passport embedding parameters;

[0108] In this embodiment, it should be noted that the passport function is a pre-set function for generating passport embedding parameters based on the top to-be-embedded passport.

[0109] Specifically, input the top to-be-embedded passport into the passport function in the passport embedding layer, and output the passport embedding parameters corresponding to the top to-be-embedded passport.

[0110] Step A30: Generate the output of the to-be-trained top passport embedding module based on the output of the to-be-embedded training network layer and the passport embedding parameters.

[0111] In this embodiment, it should be noted that the passport embedding parameters include a first passport embedding parameter and a second passport embedding parameter.

[0112] Specifically, perform passport embedding on the output of the to-be-embedded training network layer by calculating the target product between the output of the to-be-embedded training network layer and the first passport embedding parameter, and the sum between the target product and the second passport embedding parameter, to generate the output of the to-be-trained top passport embedding module, as Figure 2 shown in the schematic diagram of the to-be-trained top passport embedding module, where the passport embedding layer is the to-be-trained top passport embedding layer, the neural network layer is the to-be-trained top module neural network layer, X is the intermediate output of the to-be-trained network, P is the top passport embedding sample, X' is the output of the to-be-embedded training network layer, P' is the top to-be-embedded passport, and X~ is the output of the to-be-trained top passport embedding module.

[0113] Step S333: Based on the second part of the to-be-trained top neural network in the to-be-trained top neural network that is after the to-be-trained top passport embedding module, convert the output of the to-be-trained top passport embedding module into the output prediction label.

[0114] In this embodiment, it should be noted that the second part of the to-be-trained top neural network is the part of the to-be-trained top neural network that takes the output of the to-be-trained top passport embedding module as the input, and is used to convert the output of the to-be-trained top passport embedding module into the output prediction label.

[0115] Further, in an implementable manner, as Figure 3 shown in the schematic diagram of the structure of the to-be-trained top passport embedding layer, where X' is the output of the to-be-embedded training network layer, P' is the top to-be-embedded passport, γ and β are the passport embedding parameters, and the data transformation layer is used to perform data transformation on the output of the to-be-embedded training network layer to generate X t , X~ is the output of the to-be-trained top passport embedding module. In another implementable manner, the data transformation layer in the passport embedding layer in Figure 3 can be removed, and X' is used to replace X t to participate in the calculation.

[0116] Step S34: Calculate the total loss of the model based on the output prediction label and the preset true label.

[0117] In this embodiment, based on the output prediction label and the preset true label, the total model loss is calculated. Specifically, the difference between the output prediction label and the preset true label is used as the total model loss.

[0118] Wherein, after the step of calculating the total model loss corresponding to the to-be-trained top prediction model based on the first-party training intermediate output, the preset true label, the preset first-party top passport data, and the second-party training intermediate output with passport embeddings generated by jointly interacting with each second party through federated interaction with each second party, the federated learning modeling optimization method further includes:

[0119] Step B10, calculating the second-party gradients of the total model loss with respect to each second-party training intermediate output;

[0120] In this embodiment, specifically, the partial derivatives of the total model loss with respect to each second-party training intermediate output are obtained to obtain the second-party gradients corresponding to each second-party training intermediate output.

[0121] Step B20, sending each second-party gradient to its corresponding second party respectively, so that the second party optimizes the to-be-trained second-party bottom feature extraction model based on the second-party gradient to obtain the second-party bottom feature extraction model.

[0122] In this embodiment, specifically, each second-party gradient is sent to its corresponding second party respectively, so that the second party updates the to-be-trained second-party bottom feature extraction model based on the second-party gradient through a preset model optimization method, and determines whether the updated to-be-trained second-party bottom feature extraction model meets the preset training end condition. If the updated to-be-trained second-party bottom feature extraction model meets the preset training end condition, the to-be-trained second-party bottom feature extraction model is used as the second-party bottom feature extraction model. If the updated to-be-trained second-party bottom feature extraction model does not meet the preset training end condition, the step of generating the second-party training intermediate output is returned. Among them, the preset model optimization method includes gradient ascent method and gradient descent method, etc., and the preset training end condition includes convergence of the loss function and reaching the maximum iteration number threshold, etc. Since the first party embeds the preset first-party top passport data when generating the output prediction label, when the second party obtains the second-party gradient corresponding to the total model loss generated based on the output prediction label, since the second party does not hold the preset first-party bottom passport data and the preset first-party top passport data, the second party cannot reverse the first party's private data, such as model parameters and samples, based on the second-party gradient, achieving the purpose of protecting data privacy.

[0123] Step S40: Optimize the to-be-trained first-party bottom feature extraction model and the to-be-trained top prediction model based on the total model loss to obtain the first-party bottom feature extraction model and the top prediction model.

[0124] In this embodiment, specifically, based on the total model loss, calculate the first-party bottom model gradient corresponding to the to-be-trained first-party bottom feature extraction model and the first-party top model gradient corresponding to the to-be-trained top prediction model. Then, update the to-be-trained first-party bottom feature extraction model according to the first-party bottom model gradient, and update the to-be-trained top prediction model according to the first-party top model gradient. Then, determine whether both the to-be-trained first-party bottom feature extraction model and the to-be-trained top prediction model meet the preset training end condition. If both the to-be-trained first-party bottom feature extraction model and the to-be-trained top prediction model meet the preset training end condition, then use the to-be-trained first-party bottom feature extraction model as the first-party bottom feature extraction model, and use the to-be-trained top prediction model as the top prediction model. If both the to-be-trained first-party bottom feature extraction model and the to-be-trained top prediction model do not meet the preset training end condition, then return to execute the step of extracting the first-party to-be-trained samples and the preset true labels corresponding to the first-party training samples, as Figure 4 shown in the flowchart of vertical federated learning based on passport embedding. Among them, Party 1 is the first party, and Parties 2 to N are each of the second parties. is the intermediate output of the first-party training. to are the intermediate outputs of the training of each of the second parties, Z~ is the aggregated training intermediate output, the model with as the input is the to-be-trained top prediction model, the model with as the output is the to-be-trained first-party bottom feature extraction model, the models with to as the output are the to-be-trained bottom feature extraction models of each of the second parties, L is the total model loss, P l N represents the l-th passport embedding module of Party N. All the passport embedding network modules are passport embedding modules. X1 is the first-party training sample, and X2 to X N are the training samples of each of the second parties.

[0125] Among them, after the step of optimizing the to-be-trained first-party bottom feature extraction model and the to-be-trained top prediction model based on the total model loss to obtain the first-party bottom feature extraction model and the top prediction model, the federated learning modeling optimization method further includes:

[0126] Step C10: Obtain the first-party sample to be predicted, and based on the first-party bottom feature extraction model, jointly convert the first-party sample to be predicted and the preset first-party bottom passport data into a first-party intermediate output with passport embeddings;

[0127] In this embodiment, it should be noted that the first-party bottom feature extraction model at least includes a first-party passport embedding module, and the preset first-party bottom passport data at least includes a first-party passport sample to be embedded, where the first-party passport embedding module and the first-party passport sample to be embedded correspond one by one.

[0128] Specifically, after obtaining the first-party sample to be predicted and inputting the first-party sample to be predicted and the preset first-party bottom passport data into the first-party bottom feature extraction model, through each first-party passport embedding module in the first-party bottom feature extraction model, respectively embed the first-party passport sample to be embedded corresponding to each first-party passport embedding module in the preset first passport data into the intermediate output of the model corresponding to the first-party sample to be predicted, and after the model prediction is completed, the output layer of the first-party bottom feature extraction model outputs a first-party intermediate output with passport embeddings.

[0129] Step C20: Receive the second-party intermediate outputs with passport embeddings sent by each second party, where the second-party intermediate output is obtained by the second party based on the second-party bottom feature extraction model to convert the preset second-party bottom passport data and the second-party sample to be predicted corresponding to the first-party sample to be predicted;

[0130] In this embodiment, it should be noted that each second party is provided with a second-party bottom feature extraction model, and the second party will convert the second-party sample to be predicted corresponding to the first-party sample to be predicted and the preset second-party bottom passport data into a second-party intermediate output based on the second-party bottom feature extraction model. The specific process of the second party generating the second-party intermediate output can refer to the content of step C20 and will not be elaborated here. Although the second-party intermediate output is plaintext data, when generating the second-party intermediate output, the preset second-party bottom passport data is embedded. Therefore, since the first party does not hold the preset second-party bottom passport data, the first party cannot reverse-engineer the second party's private data based on the second-party intermediate output.

[0131] Step C30: Aggregate the first-party intermediate output and each second-party intermediate output to obtain an aggregated intermediate output;

[0132] In this embodiment, specifically, aggregate the first-party intermediate output and each second-party intermediate output into an aggregated intermediate output based on a preset aggregation method.

[0133] Step C40: Based on the top prediction model, jointly convert the aggregated intermediate output and the preset first-party top passport data into a target prediction result.

[0134] In this embodiment, specifically, after inputting the aggregated intermediate output and the preset first-party top passport data into the top prediction model, through each second-party passport embedding module in the top prediction model, respectively embed the top passport embedding samples corresponding to each second-party passport embedding module in the preset first-party top passport data into the model intermediate output corresponding to the aggregated intermediate output, and after the model prediction is completed, the output layer of the top prediction model outputs the target prediction result.

[0135] The embodiment of the present application provides a method for optimizing federated learning modeling. Compared with the prior art that protects data privacy by means of homomorphic encryption or multi-party secure computation during the federated learning process, the embodiment of the present application first obtains a first-party bottom feature extraction model to be trained and a top prediction model to be trained, and extracts the first-party samples to be trained and the preset true labels corresponding to the first-party training samples. Then, based on the first-party bottom feature extraction model to be trained, the first-party training samples and the preset first-party bottom passport data are jointly converted into a first-party training intermediate output with passport embeddings. Further, based on the first-party training intermediate output, the preset true labels, the preset first-party top passport data, and the top prediction model to be trained, through federated interaction with each second party, the second-party training intermediate output with passport embeddings generated by each second party is combined, and the total model loss corresponding to the top prediction model to be trained is calculated. Among them, the second-party training intermediate output is obtained by the second-party bottom feature extraction model to be trained obtained by the second party, and is obtained by converting the preset second-party bottom passport data and the second-party training samples corresponding to the first-party training samples. It should be noted that the second-party training intermediate output with passport embeddings is still plaintext data, and its data complexity and data volume are much smaller than those of homomorphic encrypted data. Moreover, since the passport in the preset second-party bottom passport data is embedded in the second-party training intermediate output, and the first party does not hold the preset second-party bottom passport data, even if the first party receives the second-party training intermediate output that belongs to plaintext data, it cannot reverse-engineer the second-party privacy data. Then, based on the total model loss, the first-party bottom feature extraction model to be trained and the top prediction model to be trained are optimized to obtain the first-party bottom feature extraction model and the top prediction model, thereby achieving the purpose of constructing a federated model based on passport embeddings, protecting data privacy during the federated learning process, and without the need to protect data privacy by means of homomorphic encryption or multi-party secure computation, achieving the purpose of performing data security computation in plaintext state during federated learning, reducing the data computation complexity during the federated learning process, and avoiding a large number of data encryption and decryption processes during the federated learning process. Therefore, it overcomes the technical defects in the prior art that since multi-party secure computation involves complex cryptographic operations, the communication and computation overheads are relatively large, resulting in low computation efficiency during federated learning, and the data complexity and data volume of the ciphertext data after homomorphic encryption are much larger than those of plaintext data, so the computation overhead of homomorphic encryption is extremely large, resulting in low computation efficiency during federated learning, and the homomorphic encryption method needs to perform a large number of encryption and decryption processes during the federated learning process, which will further reduce the computation efficiency during federated learning. Therefore, the computation efficiency during federated learning is improved.

[0136] Further, referring to Figure 5, in another embodiment of the present application, the federated learning modeling optimization method is applied to a second party, and the federated learning modeling optimization method includes:

[0137] Step D10, obtain the second-party bottom feature extraction model to be trained, and extract the second-party training samples;

[0138] In this embodiment, it should be noted that the federated learning modeling optimization method is applied to the vertical federated learning scenario. The first party and the second party are both participants in vertical federated learning. The introductions of the first party and the second party can specifically refer to the corresponding embodiment content part of step S10, which will not be elaborated here.

[0139] Step D20, based on the second-party bottom feature extraction model to be trained, convert the second-party training samples and the preset second-party bottom passport data into the second-party training intermediate output;

[0140] In this embodiment, it should be noted that the preset second-party bottom passport data includes at least one second-party passport sample to be embedded. Among them, the second-party passport sample to be embedded is a pre-set sample for passport embedding in the second-party bottom feature extraction model to be trained. Among them, the second-party passport sample to be embedded can be a picture sample, a text sample, or a specific coding matrix, etc. The second-party bottom feature extraction model to be trained includes at least one second-party passport embedding module to be trained, which is used to embed the second-party passport sample to be embedded into the intermediate output of the model of the second-party training samples, and jointly convert it into the final output of the model with the second-party training samples. Among them, the intermediate output of the model is the output of the intermediate network layer of the model, and the final output of the model is the output generated by the output layer. For example, assume that the neural network model A includes an input layer, 10 hidden layers, and an output layer. Then the output of the input layer and the output of the 10 hidden layers are both the intermediate output of the model.

[0141] Specifically, after inputting the second-party training samples and the preset second-party bottom passport data into the second-party bottom feature extraction model to be trained, through each second-party passport embedding module to be trained in the second-party bottom feature extraction model to be trained, each second-party passport sample to be embedded corresponding to each second-party passport embedding module to be trained is respectively embedded into the intermediate output of the model corresponding to the second-party training samples. After the model prediction is completed, the output layer of the second-party bottom feature extraction model to be trained outputs the second-party training intermediate output with passport embedding. For example, assume that the second-party bottom feature extraction model to be trained includes two second-party passport embedding modules A and B to be trained. Among them, the second-party passport embedding module A to be trained is arranged between the 3rd hidden layer and the 4th hidden layer of the second-party bottom feature extraction model to be trained. Then, through the second-party passport embedding module A to be trained, the second-party passport sample to be embedded corresponding to A is embedded into the 3rd output of the 3rd hidden layer, and the 3rd output after embedding the second-party passport sample corresponding to A is used as the input of the 4th hidden layer. The second-party passport embedding module B to be trained is arranged between the 40th hidden layer and the 41st hidden layer of the second-party bottom feature extraction model to be trained. Then, through the second-party passport embedding module B to be trained, the second-party passport sample to be embedded corresponding to B is embedded into the 40th output of the 40th hidden layer, and the 40th output after embedding the second-party passport sample corresponding to B is used as the input of the 41st hidden layer. Finally, the output layer of the second-party bottom feature extraction model to be trained outputs the second-party training intermediate output.

[0142] Among them, the second-party bottom feature extraction model to be trained includes a second-party bottom neural network to be trained. The second-party bottom neural network to be trained includes at least one second-party passport embedding module to be trained. The preset second-party bottom passport data includes at least one second-party passport sample to be embedded corresponding to the second-party passport embedding module to be trained.

[0143] The step of converting the second-party training samples and the preset second-party bottom passport data into the second-party training intermediate output based on the second-party bottom feature extraction model to be trained includes:

[0144] Step D21, based on the first part of the second-party bottom neural network to be trained in the second-party bottom neural network to be trained that is before the second-party passport embedding module to be trained, convert the second-party training samples into the second-party bottom network intermediate output to be trained;

[0145] In this embodiment, it should be noted that the first part of the second-party bottom neural network to be trained is the part of the neural network corresponding to the intermediate output of the model that serves as the input of the second-party passport embedding module in the second-party bottom neural network to be trained, and is used to convert the second-party training samples into part of the input of the second-party passport embedding module to be trained.

[0146] Specifically, after the second-party training samples are input into the second-party bottom neural network to be trained, the first part of the second-party bottom neural network to be trained before the second-party passport embedding module in the second-party bottom neural network to be trained maps the second-party training samples to the intermediate output of the second-party bottom network to be trained.

[0147] Step D22: Based on the second-party passport embedding module to be trained, jointly convert the intermediate output of the second-party bottom network to be trained and the second-party passport sample to be embedded into the output of the second-party bottom passport embedding module to be trained;

[0148] In this embodiment, specifically, based on the second-party passport embedding module to be trained, the output of the second-party bottom passport embedding module to be trained is obtained by embedding the second-party passport sample to be embedded into the embedding module output generated by the second-party passport embedding module to be trained for the intermediate output of the second-party bottom network to be trained.

[0149] Further, the second-party passport embedding module to be trained includes a second-party passport embedding layer and a second-party module neural network layer. Step D22 further includes:

[0150] Based on the second-party module neural network layer, map the intermediate output of the second-party bottom network to be trained to the second-party output to be embedded, and map the second-party passport sample to be embedded to the second-party passport to be embedded. Then, based on the passport embedding function in the second-party passport embedding layer, convert the second-party passport to be embedded into second-party embedding passport parameters. Then, based on the second-party embedding passport parameters, perform passport embedding on the second-party output to be embedded to obtain the output of the second-party bottom passport embedding module to be trained. Among them, the process of performing passport embedding on the second-party output to be embedded based on the second-party embedding passport parameters to obtain the output of the second-party bottom passport embedding module to be trained is as follows:

[0151] X = γ * X t + β

[0152] where X is the output of the second-party bottom passport embedding module to be trained, X t is the second-party output to be embedded, and γ and β are the second-party passport embedding parameters.

[0153] Step D23: Based on the second part of the to-be-trained second-party bottom neural network that is after the to-be-trained second-party passport embedding module in the to-be-trained second-party bottom neural network, convert the output of the to-be-trained second-party bottom passport embedding module into the second-party training intermediate output.

[0154] In this embodiment, it should be noted that the second part of the to-be-trained second-party bottom neural network is the part of the neural network in the to-be-trained second-party bottom neural network that takes the output of the to-be-trained second-party passport embedding module as the input, and is used to convert the output of the to-be-trained second-party passport embedding module into the second-party training intermediate output.

[0155] Specifically, through the second part of the to-be-trained second-party bottom neural network that is after the to-be-trained second-party passport embedding module in the to-be-trained second-party bottom neural network, map the output of the to-be-trained second-party bottom passport embedding module to the second-party training intermediate output.

[0156] Step D30: Send the second-party training intermediate output to the first party, so that the first party aggregates the second-party training intermediate outputs sent by each second party, and based on the first-party training samples and the first-party training intermediate output obtained by converting the preset first-party bottom passport data, obtain the aggregated training intermediate output, and based on the aggregated training intermediate output and the output prediction label obtained by converting the preset first-party top passport data, as well as the preset true label corresponding to the first-party training sample, calculate the total model loss;

[0157] In this embodiment, specifically, send the second-party training intermediate output to the first party, so that the first party aggregates the generated first-party training output and the second-party training intermediate outputs sent by each second party to obtain the aggregated training intermediate output. Then, according to the to-be-trained top prediction model, jointly convert the aggregated training intermediate output and the preset first-party top passport data into the output prediction label. Then, based on the output prediction label and the preset true label, calculate the total model loss. Furthermore, the first party optimizes the to-be-trained first-party bottom feature extraction model and the to-be-trained top prediction model based on the total model loss to obtain the first-party bottom feature extraction model and the top prediction model, and calculate the respective second-party gradients for each second-party training intermediate output. Then, the first party sends each of the second-party gradients to their corresponding second parties. Among them, the specific processes of the first party generating the first-party training intermediate output, generating the total model loss, constructing the first-party bottom feature extraction model, and constructing the top prediction model can refer to the content in Steps S10 to S40 and their refinement steps, which will not be elaborated here.

[0158] Step D40: Receive the second-party gradient of the total model loss with respect to the intermediate output of the second-party training sent by the first party, and optimize the second-party bottom feature extraction model to be trained based on the second-party gradient to obtain the second-party bottom feature extraction model.

[0159] In this embodiment, specifically, receive the second-party gradient of the total model loss with respect to the intermediate output of the second-party training sent by the first party, receive the second-party gradient of the total model loss with respect to the intermediate output of the second-party training sent by the first party, and based on the second-party gradient, update the second-party bottom feature extraction model to be trained, and determine whether the updated second-party bottom feature extraction model to be trained meets the preset training end condition. If the updated second-party bottom feature extraction model to be trained meets the preset training end condition, use the second-party bottom feature extraction model to be trained as the second-party bottom feature extraction model. If the updated second-party bottom feature extraction model to be trained does not meet the preset training end condition, return to execute the step of extracting the second-party training samples.

[0160] After the step of receiving the second-party gradient of the total model loss with respect to the intermediate output of the second-party training and optimizing the second-party bottom feature extraction model to be trained based on the second-party gradient to obtain the second-party bottom feature extraction model, the federated learning modeling optimization method further includes:

[0161] Step E10: Obtain the second-party samples to be predicted, and based on the second-party bottom feature extraction model, jointly convert the second-party samples to be predicted and the preset second-party bottom passport data into a second-party intermediate output with passport embeddings.

[0162] In this embodiment, it should be noted that the second-party bottom feature extraction model includes at least one second-party passport embedding module, and the preset second-party bottom passport data includes at least one second-party passport sample to be embedded, where the second-party passport embedding modules correspond one-to-one with the second-party passport samples to be embedded.

[0163] Specifically, obtain the second-party samples to be predicted, and after inputting the second-party samples to be predicted and the preset second-party bottom passport data into the second-party bottom feature extraction model, through each second-party passport embedding module in the second-party bottom feature extraction model, respectively embed the second-party passport samples to be embedded corresponding to each second-party passport embedding module in the preset second-party bottom passport data into the intermediate output of the model corresponding to the second-party samples to be predicted, and after the model prediction is completed, the output layer of the second-party bottom feature extraction model outputs the second-party intermediate output with passport embeddings.

[0164] Step E20: Send the second-party intermediate output to the first party, so that the first party can generate a target prediction result based on the second-party intermediate outputs sent by each second party, the first-party samples to be predicted corresponding to the second-party samples to be predicted, the preset first-party bottom passport data, and the preset first-party top passport data, through the first-party bottom feature extraction model constructed by federated learning based on the preset first-party bottom passport data and the top prediction model constructed by federated learning based on the preset first-party top passport data.

[0165] In this embodiment, specifically, the second-party intermediate output is sent to the first party, so that the first party can convert the first-party samples to be predicted and the preset first-party bottom passport data into a first-party intermediate output based on the first-party bottom feature extraction model. Then, the first party aggregates the first-party intermediate output and the second-party intermediate outputs sent by each second party to obtain an aggregated intermediate output. Further, the first party converts the aggregated intermediate output and the preset second-party bottom passport data into a target prediction result based on the top prediction model. The specific processes of the first party generating the first-party intermediate output and the target prediction result can refer to the specific content in Steps C10 to C40 and will not be elaborated here.

[0166] The embodiment of the present application provides a method for optimizing federated learning modeling. Compared with the prior art, which protects data privacy by means of homomorphic encryption or multi-party secure computation during the federated learning process, the embodiment of the present application first obtains a second-party bottom feature extraction model to be trained, extracts second-party training samples, and then, based on the second-party bottom feature extraction model to be trained, converts the second-party training samples and preset second-party bottom passport data into a second-party training intermediate output. Then, the second-party training intermediate output is sent to the first party for the first party to aggregate the second-party training intermediate outputs sent by each second party, and the first-party training intermediate output obtained by converting the first-party training samples and preset first-party bottom passport data, to obtain an aggregated training intermediate output. And based on the aggregated training intermediate output and the output prediction label obtained by converting the preset second-party bottom passport data, and the preset true label corresponding to the first-party training samples, calculate the total model loss. It should be noted that the second-party training intermediate output with passport embedding is still plaintext data, and its data complexity and data volume are much smaller than those of homomorphic encryption data. And because the passport in the preset second-party bottom passport data is embedded in the second-party training intermediate output, since the first party does not hold the preset second-party bottom passport data, even if the first party receives the second-party training intermediate output which belongs to plaintext data, it cannot reverse-engineer the private data belonging to the second party. Then, receive the second-party gradient of the total model loss with respect to the second-party training intermediate output sent by the first party. Since the second-party gradient is generated based on the total model loss, and the total model loss is generated based on the first-party training intermediate output embedded with the preset first-party bottom passport data and the preset second-party bottom passport data, in the case that the second party does not hold the preset first-party bottom passport data and the preset second-party bottom passport data, it is impossible to reverse-engineer the private data of the first party through the second-party gradient. Then, based on the second-party gradient, optimize the second-party bottom feature extraction model to be trained to obtain the second-party bottom feature extraction model, achieving the purpose of constructing a federated model based on passport embedding, protecting data privacy during the federated learning process, and without the need to protect data privacy by means of homomorphic encryption or multi-party secure computation, achieving the purpose of performing data security computation in plaintext state during federated learning, reducing the data computation complexity during the federated learning process and avoiding a large number of data encryption and decryption processes during the federated learning process. Therefore, it overcomes the technical defects in the prior art that since multi-party secure computation involves complex cryptographic operations, the communication and computation overheads are both large, resulting in low computation efficiency during federated learning, and the data complexity and data volume of the ciphertext data after homomorphic encryption are much larger than those of plaintext data, so the computation overhead of homomorphic encryption is extremely large, resulting in low computation efficiency during federated learning, and the homomorphic encryption method needs to perform a large number of encryption and decryption processes during the federated learning process, which will further reduce the computation efficiency during federated learning. Therefore,Improves the computational efficiency during federated learning.

[0167] Further, referring to Figure 6 , in another embodiment of the present application, the federated prediction optimization method is applied to the first party, and the federated prediction optimization method includes:

[0168] Step F10, obtaining the first party's samples to be predicted, the preset first party's bottom passport data, and the preset first party's top passport data;

[0169] In this embodiment, it should be noted that the federated prediction optimization method is applied to the vertical federated learning scenario. Both the first party and the second party are participants in vertical federated learning. The number of the first parties can be one or more. Among them, the first party is provided with a first party's bottom feature extraction model, a top prediction model, and the first party's samples to be predicted with preset true labels. The second party is provided with a second party's bottom feature extraction model and the second party's samples to be predicted without sample labels. Among them, the first party's samples to be predicted and the second party's samples to be predicted correspond to the same public sample ID. The first party's bottom feature extraction model is used to convert the first party's samples to be predicted into the first party's intermediate output with passport embeddings. The second party's bottom feature extraction model is used to convert the second party's samples to be predicted into the second party's intermediate output with passport embeddings. The top prediction model is used to convert the aggregation result of the output of the first bottom feature extraction model and the outputs of each second bottom feature extraction model into the target prediction result.

[0170] Step F20, based on the first party's bottom feature extraction model constructed by federated learning using the preset first party's bottom passport data, jointly convert the first party's samples to be predicted and the preset first party's bottom passport data into the first party's intermediate output with passport embeddings;

[0171] In this embodiment, it should be noted that the first party's bottom feature extraction model includes at least one first party's passport embedding module, and the preset first party's bottom passport data includes at least one first party's passport sample to be embedded. Among them, the first party's passport embedding module and the first party's passport sample to be embedded correspond one by one. The first party's bottom feature extraction model is a machine learning model constructed by the first party through vertical federated learning with the second party based on the preset first party's bottom passport data. The specific construction process of the first party's bottom feature extraction model can refer to the content in steps S10 to S40 and their refinement steps, which will not be elaborated here.

[0172] Specifically, after inputting the first-party sample to be predicted and the preset first-party bottom passport data into the first-party bottom feature extraction model, through each first-party passport embedding module in the first-party bottom feature extraction model, each first-party passport sample to be embedded corresponding to each first-party passport embedding module in the preset first-party passports is respectively embedded into the intermediate output of the model corresponding to the first-party sample to be predicted. After the model prediction is completed, the output layer of the first-party bottom feature extraction model outputs the first-party intermediate output with passport embedding.

[0173] Wherein, the first-party bottom feature extraction model includes a first-party bottom neural network, the first-party bottom neural network includes at least one first-party passport embedding module, and the preset first-party bottom passport data includes at least one first-party passport sample to be embedded corresponding to the first-party passport embedding module.

[0174] The step of converting the first-party sample to be predicted and the preset first-party bottom passport data into the first-party intermediate output with passport embedding by the first-party bottom feature extraction model constructed based on the preset first-party bottom passport data through federated learning includes:

[0175] Step F21, based on the first part of the first-party bottom neural network in the first-party bottom neural network that is before the first-party passport embedding module, convert the first-party sample to be predicted into the first-party bottom network intermediate output;

[0176] In this embodiment, it should be noted that the first part of the bottom neural network is the part of the neural network corresponding to the intermediate output of the model that is the input of the first-party passport embedding module in the first-party bottom neural network, and is used to convert the first-party sample to be predicted into a partial input of the first-party passport embedding module.

[0177] Specifically, based on the first part of the bottom neural network in the first-party bottom neural network that is before the first-party passport embedding module, map the first-party sample to be predicted to the first-party bottom network intermediate output.

[0178] Step F22, based on the first-party passport embedding module, jointly convert the first-party bottom network intermediate output and the first-party passport sample to be embedded into the first-party bottom passport embedding module output;

[0179] In this embodiment, specifically, based on the first-party passport embedding module, by embedding the first-party passport sample to be embedded into the embedding module output generated by the first-party passport embedding module for the bottom network intermediate output, the first-party bottom passport embedding module output is obtained.

[0180] Among them, the first-party passport embedding module includes a first-party passport embedding layer and a first-party module neural network layer.

[0181] The step of converting the first-party bottom network intermediate output and the first-party passport sample to be embedded into the first-party bottom passport embedding module output based on the first-party passport embedding module includes:

[0182] Step F221: Based on the first-party module neural network layer, linearly transform the first-party bottom network intermediate output into an output of the network layer to be embedded, and linearly transform the first-party passport sample to be embedded into a passport to be embedded.

[0183] Step F222: Based on the passport function in the first-party passport embedding layer, convert the passport to be embedded into passport embedding parameters.

[0184] In this embodiment, it should be noted that the passport function is a pre-set function for generating passport embedding parameters based on the passport to be embedded.

[0185] Specifically, input the passport to be embedded into the passport function in the first-party passport embedding layer, and output the passport embedding function corresponding to the passport to be embedded.

[0186] Step F223: Generate the first-party bottom passport embedding module output based on the output of the network layer to be embedded and the passport embedding parameters.

[0187] In this embodiment, it should be noted that the passport embedding parameters include a first passport embedding parameter and a second passport embedding parameter.

[0188] Specifically, perform passport embedding on the output of the network layer to be embedded by calculating the target product between the output of the network layer to be embedded and the first passport embedding parameter, and the sum between the target product and the second passport embedding parameter, and generate the first-party bottom passport embedding module output. The specific process of generating the first-party bottom passport embedding module output is as follows:

[0189] X = γ * X t + β

[0190] where X is the first-party bottom passport embedding module output, X t is the output of the network layer to be embedded, γ is the first passport embedding parameter, and β is the second passport embedding parameter.

[0191] Step F23: Based on the second part of the first-party bottom neural network after the first-party passport embedding module in the first-party bottom neural network, convert the first-party bottom passport embedding module output into the first-party intermediate output with passport embedding.

[0192] In this embodiment, based on the second part of the first-party bottom neural network after the first-party passport embedding module in the first-party bottom neural network, the output of the first-party bottom passport embedding module is mapped to the first-party intermediate output with passport embedding.

[0193] Step F30: Receive the second-party intermediate outputs with passport embedding sent by each second party. The second-party intermediate output is obtained by the second-party bottom feature extraction model constructed by the second party through federated learning based on the preset second-party bottom passport data, and transforming the preset second-party bottom passport data and the second-party prediction samples corresponding to the first-party prediction samples.

[0194] In this embodiment, it should be noted that each of the second parties is provided with a second-party bottom feature extraction model, and the second party will, based on the second-party bottom feature extraction model, transform the second-party prediction samples corresponding to the first-party prediction samples and the preset second-party bottom passport data into second-party intermediate outputs. Although the second-party intermediate output is plaintext data, since the preset second-party bottom passport data is embedded when generating the second-party intermediate output, and thus the first party does not hold the preset second-party bottom passport data, the first party cannot reverse-engineer the second party's private data based on the second-party intermediate output.

[0195] Step F40: Aggregate the first-party passport embedding intermediate output and each of the second-party passport embedding intermediate outputs to obtain an aggregated passport embedding intermediate output.

[0196] In this embodiment, specifically, based on a preset aggregation method, the first-party intermediate output and each of the second-party intermediate outputs are aggregated into an aggregated intermediate output.

[0197] Step F50: The top prediction model constructed by the first party through federated learning based on the preset first-party top passport data transforms the aggregated passport embedding intermediate output and the preset first-party top passport data into a target prediction result.

[0198] In this embodiment, specifically, after inputting the aggregated passport embedding intermediate output and the preset first-party top passport data into the top prediction model, through each top passport embedding module in the top prediction model, the top passport embedding samples corresponding to each top passport embedding module in the preset first-party top passport data are respectively embedded into the model intermediate output corresponding to the aggregated passport embedding intermediate output, and after the model prediction is completed, the output layer of the top prediction model outputs the target prediction result.

[0199] Among them, the top prediction model includes a top neural network, the top neural network at least includes a top network passport embedding module, and the preset first-party top passport data at least includes a top passport embedding sample corresponding to the top network passport embedding module.

[0200] The step of converting the aggregated passport embedding intermediate output and the preset first-party top passport data into a target prediction result by the top prediction model constructed through federated learning based on the preset first-party top passport data includes:

[0201] Step F51: Based on the first part of the top neural network in the top neural network that is before the top network passport embedding module, convert the aggregated passport embedding intermediate output into a top network layer intermediate output.

[0202] In this embodiment, it should be noted that the first part of the top neural network is the part of the neural network corresponding to the intermediate output of the model that is the input of the top network passport embedding module in the top neural network, and is used to convert the aggregated passport embedding intermediate output into part of the input of the top passport embedding module.

[0203] Specifically, based on the first part of the top neural network in the top neural network that is before the top network passport embedding module, map the aggregated passport embedding intermediate output to the top network layer intermediate output.

[0204] Step F52: Based on the top passport embedding module, jointly convert the top network layer intermediate output and the top passport embedding sample into the top passport embedding module output.

[0205] In this embodiment, specifically, based on the top passport embedding module, by embedding the top passport embedding sample into the embedding module output generated by the top passport embedding module for the top network layer intermediate output, the top passport embedding module output is obtained.

[0206] Among them, the top passport embedding module includes a top passport embedding layer and a top module neural network layer. Step F52 further includes:

[0207] Based on the top module neural network layer, map the top network layer intermediate output to a top to-be-embedded output, and map the top passport embedding sample to a top top to-be-embedded passport. Then, based on the passport embedding function in the top passport embedding layer, convert the top top to-be-embedded passport into embedding passport parameters. Then, based on the embedding passport parameters, perform passport embedding on the top to-be-embedded output to obtain the top passport embedding module output. The process of generating the top passport embedding module output is specifically as follows:

[0208] X = γ * X t + β

[0209] Wherein, X is the output of the top passport embedding module, and X t is the output to be embedded at the top, and γ and β are the top passport embedding parameters.

[0210] Step F53, based on the second part of the top neural network that is after the top network passport embedding module in the top neural network, convert the output of the top passport embedding module into the target prediction result.

[0211] In this embodiment, it should be noted that the second part of the top neural network is the part of the top neural network that takes the output of the second passport embedding module as the input, and is used to convert the output of the second passport embedding module into the target prediction result.

[0212] Specifically, based on the second part of the top neural network that is after the top network passport embedding module in the top neural network, map the output of the top passport embedding module to the target prediction result.

[0213] Wherein, before the step of converting the first-party sample to be predicted and the preset first-party bottom passport data into the first-party intermediate output with passport embedding by the first-party bottom feature extraction model constructed based on the preset first-party bottom passport data for federated learning, the federated prediction optimization method further includes:

[0214] Step F60, obtain the first-party bottom feature extraction model to be trained and the top prediction model to be trained, and extract the preset true labels corresponding to the first-party samples to be trained and the first-party training samples;

[0215] In this embodiment, it should be noted that the first-party bottom feature extraction model to be trained is an untrained first-party bottom feature extraction model, and the top prediction model to be trained is an untrained top prediction model.

[0216] Step F70, based on the first-party bottom feature extraction model to be trained, jointly convert the first-party training samples and the preset first-party bottom passport data into the first-party training intermediate output with passport embedding;

[0217] In this embodiment, specifically, after inputting the first-party training samples and the preset first-party bottom passport data into the first-party bottom feature extraction model to be trained, through each to-be-trained first-party passport embedding module in the to-be-trained first-party bottom feature extraction model, each first-party passport sample to be embedded corresponding to each to-be-trained first-party passport embedding module is embedded into the intermediate output of the model corresponding to the first-party training samples. After the model prediction is completed, the output layer of the to-be-trained first-party bottom feature extraction model outputs the first-party training intermediate output with passport embedding.

[0218] Step F80: Based on the first-party training intermediate output, the preset true label, the preset first-party top passport data, and the to-be-trained top prediction model, through federated interaction with each second party, jointly calculate the total model loss corresponding to the to-be-trained top prediction model for the second-party training intermediate outputs with passport embedding generated by each second party, where the second-party training intermediate output is obtained by the to-be-trained second-party bottom feature extraction model obtained by the second party to transform the preset second-party bottom passport data and the second-party training samples corresponding to the first-party training samples;

[0219] In this embodiment, it should be noted that each second party is provided with a to-be-trained second-party bottom feature extraction model, and the second party will, based on the to-be-trained second-party bottom feature extraction model, transform the second-party training samples corresponding to the first-party training samples and the preset second-party bottom passport data into second-party training intermediate outputs, where the second-party training samples and the first-party training samples correspond to the same common sample ID, and the common sample ID is determined when the first party aligns samples with each second party. The specific process of the second party generating the second-party training intermediate output can refer to the content in step S20 and its refinement steps, which will not be elaborated here.

[0220] Specifically, receive the second-party training intermediate outputs sent by each second party, and based on the preset aggregation method, aggregate the first-party training intermediate output and each second-party training intermediate output to generate an aggregated training intermediate output. Then, based on the to-be-trained top prediction model, jointly transform the aggregated training intermediate output and the preset first-party top passport data into output prediction labels, and further calculate the total model loss based on the output prediction labels and the preset true labels, where the preset aggregation methods include weighted average and summation, etc.

[0221] Among them, the step of calculating the total model loss corresponding to the to-be-trained top prediction model based on the first-party training intermediate output, the preset true label, the preset first-party top passport data, and the second-party training intermediate output with passport embeddings jointly generated by federated interaction with each second party includes:

[0222] Step F81, receiving the second-party training intermediate output with passport embeddings sent by each second party;

[0223] In this embodiment, it should be noted that after the second party inputs the second-party training sample and the preset second-party bottom passport data into the to-be-trained second-party bottom feature extraction model, through each to-be-trained second-party passport embedding module in the to-be-trained second-party bottom feature extraction model, the to-be-embedded passport samples corresponding to each to-be-trained second-party passport embedding module in the preset second-party bottom passport data are respectively embedded into the model intermediate output corresponding to the second-party training sample. After the model prediction is completed, the output layer of the to-be-trained second-party bottom feature extraction model outputs the second-party training intermediate output with passport embeddings. Among them, since the second party embeds the preset second-party bottom passport data when generating the second-party training intermediate output, when the first party receives the second-party training intermediate output, because the first party does not hold the preset second-party bottom passport data, it is impossible to reverse-infer the first party's private data, such as model parameters and samples, through the second-party training intermediate output, achieving the purpose of protecting data privacy.

[0224] Step F82, aggregating the first-party training intermediate output and each second-party training intermediate output to obtain an aggregated training intermediate output;

[0225] In this embodiment, it should be noted that the aggregation methods include weighted average and summation.

[0226] Step F83, through the to-be-trained top prediction model, jointly converting the aggregated training intermediate output and the preset first-party top passport data into an output prediction label;

[0227] Specifically, after inputting the aggregated training intermediate output and the preset first-party top passport data into the to-be-trained top prediction model, through each to-be-trained top passport embedding module in the to-be-trained top prediction model, the top passport embedding samples corresponding to each to-be-trained top passport embedding module in the preset first-party top passport data are respectively embedded into the model intermediate output corresponding to the aggregated training intermediate output. After the model prediction is completed, the output layer of the to-be-trained top prediction model outputs the output prediction label.

[0228] Step F84, calculate the total model loss based on the output prediction label and the preset true label.

[0229] In this embodiment, specifically, the difference between the output prediction label and the preset true label is used as the total model loss.

[0230] Among them, after the step of calculating the total model loss corresponding to the to-be-trained top prediction model based on the first-party training intermediate output, the preset true label, the preset first-party top passport data, and the second-party training intermediate output with passport embedding generated by each second party through federated interaction with each second party, the federated learning modeling optimization method further includes:

[0231] Step G10, calculate the second-party gradients of the total model loss with respect to each second-party training intermediate output;

[0232] In this embodiment, specifically, take the partial derivatives of the total model loss with respect to each second-party training intermediate output to obtain the second-party gradients corresponding to each second-party training intermediate output.

[0233] Step G20, send each second-party gradient to its corresponding second party respectively, so that the second party optimizes the to-be-trained second-party bottom feature extraction model based on the second-party gradient to obtain the second-party bottom feature extraction model.

[0234] In this embodiment, specifically, send each second-party gradient to its corresponding second party respectively, so that the second party updates the to-be-trained second-party bottom feature extraction model based on the second-party gradient through a preset model optimization method, and determines whether the updated to-be-trained second-party bottom feature extraction model meets the preset training end condition. If the updated to-be-trained second-party bottom feature extraction model meets the preset training end condition, use the to-be-trained second-party bottom feature extraction model as the second-party bottom feature extraction model. If the updated to-be-trained second-party bottom feature extraction model does not meet the preset training end condition, return to execute the step of generating the second-party training intermediate output. Among them, the preset model optimization method includes gradient ascent method and gradient descent method, etc., and the preset training end condition includes loss function convergence and reaching the maximum iteration number threshold, etc. And since the first party embeds the preset first-party top passport data when generating the output prediction label, when the second party obtains the second-party gradient corresponding to the total model loss generated based on the output prediction label, since the second party does not hold the preset first-party bottom passport data and the preset first-party top passport data, the second party cannot reverse the first party's private data, such as model parameters and samples, etc., achieving the purpose of protecting data privacy.

[0235] Step F90: Optimize the to-be-trained first-party bottom feature extraction model and the to-be-trained top prediction model based on the total model loss to obtain the first-party bottom feature extraction model and the top prediction model.

[0236] In this embodiment, specifically, based on the total model loss, calculate the first-party bottom model gradient corresponding to the to-be-trained first-party bottom feature extraction model and the first-party top model gradient corresponding to the to-be-trained top prediction model. Then, update the to-be-trained first-party bottom feature extraction model according to the first-party bottom model gradient, and update the to-be-trained top prediction model according to the first-party top model gradient. Then, determine whether both the to-be-trained first-party bottom feature extraction model and the to-be-trained top prediction model meet the preset training end condition. If both the to-be-trained first-party bottom feature extraction model and the to-be-trained top prediction model meet the preset training end condition, use the to-be-trained first-party bottom feature extraction model as the first-party bottom feature extraction model and use the to-be-trained top prediction model as the top prediction model. If both the to-be-trained first-party bottom feature extraction model and the to-be-trained top prediction model do not meet the preset training end condition, return to execute the step of extracting the preset true labels corresponding to the first-party to-be-trained samples and the first-party training samples. Among them, the specific processes in steps F60 to F90 can refer to the content in steps S10 to S40 and their refinement steps, which will not be elaborated here.

[0237] An embodiment of the present application provides a method for optimizing federated prediction. First, a first-party sample to be predicted, a preset first-party bottom passport data, and a preset first-party top passport data are obtained. Then, a first-party bottom feature extraction model constructed by federated learning based on the preset first-party bottom passport data is used to jointly convert the first-party sample to be predicted and the preset first-party bottom passport data into a first-party intermediate output with passport embedding. Next, a second-party intermediate output with passport embedding sent by each second party is received. The second-party intermediate output is obtained by the second party using a second-party bottom feature extraction model constructed by federated learning based on the preset second-party bottom passport data to convert the preset second-party bottom passport data and the second-party sample to be predicted corresponding to the first-party sample to be predicted. Although the second-party intermediate output is still plaintext data, it embeds the preset second-party bottom passport data. Therefore, in the case where the first party does not hold the preset second-party bottom passport data, the first party cannot reverse-engineer the second party's private data based on the second-party intermediate output. Then, the first-party passport-embedded intermediate output and each second-party passport-embedded intermediate output are aggregated to obtain an aggregated passport-embedded intermediate output. And a top prediction model constructed by federated learning based on the preset first-party top passport data is used to convert the aggregated passport-embedded intermediate output and the preset first-party top passport data into a target prediction result, achieving the purpose of protecting data privacy while performing the data calculation process in the federated learning process in the plaintext state. Further, compared with the federated prediction method based on homomorphic encryption or multi-party secure computation, the data calculation complexity in the federated prediction process is reduced, and a large number of data encryption and decryption processes are avoided. Therefore, the computing efficiency during federated prediction is improved.

[0238] Further, referring to Figure 7 , in another embodiment of the present application, the federated prediction optimization method is applied to the second party, and the federated prediction optimization method includes:

[0239] Step H10: Obtain a second-party sample to be predicted and preset second-party bottom passport data;

[0240] In this embodiment, the federated prediction optimization method is applied to a vertical federated learning scenario. The first party and the second party are both participants in vertical federated learning. The specific explanation content of the first party and the second party can refer to the corresponding embodiment content of step F10, which will not be elaborated here.

[0241] Step H20: Use a second-party bottom feature extraction model constructed by federated learning based on the preset second-party bottom passport data to jointly convert the second-party sample to be predicted and the preset second-party bottom passport data into a second-party intermediate output with passport embedding;

[0242] In this embodiment, it should be noted that the second-party bottom feature extraction model is a machine learning model constructed by the second party through vertical federated learning with the first party based on preset second-party bottom passport data. Among them, the second-party bottom feature extraction model at least includes a second-party passport embedding module, and the preset second-party bottom passport data at least includes a second-party passport sample to be embedded. Among them, the second-party passport embedding module corresponds one-to-one with the second-party passport sample to be embedded.

[0243] Specifically, after inputting the second-party sample to be predicted and the preset second-party bottom passport data into the second-party bottom feature extraction model, through each second-party passport embedding module in the second-party bottom feature extraction model, the second-party passport samples to be embedded corresponding to each second-party passport embedding module in the preset second-party bottom passport data are respectively embedded into the intermediate output of the model corresponding to the second-party sample to be predicted. After the model prediction is completed, the output layer of the second-party bottom feature extraction model outputs the second-party intermediate output with passport embedding.

[0244] Among them, the second-party bottom feature extraction model includes a second-party bottom neural network. The second-party bottom neural network at least includes a second-party passport embedding module. The preset second-party bottom passport data at least includes a second-party passport sample to be embedded corresponding to the second-party passport embedding module.

[0245] The step of converting the second-party sample to be predicted and the preset second-party bottom passport data into the second-party intermediate output with passport embedding by the second-party bottom feature extraction model constructed through federated learning based on the preset second-party bottom passport data includes:

[0246] Step H21: Based on the first part of the second-party bottom neural network in the second-party bottom neural network before the second-party passport embedding module, convert the second-party sample to be predicted into the second-party first-party bottom network intermediate output.

[0247] In this embodiment, it should be noted that the first part of the second-party bottom neural network is the part of the neural network corresponding to the intermediate output of the model that serves as the input of the second-party passport embedding module in the second-party bottom neural network, and is used to convert the second-party sample to be predicted into a partial input of the second-party passport embedding module.

[0248] Specifically, based on the first part of the second-party bottom neural network in the second-party bottom neural network before the second-party passport embedding module, map the second-party sample to be predicted to the second-party first-party bottom network intermediate output.

[0249] Step H22: Based on the second-party passport embedding module, convert the second-party first-party bottom network intermediate output and the second-party passport sample to be embedded into the second-party bottom passport embedding module output;

[0250] In this embodiment, specifically, based on the second-party passport embedding module, by embedding the second-party passport sample to be embedded into the embedding module output generated by the second-party passport embedding module for the second-party first-party bottom network intermediate output, obtain the second-party bottom passport embedding module output.

[0251] Furthermore, the second-party passport embedding module includes a second-party passport embedding layer and a second-party module neural network layer. Step H22 further includes:

[0252] Based on the second-party module neural network layer, map the second-party first-party bottom network intermediate output to a second-party output to be embedded, and map the second-party passport sample to be embedded to a second-party passport to be embedded. Then, based on the passport embedding function in the second-party passport embedding layer, convert the second-party passport to be embedded to second-party embedding passport parameters. Then, based on the second-party embedding passport parameters, perform passport embedding on the second-party output to be embedded to obtain the second-party bottom passport embedding module output. Among them, the process of generating the second-party bottom passport embedding module output is as follows:

[0253] X = γ * X t + β

[0254] where X is the second-party bottom passport embedding module output, X t is the second-party output to be embedded, and γ and β are the second-party passport embedding parameters.

[0255] Step H23: Based on the second part of the second-party bottom neural network in the second-party bottom neural network after the second-party passport embedding module, convert the second-party bottom passport embedding module output to the second-party intermediate output with passport embedding.

[0256] In this embodiment, it should be noted that the second part of the second-party bottom neural network is the part of the second-party bottom neural network that takes the output of the second-party passport embedding module as input, and is used to convert the output of the second-party passport embedding module to the second-party intermediate output.

[0257] Specifically, based on the second part of the second-party bottom neural network in the second-party bottom neural network after the second-party passport embedding module, map the second-party bottom passport embedding module output to the second-party intermediate output with passport embedding.

[0258] Step H30: Send the second-party intermediate output to the first party, so that the first party can generate a target prediction result based on the second-party intermediate outputs sent by each second party, the first-party samples to be predicted corresponding to the second-party samples to be predicted, the preset first-party bottom passport data, and the preset first-party top passport data, through the first-party bottom feature extraction model constructed by federated learning based on the preset first-party bottom passport data and the top prediction model constructed by federated learning based on the preset first-party top passport data.

[0259] In this embodiment, specifically, the second-party intermediate output is sent to the first party, so that the first party can convert the first-party samples to be predicted and the preset first-party bottom passport data into a first-party intermediate output based on the first-party bottom feature extraction model. Then, the first party aggregates the first-party intermediate output and the second-party intermediate outputs sent by each second party to obtain an aggregated intermediate output. Further, the first party converts the aggregated intermediate output and the preset first-party top passport data into a target prediction result based on the top prediction model. The specific processes of the first party generating the first-party intermediate output and the target prediction result can refer to the specific content in steps C10 to C40 and will not be elaborated here.

[0260] Before the step of the second-party bottom feature extraction model constructed by federated learning based on the preset second-party bottom passport data converting the second-party samples to be predicted and the preset second-party bottom passport data into a second-party intermediate output with passport embedding, the federated prediction optimization method further includes:

[0261] Step M10: Obtain the second-party bottom feature extraction model to be trained and extract second-party training samples.

[0262] In this embodiment, it should be noted that the first party is provided with a first-party bottom feature extraction model to be trained, a top prediction model to be trained, and first-party training samples with preset true labels. The second party is provided with a second-party bottom feature extraction model to be trained and second-party training samples without sample labels. Among them, the first-party bottom feature extraction model to be trained is the first-party bottom feature extraction model that has not been trained well in the first party and is used to convert the training samples in the first party into an intermediate output with passport embedding. The second-party bottom feature extraction model to be trained is the second-party bottom feature extraction model that has not been trained well in the second party and is used to convert the training samples in the second party into an intermediate output with passport embedding. The top prediction model to be trained is the top prediction model that has not been trained well in the first party and is used to convert the aggregation result of the output of the first bottom feature extraction model to be trained and the output of each second bottom feature extraction model to be trained into an output prediction label. The preset true label is the identifier of the first-party training samples to be trained and is used to identify information such as the category or attribute of the first-party training samples to be trained.

[0263] Step M20: Based on the second-party bottom feature extraction model to be trained, convert the second-party training samples and the preset second-party bottom passport data into a second-party training intermediate output.

[0264] In this embodiment, it should be noted that the preset second-party bottom passport data includes at least one second-party passport sample to be embedded. Among them, the second-party passport sample to be embedded is a sample that is pre-set for passport embedding in the second-party bottom feature extraction model to be trained. The second-party passport sample to be embedded can be a picture sample, a text sample, or a specific coding matrix, etc. The second-party bottom feature extraction model to be trained includes at least one second-party passport embedding module to be trained, which is used to embed the second-party passport sample to be embedded into the intermediate output of the model of the second-party training samples and jointly convert it into the final output of the model. The intermediate output of the model is the output of the intermediate network layer of the model, and the final output of the model is the output generated by the output layer of the model. For example, assume that neural network model A includes an input layer, 10 hidden layers, and an output layer. Then the output of the input layer and the output of the 10 hidden layers are both the intermediate output of the model.

[0265] Specifically, after inputting the second-party training samples and the preset second-party bottom passport data into the second-party bottom feature extraction model to be trained, through each second-party passport embedding module in the second-party bottom feature extraction model to be trained, each second-party passport sample to be embedded corresponding to each second-party passport embedding module is respectively embedded into the intermediate output of the model corresponding to the second-party training samples. After the model prediction is completed, the output layer of the second-party bottom feature extraction model to be trained outputs the second-party training intermediate output with passport embedding. For example, assume that the second-party bottom feature extraction model to be trained includes two second-party passport embedding modules A and B. Among them, the second-party passport embedding module A is set between the 3rd hidden layer and the 4th hidden layer of the second-party bottom feature extraction model to be trained. Then, through the second-party passport embedding module A, the second-party passport sample to be embedded corresponding to A is embedded into the 3rd output of the 3rd hidden layer, and the 3rd output after embedding the second-party passport sample corresponding to A is used as the input of the 4th hidden layer. The second-party passport embedding module B is set between the 40th hidden layer and the 41st hidden layer of the second-party bottom feature extraction model to be trained. Then, through the second-party passport embedding module B, the second-party passport sample to be embedded corresponding to B is embedded into the 40th output of the 40th hidden layer, and the 40th output after embedding the second-party passport sample corresponding to B is used as the input of the 41st hidden layer. Finally, the output layer of the second-party bottom feature extraction model to be trained outputs the second-party training intermediate output. Among them, the specific process of generating the second-party training intermediate output can refer to the specific content in steps D21 to D23 and will not be elaborated here.

[0266] Step M30: Send the second-party training intermediate output to the first party, so that the first party aggregates the second-party training intermediate outputs sent by each second party, and based on the first-party training intermediate output obtained by converting the first-party training samples and the preset first-party bottom passport data, obtains the aggregated training intermediate output, and based on the aggregated training intermediate output and the output prediction label obtained by converting the preset first-party top passport data, and the preset true label corresponding to the first-party training samples, calculates the total model loss.

[0267] In this embodiment, specifically, the second-party training intermediate output is sent to the first party, so that the first party aggregates the generated first-party training output and the second-party training intermediate outputs sent by each second party to obtain an aggregated training intermediate output. Then, based on the to-be-trained top prediction model, the aggregated training intermediate output and the preset first-party top passport data are jointly converted into output prediction labels. Further, based on the output prediction labels and the preset true labels, the total model loss is calculated. Then, based on the total model loss, the first party optimizes the to-be-trained first-party bottom feature extraction model and the to-be-trained top prediction model to obtain the first-party bottom feature extraction model and the top prediction model, and calculates the second-party gradients for each second-party training intermediate output. Then, the first party sends each of the second-party gradients to its corresponding second party. Among them, the specific processes of the first party generating the first-party training intermediate output, generating the total model loss, constructing the first-party bottom feature extraction model, and constructing the top prediction model can refer to the content in steps S10 to S40 and their refinement steps, which will not be elaborated here.

[0268] Step M40: Receive the second-party gradients of the total model loss with respect to the second-party training intermediate output sent by the first party, and based on the second-party gradients, optimize the to-be-trained second-party bottom feature extraction model to obtain the second-party bottom feature extraction model.

[0269] In this embodiment, specifically, receive the second-party gradients of the total model loss with respect to the second-party training intermediate output sent by the first party, receive the second-party gradients of the total model loss with respect to the second-party training intermediate output sent by the first party, and based on the second-party gradients, update the to-be-trained second-party bottom feature extraction model, and determine whether the updated to-be-trained second-party bottom feature extraction model meets the preset training end condition. If the updated to-be-trained second-party bottom feature extraction model meets the preset training end condition, use the to-be-trained second-party bottom feature extraction model as the second-party bottom feature extraction model. If the updated to-be-trained second-party bottom feature extraction model does not meet the preset training end condition, return to execute the step of extracting the second-party training samples.

[0270] An embodiment of the present application provides a method for optimizing federated prediction. First, obtain the second-party samples to be predicted and the preset second-party bottom passport data. Then, based on the preset second-party bottom passport data, construct a second-party bottom feature extraction model through federated learning, and convert the second-party samples to be predicted and the preset second-party bottom passport data into a second-party intermediate output with passport embeddings. Then, send the second-party intermediate output to the first party. For the first party, based on the second-party intermediate outputs sent by each second party, the first-party samples to be predicted corresponding to the second-party samples to be predicted, the preset first-party bottom passport data, and the preset first-party top passport data, through the first-party bottom feature extraction model constructed through federated learning based on the preset first-party bottom passport data and the top prediction model constructed through federated learning based on the preset first-party top passport data, generate a target prediction result. Then, the first party can feedback the target prediction result to each second party. Among them, although the second-party intermediate output is still plaintext data, the second-party intermediate output embeds the preset second-party bottom passport data. Therefore, in the case where the first party does not hold the preset second-party bottom passport data, the first party cannot reverse-engineer the second party's private data based on the second-party intermediate output, achieving the purpose of protecting data privacy while performing the data calculation process in the plaintext state during the federated learning process. Furthermore, compared with the federated prediction method based on homomorphic encryption or multi-party secure computation, the data calculation complexity in the federated prediction process is reduced, and a large number of data encryption and decryption processes are avoided. Therefore, the computing efficiency during federated prediction is improved.

[0271] Refer to Figure 8 , Figure 8 It is a schematic diagram of the device structure of the hardware operating environment involved in the solution of the embodiment of the present application.

[0272] As Figure 8 shown, the federated learning modeling optimization device may include: a processor 1001, such as a CPU, a memory 1005, and a communication bus 1002. Among them, the communication bus 1002 is used to realize the connection and communication between the processor 1001 and the memory 1005. The memory 1005 can be a high-speed RAM memory or a stable memory (non-volatile memory), such as a disk memory. Optionally, the memory 1005 can also be a storage device independent of the aforementioned processor 1001.

[0273] Optionally, the federated learning modeling optimization device may further include a rectangular user interface, a network interface, a camera, an RF (Radio Frequency) circuit, sensors, an audio circuit, a WiFi module, and so on. The rectangular user interface may include a display screen and an input sub-module such as a keyboard. Optionally, the rectangular user interface may further include a standard wired interface and a wireless interface. The network interface may optionally include a standard wired interface and a wireless interface (such as a WI-FI interface).

[0274] Those skilled in the art can understand that Figure 8 the structure of the federated learning modeling optimization device shown in does not constitute a limitation on the federated learning modeling optimization device, and it may include more or fewer components than shown, or combine certain components, or have different component arrangements.

[0275] Such as Figure 8 As shown, in the memory 1005 as a computer storage medium, there may be included an operating system, a network communication module, and a federated learning modeling optimization program. The operating system is a program for managing and controlling the hardware and software resources of the federated learning modeling optimization device, and supports the operation of the federated learning modeling optimization program and other software and / or programs. The network communication module is used to implement communication between components inside the memory 1005, and communication with other hardware and software in the federated learning modeling optimization system.

[0276] In Figure 8 the federated learning modeling optimization device shown, the processor 1001 is used to execute the federated learning modeling optimization program stored in the memory 1005 to implement the steps of the federated learning modeling optimization method described in any one of the above.

[0277] The specific implementation manner of the federated learning modeling optimization device in this application is basically the same as that of each embodiment of the above federated learning modeling optimization method, and will not be elaborated here.

[0278] Referring to Figure 9 , Figure 9 is a schematic diagram of the device structure of the hardware operating environment involved in the solution of the embodiment of this application.

[0279] Such as Figure 9 As shown, the federated prediction optimization device may include: a processor 1001, such as a CPU, a memory 1005, and a communication bus 1002. Among them, the communication bus 1002 is used to implement the connection communication between the processor 1001 and the memory 1005. The memory 1005 may be a high-speed RAM memory, or a stable memory (non-volatile memory), such as a disk memory. Optionally, the memory 1005 may also be a storage device independent of the aforementioned processor 1001.

[0280] Optionally, the federated prediction optimization device may further include a rectangular user interface, a network interface, a camera, an RF (Radio Frequency) circuit, sensors, an audio circuit, a WiFi module, and so on. The rectangular user interface may include a display screen and an input sub-module such as a keyboard. Optionally, the rectangular user interface may further include a standard wired interface and a wireless interface. The network interface may optionally include a standard wired interface and a wireless interface (such as a WI-FI interface).

[0281] Those skilled in the art can understand that Figure 9 the structure of the federated prediction optimization device shown in does not constitute a limitation on the federated prediction optimization device. It may include more or fewer components than shown, or combine certain components, or have a different component layout.

[0282] As Figure 9 shown, in the memory 1005 as a computer storage medium, there may be included an operating system, a network communication module, and a federated prediction optimization program. The operating system is a program for managing and controlling the hardware and software resources of the federated prediction optimization device, and supports the operation of the federated prediction optimization program and other software and / or programs. The network communication module is used to implement the communication between the components inside the memory 1005, as well as the communication with other hardware and software in the federated prediction optimization system.

[0283] In Figure 9 the federated prediction optimization device shown, the processor 1001 is used to execute the federated prediction optimization program stored in the memory 1005 to implement the steps of the federated prediction optimization method described in any one of the above.

[0284] The specific implementation manner of the federated prediction optimization device in this application is basically the same as that of each embodiment of the above federated prediction optimization method, and will not be elaborated here.

[0285] This application embodiment also provides a federated learning modeling optimization device. The federated learning modeling optimization device is applied to the first party, and the federated learning modeling optimization device includes:

[0286] An extraction module, configured to obtain a first-party bottom feature extraction model to be trained and a top prediction model to be trained, and extract a first-party sample to be trained and a preset true label corresponding to the first-party training sample;

[0287] A conversion module, configured to convert the first-party training sample and preset first-party bottom passport data into a first-party training intermediate output with passport embedding based on the first-party bottom feature extraction model to be trained;

[0288] A calculation module, configured to calculate the total model loss corresponding to the to-be-trained top prediction model based on the first-party training intermediate output, the preset true label, the preset first-party top passport data, and the to-be-trained top prediction model, by federating with each second party and jointly using the second-party training intermediate output with passport embeddings generated by each second party, where the second-party training intermediate output is obtained by the second party's to-be-trained second-party bottom feature extraction model to transform the preset second-party bottom passport data and the second-party training samples corresponding to the first-party training samples;

[0289] An optimization module, configured to optimize the to-be-trained first-party bottom feature extraction model and the to-be-trained top prediction model based on the total model loss, to obtain the first-party bottom feature extraction model and the top prediction model.

[0290] Optionally, the calculation module is further configured to:

[0291] Receive the second-party training intermediate output with passport embeddings sent by each second party;

[0292] Aggregate the first-party training intermediate output and each second-party training intermediate output to obtain an aggregated training intermediate output;

[0293] Through the to-be-trained top prediction model, jointly transform the aggregated training intermediate output and the preset first-party top passport data into an output prediction label;

[0294] Calculate the total model loss based on the output prediction label and the preset true label.

[0295] Optionally, the calculation module is further configured to:

[0296] Based on the first part of the to-be-trained top neural network before the to-be-trained top passport embedding module, transform the aggregated training intermediate output into a to-be-trained top network intermediate output;

[0297] Based on the to-be-trained top passport embedding module, jointly transform the to-be-trained top network intermediate output and the top passport embedding sample into an output of the to-be-trained top passport embedding module;

[0298] Based on the second part of the to-be-trained top neural network after the to-be-trained top passport embedding module, transform the output of the to-be-trained top passport embedding module into the output prediction label.

[0299] Optionally, the calculation module is further configured to:

[0300] Based on the neural network layer of the to-be-trained top module, linearly transform the intermediate output of the to-be-trained network into the output of the to-be-embedded training network layer, and linearly transform the top passport embedding sample into the to-be-embedded passport;

[0301] Based on the passport function in the first-party passport embedding layer, convert the to-be-embedded passport into passport embedding parameters;

[0302] Generate the output of the to-be-trained top passport embedding module based on the output of the to-be-embedded training network layer and the passport embedding parameters.

[0303] Optionally, the computing module is further configured to:

[0304] Based on the first part of the to-be-trained first-party bottom neural network that is before the to-be-trained first-party passport embedding module in the to-be-trained first-party bottom neural network, convert the first-party training sample into the intermediate output of the to-be-trained first-party bottom network;

[0305] Based on the to-be-trained first-party passport embedding module, jointly convert the intermediate output of the to-be-trained first-party bottom network and the first-party to-be-embedded passport sample into the output of the to-be-trained second-party bottom passport embedding module;

[0306] Based on the second part of the to-be-trained first-party bottom neural network that is after the to-be-trained first-party passport embedding module in the to-be-trained first-party bottom neural network, convert the output of the to-be-trained second-party bottom passport embedding module into the intermediate output of the first-party training.

[0307] Optionally, the federated learning modeling optimization device is further configured to:

[0308] Calculate the second-party gradients of the total model loss with respect to each of the second-party training intermediate outputs;

[0309] Send each of the second-party gradients to their respective corresponding second parties, for the second parties to optimize the to-be-trained second-party bottom feature extraction model based on the second-party gradients to obtain the second-party bottom feature extraction model.

[0310] Optionally, the federated learning modeling optimization device is further configured to:

[0311] Obtain the first-party to-be-predicted sample, and based on the first-party bottom feature extraction model, jointly convert the first-party to-be-predicted sample and the preset first-party bottom passport data into the first-party intermediate output with passport embedding;

[0312] Receive the second - party intermediate output with passport embedding sent by each of the second parties, where the second - party intermediate output is obtained by the second party based on the second - party bottom - end feature extraction model to transform the preset second - party bottom - end passport data and the second - party samples to be predicted corresponding to the first - party samples to be predicted;

[0313] Aggregate the first - party intermediate output and each of the second - party intermediate outputs to obtain an aggregated intermediate output;

[0314] Based on the top - end prediction model, jointly transform the aggregated intermediate output and the preset first - party top - end passport data into a target prediction result.

[0315] The specific implementation manner of the federated learning modeling optimization device of this application is basically the same as those of the above - mentioned embodiments of the federated learning modeling optimization method, and will not be elaborated here.

[0316] An embodiment of this application also provides a federated learning modeling optimization device. The federated learning modeling optimization device is applied to the second party and includes:

[0317] An extraction module, configured to obtain a second - party bottom - end feature extraction model to be trained and extract second - party training samples;

[0318] A conversion module, configured to transform the second - party training samples and the preset second - party bottom - end passport data into a second - party training intermediate output based on the second - party bottom - end feature extraction model to be trained;

[0319] A calculation module, configured to send the second - party training intermediate output to the first party for the first party to aggregate the second - party training intermediate outputs sent by each of the second parties, and the first - party training intermediate output obtained by transforming the first - party training samples and the preset first - party bottom - end passport data, to obtain an aggregated training intermediate output, and calculate the total model loss based on the output prediction label obtained by transforming the aggregated training intermediate output and the preset first - party top - end passport data, and the preset true label corresponding to the first - party training samples;

[0320] An optimization module, configured to receive the second - party gradient of the total model loss with respect to the second - party training intermediate output sent by the first party, and optimize the second - party bottom - end feature extraction model to be trained based on the second - party gradient to obtain a second - party bottom - end feature extraction model.

[0321] Optionally, the conversion module is further configured to:

[0322] Based on the first part of the second - party bottom - end neural network to be trained that is before the second - party passport embedding module in the second - party bottom - end neural network to be trained, transform the second - party training samples into a second - party bottom - end network intermediate output;

[0323] Based on the to-be-trained second-party passport embedding module, convert the intermediate output of the bottom network of the to-be-trained second party and the second-party passport sample to be embedded into the output of the to-be-trained second-party bottom passport embedding module;

[0324] Based on the second part of the to-be-trained second-party bottom neural network in the to-be-trained second-party bottom neural network that is after the to-be-trained second-party passport embedding module, convert the output of the to-be-trained second-party bottom passport embedding module to the intermediate output of the second-party training.

[0325] Optionally, the federated learning modeling optimization device is further configured to:

[0326] Obtain a second-party sample to be predicted, and based on the second-party bottom feature extraction model, jointly convert the second-party sample to be predicted and the preset second-party bottom passport data into a second-party intermediate output with passport embedding;

[0327] Send the second-party intermediate output to the first party, so that the first party can, based on the second-party intermediate outputs sent by each second party, the first-party sample to be predicted corresponding to the second-party sample to be predicted, the preset first-party bottom passport data, and the preset first-party top passport data, generate a target prediction result through the first-party bottom feature extraction model constructed by federated learning based on the preset first-party bottom passport data and the top prediction model constructed by federated learning based on the preset first-party top passport data.

[0328] The specific implementation manners of the federated learning modeling optimization device of this application are basically the same as those of the various embodiments of the above-mentioned federated learning modeling method, and will not be elaborated here.

[0329] An embodiment of this application further provides a federated prediction optimization device. The federated prediction optimization device is applied to the first party, and the federated prediction optimization device includes:

[0330] An acquisition module, configured to acquire a first-party sample to be predicted, preset first-party bottom passport data, and preset first-party top passport data;

[0331] A first conversion module, configured to jointly convert the first-party sample to be predicted and the preset first-party bottom passport data into a first-party intermediate output with passport embedding through the first-party bottom feature extraction model constructed by federated learning based on the preset first-party bottom passport data;

[0332] A receiving module, configured to receive the second-party intermediate outputs with passport embeddings sent by each second party, where the second-party intermediate outputs are obtained by a second-party bottom feature extraction model constructed by the second party through federated learning based on preset second-party bottom passport data, and converting the preset second-party bottom passport data and second-party prediction samples corresponding to the first-party samples to be predicted;

[0333] An aggregation module, configured to aggregate the first-party passport embedding intermediate output and each of the second-party passport embedding intermediate outputs to obtain an aggregated passport embedding intermediate output;

[0334] A second conversion module, configured to convert the aggregated passport embedding intermediate output and the preset first-party top passport data into a target prediction result based on a top prediction model constructed by the first party through federated learning based on the preset first-party top passport data.

[0335] Optionally, the first conversion module is further configured to:

[0336] Based on a first part of the first-party bottom neural network that is before the first-party passport embedding module in the first-party bottom neural network, convert the first-party sample to be predicted into a first-party bottom network intermediate output;

[0337] Based on the first-party passport embedding module, jointly convert the first-party bottom network intermediate output and the first-party passport sample to be embedded into an output of the first-party bottom passport embedding module;

[0338] Based on a second part of the first-party bottom neural network that is after the first-party passport embedding module in the first-party bottom neural network, convert the output of the first-party bottom passport embedding module into the first-party intermediate output with passport embedding.

[0339] Optionally, the first conversion module is further configured to:

[0340] Based on the first-party module neural network layer, linearly transform the first-party bottom network intermediate output into an output of the network layer to be embedded, and linearly transform the first-party passport sample to be embedded into a passport to be embedded;

[0341] Based on the passport function in the first-party passport embedding layer, convert the passport to be embedded into passport embedding parameters;

[0342] Generate the output of the second-party bottom passport embedding module based on the output of the network layer to be embedded and the passport embedding parameters.

[0343] Optionally, the second conversion module is further configured to:

[0344] Based on the first part of the top neural network before the top network passport embedding module in the top neural network, convert the aggregated passport embedding intermediate output into a top network layer intermediate output;

[0345] Based on the top passport embedding module, jointly convert the top network layer intermediate output and the top passport embedding sample into a top passport embedding module output;

[0346] Based on the second part of the top neural network after the top network passport embedding module in the top neural network, convert the top passport embedding module output into the target prediction result.

[0347] Optionally, the federated prediction optimization device further includes:

[0348] Obtain the first-party bottom feature extraction model to be trained and the top prediction model to be trained, and extract the first-party samples to be trained and the preset true labels corresponding to the first-party training samples;

[0349] Based on the first-party bottom feature extraction model to be trained, jointly convert the first-party training samples and the preset first-party bottom passport data into a first-party training intermediate output with passport embedding;

[0350] Based on the first-party training intermediate output, the preset true labels, the preset first-party top passport data, and the top prediction model to be trained, through federated interaction with each second party, jointly calculate the total model loss corresponding to the top prediction model to be trained with the second-party training intermediate output with passport embedding generated by each second party, where the second-party training intermediate output is obtained by the second-party bottom feature extraction model to be trained by the second party and converting the preset second-party bottom passport data and the second-party training samples corresponding to the first-party training samples;

[0351] Based on the total model loss, optimize the first-party bottom feature extraction model to be trained and the top prediction model to be trained to obtain a first-party bottom feature extraction model and a top prediction model.

[0352] Optionally, the federated prediction optimization device further includes:

[0353] Receive the second-party training intermediate output with passport embedding sent by each second party;

[0354] Aggregate the first-party training intermediate output and each second-party training intermediate output to obtain an aggregated training intermediate output;

[0355] Through the top prediction model to be trained, jointly convert the aggregated training intermediate output and the preset first-party top passport data into an output prediction label;

[0356] Based on the output prediction label and the preset true label, calculate the total loss of the model.

[0357] Optionally, the federated prediction optimization device further includes:

[0358] Calculate the second-party gradients of the total loss of the model with respect to each of the second-party training intermediate outputs;

[0359] Send each of the second-party gradients to their respective corresponding second parties, so that the second parties can optimize the to-be-trained second-party bottom feature extraction model based on the second-party gradients to obtain a second-party bottom feature extraction model.

[0360] The specific implementation manners of the federated prediction optimization device of this application are basically the same as those of the above-mentioned embodiments of the federated prediction optimization method, and will not be elaborated here.

[0361] This application embodiment also provides a federated prediction optimization device. The federated prediction optimization device is applied to a second party, and the federated prediction optimization device includes:

[0362] An acquisition module, configured to acquire a second-party to-be-predicted sample and preset second-party bottom passport data;

[0363] A conversion module, configured to convert the second-party to-be-predicted sample and the preset second-party bottom passport data into a second-party intermediate output with passport embeddings through a second-party bottom feature extraction model constructed based on the preset second-party bottom passport data;

[0364] A sending module, configured to send the second-party intermediate output to a first party, so that the first party can generate a target prediction result through a first-party bottom feature extraction model constructed based on the preset first-party bottom passport data and a top prediction model constructed based on the preset first-party top passport data, based on the second-party intermediate outputs sent by each second party, the first-party to-be-predicted sample corresponding to the second-party to-be-predicted sample, the preset first-party bottom passport data, and the preset first-party top passport data.

[0365] Optionally, the conversion module is further configured to:

[0366] Based on the first part of the second-party bottom neural network before the second-party passport embedding module in the second-party bottom neural network, convert the second-party to-be-predicted sample into a second-party first-party bottom network intermediate output;

[0367] Based on the second-party passport embedding module, convert the second-party first-party bottom network intermediate output and the second-party passport sample to be embedded into an output of the second-party bottom passport embedding module;

[0368] Based on the second part of the second - party bottom - end neural network after the second - party passport embedding module in the second - party bottom - end neural network, convert the output of the second - party passport embedding module into the second - party intermediate output with passport embedding.

[0369] Optionally, the federated prediction optimization device is further configured to:

[0370] Obtain the second - party bottom - end feature extraction model to be trained, and extract second - party training samples;

[0371] Based on the second - party bottom - end feature extraction model to be trained, convert the second - party training samples and the preset second - party bottom - end passport data into second - party training intermediate outputs;

[0372] Send the second - party training intermediate outputs to the first party for the first party to aggregate the second - party training intermediate outputs sent by each second - party, and based on the first - party training samples and the first - party training intermediate outputs converted from the preset first - party bottom - end passport data, obtain aggregated training intermediate outputs, and based on the aggregated training intermediate outputs and the output prediction labels converted from the preset first - party top - end passport data, and the preset true labels corresponding to the first - party training samples, calculate the total model loss;

[0373] Receive the second - party gradient of the total model loss with respect to the second - party training intermediate outputs sent by the first party, and based on the second - party gradient, optimize the second - party bottom - end feature extraction model to be trained to obtain the second - party bottom - end feature extraction model.

[0374] The specific implementation manner of the federated prediction optimization device of this application is basically the same as that of each embodiment of the above - mentioned federated prediction optimization method, and will not be elaborated here.

[0375] An embodiment of this application provides a medium, the medium is a readable storage medium, and the readable storage medium stores one or more programs, and the one or more programs can also be executed by one or more processors to be used to implement the steps of the federated learning modeling optimization method described in any one of the above.

[0376] The specific implementation manner of the readable storage medium of this application is basically the same as that of each embodiment of the above - mentioned federated learning modeling optimization method, and will not be elaborated here.

[0377] An embodiment of this application provides a medium, the medium is a readable storage medium, and the readable storage medium stores one or more programs, and the one or more programs can also be executed by one or more processors to be used to implement the steps of the federated prediction optimization method described in any one of the above.

[0378] The specific implementation manner of the readable storage medium of the present application is basically the same as that of each embodiment of the above-mentioned federated prediction optimization method, and will not be elaborated here.

[0379] An embodiment of the present application provides a product, which is a computer program product, and the computer program product includes one or more computer programs, and the one or more computer programs can also be executed by one or more processors to be used to implement the steps of the federated learning modeling optimization method described in any one of the above.

[0380] The specific implementation manner of the computer program product of the present application is basically the same as that of each embodiment of the above-mentioned federated learning modeling optimization method, and will not be elaborated here.

[0381] An embodiment of the present application provides a product, which is a computer program product, and the computer program product includes one or more computer programs, and the one or more computer programs can also be executed by one or more processors to be used to implement the steps of the federated prediction optimization method described in any one of the above.

[0382] The specific implementation manner of the computer program product of the present application is basically the same as that of each embodiment of the above-mentioned federated prediction optimization method, and will not be elaborated here.

[0383] The above are only the preferred embodiments of the present application, and do not limit the patent scope of the present application. Any equivalent structure or equivalent process transformation made by using the content of the specification and drawings of the present application, or directly or indirectly applied in other related technical fields, shall be included in the patent scope of the present application by the same token.

Claims

1. A method for optimizing federated learning modeling, characterized in that The above-mentioned federated learning modeling optimization method is applied to the first party, which is a federated learning modeling optimization device, and the second party is another federated learning modeling optimization device. The federated learning modeling optimization device is a physical device, and there is a communication connection between the first party and the second party; The federated learning modeling optimization method includes: Obtain the to-be-trained first-party bottom feature extraction model and the to-be-trained top prediction model, and extract the first-party training samples and the preset true labels corresponding to the first-party training samples; Based on the to-be-trained first-party bottom feature extraction model, jointly convert the first-party training samples and the preset first-party bottom passport data into a first-party training intermediate output with passport embeddings; Based on the first-party training intermediate output, the preset true labels, the preset first-party top passport data, and the to-be-trained top prediction model, through federated interaction with each second party, jointly calculate the model total loss corresponding to the to-be-trained top prediction model with the second-party training intermediate output with passport embeddings generated by each second party. The second-party training intermediate output is obtained by the to-be-trained second-party bottom feature extraction model obtained by the second party to convert the preset second-party bottom passport data and the second-party training samples corresponding to the first-party training samples; Based on the model total loss, optimize the to-be-trained first-party bottom feature extraction model and the to-be-trained top prediction model to obtain the first-party bottom feature extraction model and the top prediction model; Among them, the preset true label is the identifier of the first-party training sample, which is used to identify the category or attribute information of the first-party training sample; The to-be-trained first-party bottom feature extraction model includes a to-be-trained first-party bottom neural network. The to-be-trained first-party bottom neural network includes at least one to-be-trained first-party passport embedding module. The preset first-party bottom passport data includes at least one first-party passport sample to be embedded corresponding to the to-be-trained first-party passport embedding module. The first-party passport sample to be embedded is a picture, text, or a specific coding matrix. The to-be-trained first-party passport embedding module is used to embed the first-party passport sample to be embedded into the model intermediate output of the first-party training sample in the to-be-trained first-party bottom feature extraction model; The second-party training samples correspond to the same common sample ID as the first-party training samples, and the common sample ID is determined when the first party aligns samples with each second party.

2. The federated learning modeling optimization method according to claim 1, wherein The step of calculating the model total loss corresponding to the to-be-trained top prediction model through federated interaction with each second party, jointly with the second-party training intermediate output with passport embeddings generated by each second party, based on the first-party training intermediate output, the preset true labels, the preset first-party top passport data, and the to-be-trained top prediction model includes: Receive the second-party training intermediate output with passport embeddings sent by each second party; Aggregate the first-party training intermediate output and the second-party training intermediate outputs of each second party to obtain an aggregated training intermediate output; Through the to-be-trained top prediction model, jointly convert the aggregated training intermediate output and the preset first-party top passport data into an output prediction label; Based on the output prediction label and the preset true label, calculate the total model loss.

3. The federated learning modeling optimization method according to claim 2, wherein The to-be-trained top prediction model includes a to-be-trained top neural network, the to-be-trained top neural network at least includes a to-be-trained top passport embedding module, and the preset first-party top passport data at least includes a top passport embedding sample corresponding to the to-be-trained top passport embedding module. The step of jointly converting the aggregated training intermediate output and the preset first-party top passport data into an output prediction label through the to-be-trained top prediction model includes: Based on the first part of the to-be-trained top neural network in the to-be-trained top neural network that is before the to-be-trained top passport embedding module, convert the aggregated training intermediate output into a to-be-trained top network intermediate output; Based on the to-be-trained top passport embedding module, jointly convert the to-be-trained top network intermediate output and the top passport embedding sample into an output of the to-be-trained top passport embedding module; Based on the second part of the to-be-trained top neural network in the to-be-trained top neural network that is after the to-be-trained top passport embedding module, convert the output of the to-be-trained top passport embedding module into the output prediction label.

4. The federated learning modeling optimization method according to claim 3, wherein The to-be-trained top passport embedding module includes a to-be-trained top passport embedding layer and a to-be-trained module neural network layer. The step of jointly converting the to-be-trained top network intermediate output and the top passport embedding sample into an output of the to-be-trained top passport embedding module based on the to-be-trained top passport embedding module includes: Based on the to-be-trained module neural network layer, linearly transform the to-be-trained network intermediate output into an output of the to-be-trained embedding network layer, and linearly transform the top passport embedding sample into a to-be-embedded passport; Based on the passport function in the first-party passport embedding layer, convert the to-be-embedded passport into passport embedding parameters; Based on the output of the to-be-trained embedding network layer and the passport embedding parameters, generate the output of the to-be-trained top passport embedding module.

5. The federated learning modeling optimization method according to claim 1, wherein The step of jointly converting the first-party training sample and the preset first-party bottom passport data into a first-party training intermediate output with passport embedding based on the to-be-trained first-party bottom feature extraction model includes: Based on the first part of the to-be-trained first-party bottom neural network in the to-be-trained first-party bottom neural network that is before the to-be-trained first-party passport embedding module, convert the first-party training sample into a to-be-trained first-party bottom network intermediate output; Based on the to-be-trained first-party passport embedding module, jointly convert the to-be-trained first-party bottom network intermediate output and the first-party to-be-embedded passport sample into an output of the to-be-trained second-party bottom passport embedding module; Based on the second part of the to-be-trained first-party bottom neural network in the to-be-trained first-party bottom neural network that is after the to-be-trained first-party passport embedding module, convert the output of the to-be-trained second-party bottom passport embedding module into the first-party training intermediate output.

6. The federated learning modeling optimization method according to claim 1, wherein, After the step of calculating the total model loss corresponding to the to-be-trained top prediction model based on the first-party training intermediate output, the preset true label, the preset first-party top passport data, and the second-party training intermediate output with passport embeddings generated by jointly aggregating each second party through federal interaction with each second party, the federated learning modeling optimization method further includes: Calculating the second-party gradients of the total model loss with respect to each second-party training intermediate output; Sending each of the second-party gradients to its corresponding second party for the second party to optimize the to-be-trained second-party bottom feature extraction model based on the second-party gradients to obtain a second-party bottom feature extraction model.

7. The federated learning modeling optimization method according to claim 1, wherein After the step of optimizing the to-be-trained first-party bottom feature extraction model and the to-be-trained top prediction model based on the total model loss to obtain a first-party bottom feature extraction model and a top prediction model, the federated learning modeling optimization method further includes: Obtaining a first-party sample to be predicted, and based on the first-party bottom feature extraction model, jointly converting the first-party sample to be predicted and the preset first-party bottom passport data into a first-party intermediate output with passport embeddings; Receiving the second-party intermediate outputs with passport embeddings sent by each second party, where the second-party intermediate output is obtained by the second party converting the preset second-party bottom passport data and the second-party sample to be predicted corresponding to the first-party sample to be predicted based on the second-party bottom feature extraction model; Aggregating the first-party intermediate output and each second-party intermediate output to obtain an aggregated intermediate output; Based on the top prediction model, jointly converting the aggregated intermediate output and the preset first-party top passport data into a target prediction result.

8. A method for optimizing federated learning modeling, characterized in that The federated learning modeling optimization method is applied to a second party, the second party is a federated learning modeling optimization device, the first party is another federated learning modeling optimization device, the federated learning modeling optimization device is a physical device, and the first party and the second party are communicatively connected; The federated learning modeling optimization method includes: Obtaining a to-be-trained second-party bottom feature extraction model and extracting second-party training samples; Based on the to-be-trained second-party bottom feature extraction model, converting the second-party training samples and the preset second-party bottom passport data into a second-party training intermediate output; Sending the second-party training intermediate output to the first party for the first party to aggregate the second-party training intermediate outputs sent by each second party, and the first-party training intermediate output obtained by converting the first-party training samples and the preset first-party bottom passport data, to obtain an aggregated training intermediate output, and calculating the total model loss based on the output prediction label obtained by converting the aggregated training intermediate output and the preset first-party top passport data, and the preset true label corresponding to the first-party training samples; Receive the second-party gradient of the total model loss sent by the first party with respect to the intermediate output of the second-party training, and optimize the second-party bottom feature extraction model to be trained based on the second-party gradient to obtain the second-party bottom feature extraction model; Wherein, the preset true label is the identifier of the first-party training sample, and is used to identify the category or attribute information of the first-party training sample; The second-party bottom feature extraction model to be trained includes a second-party bottom neural network to be trained, and the second-party bottom neural network to be trained includes at least one second-party passport embedding module to be trained. The preset second-party bottom passport data includes at least one second-party passport sample to be embedded corresponding to the second-party passport embedding module to be trained. The second-party passport sample to be embedded is a picture sample, a text sample or a specific coding matrix, and the second-party passport embedding module to be trained is used to embed the second-party passport sample to be embedded into the intermediate output of the model of the second-party training sample; The second-party training sample corresponds to the first-party training sample with the same common sample ID, and the common sample ID is determined when the first party aligns samples with each second party.

9. The federated learning modeling optimization method according to claim 8, wherein The step of converting the second-party training sample and the preset second-party bottom passport data into the second-party training intermediate output based on the second-party bottom feature extraction model to be trained includes: Based on the first part of the second-party bottom neural network to be trained before the second-party passport embedding module to be trained, convert the second-party training sample into the intermediate output of the second-party bottom network to be trained; Based on the second-party passport embedding module to be trained, jointly convert the intermediate output of the second-party bottom network to be trained and the second-party passport sample to be embedded into the output of the second-party bottom passport embedding module to be trained; Based on the second part of the second-party bottom neural network to be trained after the second-party passport embedding module to be trained, convert the output of the second-party bottom passport embedding module to be trained into the second-party training intermediate output.

10. The federated learning modeling optimization method according to claim 8, wherein After the step of receiving the second-party gradient of the total model loss sent by the first party with respect to the intermediate output of the second-party training, and optimizing the second-party bottom feature extraction model to be trained based on the second-party gradient to obtain the second-party bottom feature extraction model, the federated learning modeling optimization method further includes: Obtain the second-party sample to be predicted, and based on the second-party bottom feature extraction model, jointly convert the second-party sample to be predicted and the preset second-party bottom passport data into the second-party intermediate output with passport embedding; Send the second-party intermediate output to the first party for the first party to generate a target prediction result based on the second-party intermediate outputs sent by each second party, the first-party samples to be predicted corresponding to the second-party samples to be predicted, the preset first-party bottom passport data, and the preset first-party top passport data, through the first-party bottom feature extraction model constructed by federated learning based on the preset first-party bottom passport data and the top prediction model constructed by federated learning based on the preset first-party top passport data.

11. A federal prediction optimization method, characterized in that, The federated prediction optimization method is applied to the first party, the first party is a federated prediction optimization device, the second party is another federated prediction optimization device, the federated prediction optimization device is a physical device, and the first party and the second party are communicatively connected; the federated prediction optimization method includes: Obtain the first-party samples to be predicted, the preset first-party bottom passport data, and the preset first-party top passport data; The first-party bottom feature extraction model constructed by federated learning based on the preset first-party bottom passport data converts the first-party samples to be predicted and the preset first-party bottom passport data together into a first-party intermediate output with passport embeddings; Receive the second-party intermediate outputs with passport embeddings sent by each second party, where the second-party intermediate output is obtained by the second-party bottom feature extraction model constructed by the second party through federated learning based on the preset second-party bottom passport data to convert the preset second-party bottom passport data and the second-party samples to be predicted corresponding to the first-party samples to be predicted; Aggregate the first-party passport embedding intermediate output and each second-party passport embedding intermediate output to obtain an aggregated passport embedding intermediate output; The top prediction model constructed by federated learning based on the preset first-party top passport data converts the aggregated passport embedding intermediate output and the preset first-party top passport data into a target prediction result.

12. The federal prediction optimization method according to claim 11, wherein The first-party bottom feature extraction model includes a first-party bottom neural network, the first-party bottom neural network includes at least one first-party passport embedding module, and the preset first-party bottom passport data includes at least one first-party passport sample to be embedded corresponding to the first-party passport embedding module. The step of the first-party bottom feature extraction model constructed by federated learning based on the preset first-party bottom passport data converting the first-party samples to be predicted and the preset first-party bottom passport data together into a first-party intermediate output with passport embeddings includes: Based on the first part of the first-party bottom neural network in the first-party bottom neural network before the first-party passport embedding module, convert the first-party samples to be predicted into a first-party bottom network intermediate output; Based on the first-party passport embedding module, convert the first-party bottom network intermediate output and the first-party passport sample to be embedded together into a first-party bottom passport embedding module output; Based on the second part of the first - party bottom neural network after the first - party passport embedding module in the first - party bottom neural network, convert the output of the first - party bottom passport embedding module into the first - party intermediate output with passport embedding.

13. The federal prediction optimization method according to claim 12, wherein, The first - party passport embedding module includes a first - party passport embedding layer and a first - party module neural network layer. The step of converting the first - party bottom - network intermediate output and the first - party passport sample to be embedded into the output of the first - party bottom passport embedding module based on the first - party passport embedding module includes: Based on the first - party module neural network layer, linearly transform the first - party bottom - network intermediate output into the output of the network layer to be embedded, and linearly transform the first - party passport sample to be embedded into the passport to be embedded. Based on the passport function in the first - party passport embedding layer, convert the passport to be embedded into passport embedding parameters. Based on the output of the network layer to be embedded and the passport embedding parameters, generate the output of the second - party bottom passport embedding module.

14. The federal prediction optimization method according to claim 11, wherein The top - end prediction model includes a top - end neural network. The top - end neural network at least includes a top - end network passport embedding module. The preset first - party top - end passport data at least includes a top - end passport embedding sample corresponding to the top - end network passport embedding module. The step of converting the aggregated passport embedding intermediate output and the preset first - party top - end passport data into a target prediction result by the top - end prediction model constructed by federated learning based on the preset first - party top - end passport data includes: Based on the first part of the top - end neural network before the top - end network passport embedding module in the top - end neural network, convert the aggregated passport embedding intermediate output into the top - end network layer intermediate output. Based on the top - end passport embedding module, jointly convert the top - end network layer intermediate output and the top - end passport embedding sample into the output of the top - end passport embedding module. Based on the second part of the top - end neural network after the top - end network passport embedding module in the top - end neural network, convert the output of the top - end passport embedding module into the target prediction result.

15. The federal prediction optimization method according to claim 11, characterized in that, Before the step of converting the first - party sample to be predicted and the preset first - party bottom - end passport data into the first - party intermediate output with passport embedding by the first - party bottom - end feature extraction model constructed by federated learning based on the preset first - party bottom - end passport data, the federated prediction optimization method further includes: Obtain the first - party bottom - end feature extraction model to be trained and the top - end prediction model to be trained, and extract the first - party training samples and the preset true labels corresponding to the first - party training samples. Based on the first - party bottom - end feature extraction model to be trained, jointly convert the first - party training samples and the preset first - party bottom - end passport data into the first - party training intermediate output with passport embedding. Based on the first - party training intermediate output, the preset true label, the preset first - party top - level passport data, and the to - be - trained top - level prediction model, through federated interaction with each second - party, jointly calculate the total model loss corresponding to the to - be - trained top - level prediction model with the second - party training intermediate output with passport embeddings generated by each second - party, where the second - party training intermediate output is obtained by the to - be - trained second - party bottom - level feature extraction model acquired by the second - party to transform the preset second - party bottom - level passport data and the second - party training samples corresponding to the first - party training samples; Based on the total model loss, optimize the to - be - trained first - party bottom - level feature extraction model and the to - be - trained top - level prediction model to obtain the first - party bottom - level feature extraction model and the top - level prediction model.

16. The federal prediction optimization method according to claim 15, wherein The step of calculating the total model loss corresponding to the to - be - trained top - level prediction model based on the first - party training intermediate output, the preset true label, the preset first - party top - level passport data, and the to - be - trained top - level prediction model, through federated interaction with each second - party, jointly with the second - party training intermediate output with passport embeddings generated by each second - party includes: Receive the second - party training intermediate output with passport embeddings sent by each second - party; Aggregate the first - party training intermediate output and each second - party training intermediate output to obtain the aggregated training intermediate output; Through the to - be - trained top - level prediction model, jointly transform the aggregated training intermediate output and the preset first - party top - level passport data into output prediction labels; Based on the output prediction labels and the preset true label, calculate the total model loss.

17. The federal prediction optimization method according to claim 15, characterized in that, After the step of calculating the total model loss corresponding to the to - be - trained top - level prediction model based on the first - party training intermediate output, the preset true label, the preset first - party top - level passport data, and the to - be - trained top - level prediction model, through federated interaction with each second - party, jointly with the second - party training intermediate output with passport embeddings generated by each second - party, the federated learning modeling optimization method further includes: Calculate the second - party gradients of the total model loss with respect to each second - party training intermediate output; Send each second - party gradient to its corresponding second - party respectively for the second - party to optimize the to - be - trained second - party bottom - level feature extraction model based on the second - party gradient to obtain the second - party bottom - level feature extraction model.

18. A federal prediction optimization method, characterized in that, The federated prediction optimization method is applied to the second - party, the second - party is a federated prediction optimization device, the first - party is another federated prediction optimization device, the federated prediction optimization device is a physical device, and the first - party and the second - party are communicatively connected; the federated prediction optimization method includes: Obtain the second - party samples to be predicted and the preset second - party bottom - level passport data; Based on the second - party bottom - level feature extraction model constructed by federated learning with the preset second - party bottom - level passport data, jointly transform the second - party samples to be predicted and the preset second - party bottom - level passport data into a second - party intermediate output with passport embeddings; Send the second-party intermediate output to the first party, so that the first party can generate a target prediction result based on the second-party intermediate outputs sent by each second party, the first-party samples to be predicted corresponding to the second-party samples to be predicted, the preset first-party bottom passport data, and the preset first-party top passport data, through the first-party bottom feature extraction model constructed by federated learning based on the preset first-party bottom passport data and the top prediction model constructed by federated learning based on the preset first-party top passport data.

19. The federal prediction optimization method according to claim 18, wherein The second-party bottom feature extraction model includes a second-party bottom neural network, and the second-party bottom neural network includes at least one second-party passport embedding module. The preset second-party bottom passport data includes at least one second-party passport sample to be embedded corresponding to the second-party passport embedding module. In the second-party bottom feature extraction model constructed by federated learning based on the preset second-party bottom passport data, the steps of jointly converting the second-party samples to be predicted and the preset second-party bottom passport data into a second-party intermediate output with passport embedding include: Based on the first part of the second-party bottom neural network in the second-party bottom neural network that is before the second-party passport embedding module, convert the second-party samples to be predicted into a second-party first-party bottom network intermediate output; Based on the second-party passport embedding module, jointly convert the second-party first-party bottom network intermediate output and the second-party passport samples to be embedded into a second-party bottom passport embedding module output; Based on the second part of the second-party bottom neural network in the second-party bottom neural network that is after the second-party passport embedding module, convert the second-party bottom passport embedding module output into the second-party intermediate output with passport embedding.

20. The federal prediction optimization method according to claim 18, characterized in that, Before the step of the second-party bottom feature extraction model constructed by federated learning based on the preset second-party bottom passport data, jointly converting the second-party samples to be predicted and the preset second-party bottom passport data into a second-party intermediate output with passport embedding, the federated prediction optimization method further includes: Obtain a second-party bottom feature extraction model to be trained, and extract second-party training samples; Based on the second-party bottom feature extraction model to be trained, convert the second-party training samples and the preset second-party bottom passport data into a second-party training intermediate output; Send the second-party training intermediate output to the first party, so that the first party aggregates the second-party training intermediate outputs sent by each second party, and based on the first-party training intermediate output converted from the first-party training samples and the preset first-party bottom passport data, obtains an aggregated training intermediate output, and based on the output prediction label converted from the aggregated training intermediate output and the preset first-party top passport data, and the preset true label corresponding to the first-party training samples, calculates the total model loss. Receive the second-party gradient of the total model loss sent by the first party with respect to the intermediate output of the second-party training, and optimize the second-party bottom feature extraction model to be trained based on the second-party gradient, so as to obtain the second-party bottom feature extraction model.

21. A federated learning modeling optimization device, characterized in that, The federated learning modeling optimization device includes: a memory, a processor, and a program stored on the memory for implementing the federated learning modeling optimization method. The memory is used to store the program for implementing the federated learning modeling optimization method. The processor is used to execute the program for implementing the federated learning modeling optimization method to implement the steps of the federated learning modeling optimization method as described in any one of claims 1 to 7 or 8 to 10.

22. A federal prediction optimization device, characterized in that, The federated prediction optimization device includes: a memory, a processor, and a program stored on the memory for implementing the federated prediction optimization method. The memory is used to store the program for implementing the federated prediction optimization method. The processor is used to execute the program for implementing the federated prediction optimization method to implement the steps of the federated prediction optimization method as described in any one of claims 11 to 17 or 18 to 20.

23. A medium, the medium being a readable storage medium, characterized in that, A program for implementing the federated learning modeling optimization method is stored on the readable storage medium, and the program for implementing the federated learning modeling optimization method is executed by the processor to implement the steps of the federated learning modeling optimization method as described in any one of claims 1 to 7 or 8 to 10.

24. A medium, the medium being a readable storage medium, characterized in that, A program for implementing the federated prediction optimization method is stored on the readable storage medium, and the program for implementing the federated prediction optimization method is executed by the processor to implement the steps of the federated prediction optimization method as described in any one of claims 11 to 17 or 18 to 20.

25. A product, the product being a computer program product comprising a computer program, characterized in that, When the computer program is executed by the processor, it implements the steps of the federated learning modeling optimization method as described in any one of claims 1 to 7 or 8 to 10 or implements the steps of the federated prediction optimization method as described in any one of claims 11 to 17 or 18 to 20.

Images