Longitudinal federal prediction optimization method and device, equipment, storage medium and product

Abstract

The invention discloses a longitudinal federal prediction optimization method and device, equipment, a storage medium and a product, and the method comprises the steps: carrying out the sample alignment with second equipment, and determining an aligned sample; obtaining first-party training sample business data, a first sample business label and a first business label training prediction result of the aligned sample, and determining a residual error true value; receiving a training residual prediction value sent by the second equipment; determining a first residual prediction model gradient based on the training residual prediction value and the residual true value, and inputting the training residual prediction value into an adversarial learning model to obtain a second service label training prediction result; and training a prediction result based on the first sample service label and the second service label to determine a second residual prediction model gradient, performing aggregation to obtain a target residual prediction model gradient, and sending the target residual prediction model gradient to a second device to update the residual prediction model. The technical problem that the privacy disclosure risk of longitudinal federal learning is relatively high in related technologies is solved.

Description

Technical Field

[0001] This application relates to the field of artificial intelligence technology in financial technology (Fintech), and particularly to a vertical federated prediction optimization method, device, equipment, storage medium and product. Background Technique

[0002] With the continuous development of financial technology, especially Internet technology finance, more and more technologies (such as distributed, artificial intelligence, etc.) are applied in the financial field, but the financial industry also puts forward higher requirements for technologies.

[0003] The application of artificial intelligence in the financial industry is becoming more and more extensive. The training of models often requires a large amount of sample business data. Federated learning can effectively expand the scale of sample business data, thereby improving the performance of models. Vertical federated learning is to take out the part of users and data with the same users but different sample business data characteristics when the data characteristics of participants overlap less and the user overlap is more for joint machine learning training.

[0004] However, in related technologies, each participating party in vertical federated learning cooperates with the label holder to train a vertical federated model, and the vertical federated model is used to perform label prediction for the label holder. The label holder holds the labels, while the participating party only holds auxiliary features. However, in this way, the participating party can infer the label information of the test samples of the label holder based on the locally trained model, thus bringing a serious risk of privacy leakage. Currently, there are two categories of methods for protecting the privacy of actively participating parties' labels. One is the privacy protection method based on cryptography, which uses technologies such as homomorphic encryption / multi-party secure computing to protect data privacy. However, this type of solution has a huge computational overhead. The other is the privacy protection method based on perturbation, which protects privacy by adding noise to the data. However, this will cause a significant decline in the performance of the federated learning model. Summary of the Invention

[0005] The main purpose of this application is to provide a vertical federated prediction optimization method, device, equipment, storage medium and product, aiming to solve the technical problem of relatively high privacy leakage risk in vertical federated learning in related technologies.

[0006] To achieve the above object, this application provides a vertical federated prediction optimization method. The vertical federated prediction optimization method is applied to a first device in a vertical federated learning system. A business label prediction model and an adversarial learning model are deployed on the first device. The vertical federated learning system further includes a second device, and a residual prediction model is deployed on the second device. The vertical federated prediction optimization method includes the following steps:

[0007] Perform sample alignment with the second device to determine aligned samples;

[0008] Obtain the first-party training sample service data and the first sample service label of the aligned sample, and obtain the first service label training prediction result obtained by the trained service label prediction model for business label prediction based on the first-party training sample service data. Based on the first sample service label and the first service label training prediction result, determine the residual true value;

[0009] Receive the training residual prediction value sent by the second device, where the training residual prediction value is obtained by the second device through the residual prediction model for residual prediction based on the second-party training sample service data of the aligned sample;

[0010] Based on the training residual prediction value and the residual true value, determine the first residual prediction model gradient, and input the training residual prediction value into the adversarial learning model for business label prediction to obtain the second service label training prediction result;

[0011] Based on the first sample service label and the second service label training prediction result, iteratively update the adversarial learning model, and determine the second residual prediction model gradient;

[0012] Aggregate the first residual prediction model gradient and the second residual prediction model gradient to obtain the target residual prediction model gradient, and send the target residual prediction model gradient to the second device for the second device to update the residual prediction model based on the target residual prediction model gradient.

[0013] This application also provides a vertical federated prediction optimization method, which is applied to a second device, and a residual prediction model is deployed on the second device; the vertical federated prediction optimization method includes the following steps:

[0014] Perform sample alignment with the first device to determine the aligned sample, and obtain the second-party training sample service data of the aligned sample;

[0015] Input the second-party training sample service data into the residual prediction model for residual prediction to obtain the training residual prediction value;

[0016] Send the training residual prediction value to the first device for the first device to determine the target residual prediction model gradient based on the training residual prediction value;

[0017] Receive the target residual prediction model gradient sent by the first device, and update the residual prediction model based on the target residual prediction model gradient.

[0018] This application also provides a vertical federated prediction optimization method, which is applied to a third device, and includes the following steps:

[0019] Obtain the first-party business data of the sample to be predicted, and obtain the local business label prediction result obtained by the business label prediction model based on the first-party business data of the sample to be predicted;

[0020] Align samples with at least one second device, so that the target second device containing the second-party business data of the sample to be predicted in each second device performs residual prediction based on the second-party business data of the sample to be predicted through the residual prediction model deployed by itself, where the residual prediction model is trained by using the vertical federated prediction optimization method described above;

[0021] Receive the residual prediction results sent by each target second device, and aggregate the local business label prediction result and the residual prediction results to obtain the federated business label prediction result.

[0022] This application also provides a vertical federated prediction optimization device, which is applied to a first device in a vertical federated learning system. A business label prediction model and an adversarial learning model are deployed on the first device; the vertical federated learning system further includes a second device, and a residual prediction model is deployed on the second device; the vertical federated prediction optimization device includes:

[0023] A first alignment module, configured to align samples with the second device to determine aligned samples;

[0024] A first acquisition module, configured to obtain the first-party training sample business data and the first sample business label of the aligned samples, and obtain the first business label training prediction result obtained by the trained business label prediction model based on the first-party training sample business data, and determine the residual true value based on the first sample business label and the first business label training prediction result;

[0025] A receiving module, configured to receive the training residual prediction value sent by the second device, where the training residual prediction value is obtained by the second device through the residual prediction model based on the second-party training sample business data of the aligned samples;

[0026] A business label prediction module, configured to determine the first residual prediction model gradient based on the training residual prediction value and the residual true value, and input the training residual prediction value into the adversarial learning model for business label prediction to obtain the second business label training prediction result;

[0027] The first update module is used to iteratively update the adversarial learning model based on the first sample service label and the prediction result of the second service label, and determine the gradient of the second residual prediction model;

[0028] The first aggregation module is used to aggregate the gradient of the first residual prediction model and the gradient of the second residual prediction model to obtain the target residual prediction model gradient, and send the target residual prediction model gradient to the second device for the second device to update the residual prediction model based on the target residual prediction model gradient.

[0029] This application also provides a vertical federated prediction optimization device, which is applied to a second device, and a residual prediction model is deployed on the second device; the vertical federated prediction optimization device includes:

[0030] The second alignment module is used to align samples with the first device, determine the aligned samples, and obtain the second-party training sample service data of the aligned samples;

[0031] The residual prediction module is used to input the second-party training sample service data into the residual prediction model for residual prediction to obtain the training residual prediction value;

[0032] The sending module is used to send the training residual prediction value to the first device for the first device to determine the target residual prediction model gradient based on the training residual prediction value;

[0033] The second update module is used to receive the target residual prediction model gradient sent by the first device and update the residual prediction model based on the target residual prediction model gradient.

[0034] This application also provides a vertical federated prediction optimization device, which is applied to a third device, and the vertical federated prediction optimization device includes:

[0035] The second acquisition module is used to acquire the first-party to-be-predicted sample service data of the to-be-predicted sample, and acquire the local service label prediction result obtained by the service label prediction model for service label prediction based on the first-party to-be-predicted sample service data;

[0036] The third alignment module is used to align samples with at least one second device, so that the target second device in each of the second devices that contains the second-party to-be-predicted sample service data corresponding to the to-be-predicted sample performs residual prediction based on the second-party to-be-predicted sample service data through the locally deployed residual prediction model to obtain the residual prediction result, where the residual prediction model is trained by using the vertical federated prediction optimization method as described above;

[0037] A second aggregation module, configured to receive the residual prediction results sent by each of the target second devices, aggregate the local service label prediction results and the residual prediction results, and obtain a federated service label prediction result.

[0038] The present application further provides an electronic device, which is a physical device. The electronic device includes: a memory, a processor, and a program of the longitudinal federated prediction optimization method stored on the memory and executable on the processor. When the program of the longitudinal federated prediction optimization method is executed by the processor, the steps of the longitudinal federated prediction optimization method as described above can be implemented.

[0039] The present application further provides a storage medium, which is a computer-readable storage medium. A program for implementing the longitudinal federated prediction optimization method is stored on the computer-readable storage medium. When the program of the longitudinal federated prediction optimization method is executed by the processor, the steps of the longitudinal federated prediction optimization method as described above are implemented.

[0040] The present application further provides a computer program product, including a computer program. When the computer program is executed by the processor, the steps of the longitudinal federated prediction optimization method as described above are implemented.

[0041] The present application provides a vertical federated prediction optimization method, apparatus, device, storage medium and product. The vertical federated prediction optimization method is applied to a first device in a vertical federated learning system. A business label prediction model and an adversarial learning model are deployed on the first device. The vertical federated learning system further includes a second device on which a residual prediction model is deployed. First, sample alignment is performed with the second device to determine aligned samples, and the first-party training sample business data and the first sample business label of the aligned samples are obtained. The first business label training prediction result obtained by the trained business label prediction model for business label prediction based on the first-party training sample business data is also obtained. Based on the first sample business label and the first business label training prediction result, the residual true value is determined, realizing the determination of the residual true value generated by the trained business label prediction model for label prediction. Then, the training residual prediction value sent by the second device is received, where the training residual prediction value is obtained by the second device through the residual prediction model for residual prediction based on the second-party training sample business data of the aligned samples. In this way, the second-party training sample business data owned by the second device can be used for residual prediction, achieving the purpose of determining the residual prediction value. Then, based on the training residual prediction value and the residual true value, the first residual prediction model gradient is determined, and the training residual prediction value is input into the adversarial learning model for business label prediction to obtain the second business label training prediction result. The adversarial learning model is iteratively updated based on the first sample business label and the second business label training prediction result, and the second residual prediction model gradient is determined. The first residual prediction model gradient determined in this way can effectively improve the model performance of the residual prediction model deployed on the second device. However, in order to better protect the label information privacy of the first device, the adversarial learning model is used to perform label prediction based on the training residual prediction value, and then the second residual prediction model gradient is obtained. Then, by aggregating the first residual prediction model gradient and the second residual prediction model gradient, the target residual prediction model gradient is obtained, and the target residual prediction model gradient is sent to the second device for the second device to update the residual prediction model based on the target residual prediction model gradient, realizing the trade-off of the first residual prediction model gradient by the second residual prediction model gradient. In this way, the trade-off between label prediction accuracy and privacy protection can be achieved.In this way, during the entire training process, only the training residual prediction values and the gradients of the first residual prediction model are shared between the first device and the second device. For the second device, since the first device cannot know which sample features the second device uses to obtain the training residual prediction values, it is impossible to infer the second-party training sample service data owned by the second device. Therefore, privacy protection for the second device can be achieved. For the first device, since the residual is the difference between the first sample service label and the local model training prediction result, which reflects the model performance. When the second device neither knows the first-party training sample service data, nor the first sample service label, nor the service label prediction model, the second device cannot infer the label information or privacy data owned by the first device only based on the residual. Therefore, decoupling of the label information and the residual information can be achieved, thereby realizing privacy protection for the first device. Thus, it overcomes the technical defect that each participating party in vertical federated learning cooperates with the label holder to train a vertical federated model, and then uses the vertical federated model to perform label prediction for the label holder. In this way, the participating party can infer the labels of the test samples of the label holder based on the locally trained model, thus bringing a serious risk of privacy leakage. Moreover, compared with the service label prediction model trained only based on local data, vertical federated learning can utilize a variety of sample service data distributed on multiple second devices to more accurately predict the residual between the prediction result and the true result of the service label prediction model, thereby correcting the prediction result of the service label prediction model, that is, improving the accuracy of the final output prediction result, and can balance the requirements of label prediction accuracy and privacy protection through the adversarial learning model. Therefore, both prediction accuracy can be improved and privacy leakage can be avoided. And since the privacy protection method of this application is in plain text throughout the process and does not involve encryption algorithms, the learning and prediction processes are very efficient. Therefore, it takes into account the efficiency, performance, and privacy of federated learning, and has high practical value and commercial value. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] The accompanying drawings herein are incorporated into the specification and constitute a part of this specification, showing embodiments consistent with the present application, and are used together with the specification to explain the principles of the present application.

[0043] In order to more clearly illustrate the technical solutions in the embodiments of the present application or related technologies, the following will briefly introduce the drawings required for use in the description of the embodiments or related technologies. Obviously, for those of ordinary skill in the art, without creative efforts, other drawings can also be obtained based on these drawings.

[0044] Figure 1 Schematic flowchart of the first embodiment of the vertical federated prediction optimization method of the present application;

[0045] Figure 2 It is a schematic diagram of a scenario of an implementable manner of the federated learning system in the embodiments of the present application;

[0046] Figure 3 It is a schematic flowchart of the second embodiment of the vertical federated prediction optimization method of the present application;

[0047] Figure 4 It is a schematic flowchart of the third embodiment of the vertical federated prediction optimization method of the present application;

[0048] Figure 5 It is a schematic structural diagram of the vertical federated prediction optimization device in the embodiments of the present application;

[0049] Figure 6 It is a schematic diagram of the device structure of the hardware operating environment involved in the vertical federated prediction optimization method in the embodiments of the present application.

[0050] The realization of the purpose, functional features and advantages of the present application will be further described with reference to the embodiments and the accompanying drawings. Specific Embodiments

[0051] In order to make the above objects, features and advantages of the present invention more obvious and understandable, the technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all the embodiments. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present invention without creative efforts shall fall within the protection scope of the present invention.

[0052] Artificial intelligence is more and more widely used in the financial industry. The training of models often requires a large amount of sample business data. Each enterprise can obtain limited sample business data, and the features are relatively single. The user portraits depicted may be inaccurate. If each enterprise only trains a prediction model based on the local sample business data it owns, the performance of the obtained prediction model is limited and the prediction accuracy is low. Therefore, federated learning can be used to expand the scale of sample business data, thereby improving the model performance.

[0053] Vertical federated learning is to extract the part of users and data with the same users but different sample business data features when the data features of the participating parties overlap less and the user overlap is more. In a vertical federated learning system, it usually includes a participating party with label information and a participating party without label information. For the convenience of description, in the following embodiments, the first device refers to the participating party with label information, and the first device needs to perform a business label prediction task. The second device refers to the participating party without label information, and the second device can assist the first device to complete the business label prediction task and improve the prediction accuracy of the business label prediction task.

[0054] Exemplarily, the business label prediction task can be the prediction of information click-through rate. The first device can be a device of an information recommendation platform, such as a video website, a short video platform, etc. The second device can be a device of an e-commerce company, a financial institution, a bank, etc. The information recommendation platform can cooperate with an e-commerce company, a financial institution, a bank, etc. to jointly predict the information click behavior of the common users. Thus, the information recommendation platform can recommend information to users more accurately. The information recommendation platform has the click records of users and can use the click records as sample business labels for model training. The information recommendation platform can also have first-party sample business data such as the information browsing behavior and page stay time of users. These first-party sample business data can be used for local training of the business label prediction model deployed on its own side, so as to better predict the information click-through rate. The second device can have other sample business data of the same users as the information recommendation platform, such as purchase behavior, purchase preference, financial management information, etc. These second-party sample business data can provide auxiliary features for the business label prediction model deployed on the first device, so as to correct the information click-through rate predicted by the business label prediction model deployed on the first device and obtain a more accurate information click-through rate. In this process, neither the information recommendation platform nor an e-commerce company, a financial institution, a bank, etc. wants to disclose the privacy data of users.

[0055] In the related art, a first device and each second device cooperate to train a vertical federated model, which is used to predict labels for the first device. The label holder holds the labels, while the participating parties only hold auxiliary features. However, in this way, the participating parties can infer the label information of the test samples of the label holder based on the locally trained model, thus bringing a serious risk of privacy leakage. Currently, there are two types of methods for protecting the privacy of the actively participating label. One is the privacy protection method based on cryptography, which uses technologies such as homomorphic encryption / multi-party secure computation to protect data privacy. However, such a scheme has a huge computational overhead. The other is the privacy protection method based on perturbation, which protects privacy by adding noise to the data. However, this will cause a significant decline in the performance of the federated learning model.

[0056] Currently, the methods for protecting the label privacy of the first device include privacy protection methods based on cryptography, that is, using technologies such as homomorphic encryption or multi-party secure computation to protect data privacy. However, such a scheme has a huge computational overhead. In addition, the methods for protecting the label privacy of the first device also include privacy protection methods based on perturbation, that is, protecting privacy by adding noise to the data. However, this will cause a significant decline in the performance of the vertical federated learning model.

[0057] In this application, a business label prediction model is deployed on the first device, and a residual prediction model is deployed on the second device. The entire business label prediction task is divided into two parts: prediction and result correction based on residuals. In this way, the first device can train the business label prediction model only based on the sample business data it locally has. At the same time, due to the limited sample business data locally available, the performance of the business label prediction model is limited, and the prediction accuracy is relatively low. There is still a large prediction residual between the prediction result obtained by using the business label prediction model to predict all aligned data and the true result. Therefore, the sample business data of the samples aligned between the second device and the first device is used to predict the prediction residual of the business label prediction model on the first device through the residual prediction model on the second device. By jointly using the prediction residual of the second device to correct the prediction result of the business label prediction model, a federated business label prediction result closer to the true result can be obtained, thus achieving the purpose of improving the prediction effect. Further, in order to better protect the privacy of the label information of the first device, an adversarial learning model is used to perform label prediction based on the training residual prediction value, and then the gradient of the second residual prediction model is obtained. Then, by aggregating the gradient of the first residual prediction model and the gradient of the second residual prediction model, the target residual prediction model gradient is obtained, and the target residual prediction model gradient is sent to the second device for the second device to update the residual prediction model based on the target residual prediction model gradient, realizing the trade-off of the gradient of the first residual prediction model by the gradient of the second residual prediction model. In this way, the trade-off between label prediction accuracy and privacy protection can be achieved. Moreover, during the entire training process, only the training residual prediction value and the gradient of the first residual prediction model are shared between the first device and the second device. For the second device, since the first device cannot know which sample features the second device uses to obtain the training residual prediction value, it cannot infer the second-party training sample business data owned by the second device. Therefore, the privacy protection of the second device can be achieved; for the first device, since the residual is the difference between the first sample business label and the local model training prediction result, which reflects the model performance. When the second device neither knows the first-party training sample business data, nor the first sample business label, nor the business label prediction model, the second device cannot infer the label information or privacy data owned by the first device only based on the residual. Therefore, the decoupling of label information and residual information can be achieved, thereby realizing the privacy protection of the first device. Therefore, it overcomes the technical defect that each participating party in vertical federated learning cooperates with the label holder to train a vertical federated model, and then uses the vertical federated model to perform label prediction for the label holder. In this way, the participating party can infer the label of the test sample of the label holder based on the locally trained model, thus bringing a serious risk of privacy leakage.Moreover, compared with the service label prediction model trained only based on local data, vertical federated learning can utilize diverse sample service data distributed across multiple second devices to more accurately predict the residual between the prediction result and the true result of the service label prediction model, thereby correcting the prediction result of the service label prediction model, that is, improving the accuracy of the finally output prediction result, and the requirements of both label prediction accuracy and privacy protection can be balanced through the adversarial learning model. Therefore, both the prediction accuracy can be improved and privacy leakage can be avoided. And since the privacy protection method of this application is in plaintext throughout the process and does not involve encryption algorithms, the learning and prediction processes are very efficient. Therefore, it takes into account the efficiency, performance, and privacy of federated learning and has high practical and commercial value.

[0058] Embodiment 1

[0059] An embodiment of this application provides a vertical federated prediction optimization method. In the first embodiment of the vertical federated prediction optimization method of this application, the vertical federated prediction optimization method is applied to a first device in a vertical federated learning system. A service label prediction model and an adversarial learning model are deployed on the first device; the vertical federated learning system further includes a second device, and a residual prediction model is deployed on the second device; referring to Figure 1 , the vertical federated prediction optimization method includes the following steps:

[0060] Step S10, perform sample alignment with the second device to determine aligned samples;

[0061] The execution subject of the method in this embodiment can be a vertical federated prediction optimization device, or a vertical federated prediction optimization terminal device or server. This embodiment takes the vertical federated prediction optimization device as an example. The vertical federated prediction optimization device can be integrated on terminal devices such as smartphones and computers with data processing functions.

[0062] As an example, the step S10 includes: the first device performs sample alignment with the second device to determine aligned samples. Among them, the sample alignment relationship between each first-party training sample service data and each second-party training sample service data can be determined by aligning sample IDs, or the sample alignment relationship between each first-party training sample service data and each second-party training sample service data can be determined by other identification information of the aligned samples, which can be specifically determined according to the actual situation. This embodiment does not limit this, and for the sake of illustration, subsequent descriptions will be made taking the aligned sample ID as an example.

[0063] In an implementable manner, there may be multiple second devices. In this case, the aligned samples refer to the samples in which the first device is aligned with at least one second device. That is, the first device has the sample data of all the aligned samples, and each second device may have the sample data of some or all of the aligned samples.

[0064] Step S20: Obtain the first-party training sample service data and the first sample service label of the aligned samples, and obtain the first service label training prediction result obtained by the trained service label prediction model for business label prediction based on the first-party training sample service data. Based on the first sample service label and the first service label training prediction result, determine the residual true value;

[0065] In this embodiment, it should be noted that the vertical federated learning system includes at least one first device and at least one second device corresponding to each first device. The first device may have the first-party training sample service data of multiple aligned samples, as well as the first sample service label corresponding to each of the aligned samples; for each second device, it may have the second-party training sample service data of some or all of the aligned samples, and through the sample alignment method, each second device can extract the second-party training sample service data of the aligned samples from the sample service data it owns. For example, the first device has the deposit data of aligned sample S1, aligned sample S2, aligned sample S3, and aligned sample S4. The first-party training sample service data is the deposit data of aligned sample S1, aligned sample S2, aligned sample S3, and aligned sample S4. The second device P1 has the consumption data of S1, S2, S3, and aligned sample S5, and the second device P2 has the credit investigation data of S1, S4, and aligned sample S6. Through sample alignment, it can be determined that the second-party training sample service data owned by P1 is the consumption data of S1, S2, and S3, and the second-party training sample service data owned by P2 is the credit investigation data of S1 and S4. The way for each second device to perform federated training with the first device is similar. For the convenience of description, in the following, one first device and one second device corresponding to this first device will be taken as an example for description.

[0066] The vertical federated prediction optimization method includes the training process of a vertical federated learning model. The vertical federated learning model includes a service label prediction model and an adversarial learning model deployed on the first device, and a residual prediction model deployed on each of the second devices. Among them, the service label prediction model can be determined according to the actual service label prediction task and the form of sample data. For example, a convolutional neural network model can be selected for image data, and a classification model can be selected for classification tasks, etc. This embodiment does not limit this; the adversarial learning model is used to generate the gradient of the second residual prediction model. The gradient of the second residual prediction model can be aggregated with the true gradient of the first residual prediction model as an interference value, and then the residual prediction model of the second device is interfered, so that it is more difficult to infer the label information owned by the first device from the residual prediction value predicted based on the residual prediction model, thereby realizing the privacy protection of the first device. The introduction of the adversarial learning model will, to a certain extent, reduce the prediction accuracy of the residual prediction model. The trade-off curve can be combined, and by setting a smaller weight for the gradient of the second residual prediction model, the design of the adversarial learning model, etc., the prediction accuracy and privacy protection can be balanced. The adversarial learning model in this embodiment is a model for predicting service labels based on the training of residual prediction values. The interference value, that is, the gradient of the second residual prediction model, is determined based on the predicted result of the second service label training output by the adversarial learning model and the first sample service label. The introduced interference value can not only effectively increase the difficulty of inferring service label information from the residual prediction value output based on the residual prediction model and improve the privacy protection of the first device, but also better reduce the impact of the interference value on the model performance, and can better balance privacy protection and model performance; the residual prediction model can be an improved model based on residual learning, and the true value of the residual of the service label prediction model on the first device is fitted by executing the forward propagation algorithm. Therefore, the training process of the vertical federated learning model requires the joint completion of the first device and the second device. The vertical federated prediction optimization method provided in this embodiment is applied to the first device in the vertical federated learning system.

[0067] During the training process of the vertical federated learning model, first train the service label prediction model, and then use the trained service label prediction model to train the adversarial learning model and the residual prediction model. The training of the service label prediction model can be carried out on the first device or on other devices, and then deploy the trained service label prediction model to the first device, and train the residual prediction model through the first device. This embodiment does not limit this. The service label prediction model is used to predict service labels based on the sample service data owned by the first device. After the service label prediction model is trained, the residual true value between the first service label training prediction result obtained by the service label prediction model performing the service label prediction task and the first sample service label owned by the first device can be used as the true value for the residual prediction model on the second device to perform residual prediction; the residual prediction model is used to predict the residual true value of the service label prediction model on the first device to obtain a predicted value of the residual true value; thus, the residual prediction model on the second device can be updated by the gradient descent method. The adversarial learning model can perform service label prediction based on the predicted value of the residual true value, and the obtained second service label training prediction result can be compared with the first sample service label to iteratively update the adversarial learning model.

[0068] As an example, the step S20 includes: obtaining a batch of first-party training sample service data and the corresponding first sample service labels of the first-party training sample service data; further, inputting the first-party training sample service data into the trained service label prediction model to perform service label prediction to obtain a first service label training prediction result; further, calculating the residual true value according to the difference between the first sample service label and the first service label training prediction result.

[0069] In an implementable manner, the service label prediction model may include a feature extractor, a fully connected layer, and an activation layer. The step of inputting the first-party training sample service data into the trained service label prediction model to perform service label prediction to obtain a first service label training prediction result includes: after inputting the first-party training sample service data into the trained service label prediction model, first perform feature extraction on the first-party training sample service data based on the feature extractor to obtain first-party training sample features, and then the first-party training sample features can be concatenated into a first-party training sample feature vector or a first-party training sample feature matrix, perform a full connection on the first-party training sample feature vector or the first-party training sample feature matrix through the fully connected layer, and then activate the output of the fully connected layer through the activation function preset by the activation layer to obtain a first service label training prediction result.

[0070] Further, before the step of obtaining the first - party training sample service data and the first sample service label of the alignment sample, and obtaining the first service label training prediction result obtained by the trained service label prediction model for predicting the service label based on the first - party training sample service data, and determining the true value of the residual based on the first sample service label and the first service label training prediction result, the method further includes:

[0071] Obtain the service label prediction model training sample service data and the second sample service label, and iteratively optimize the service label prediction model based on the service label prediction model training sample service data and the second sample service label to obtain the trained service label prediction model.

[0072] In this embodiment, it should be noted that since the true value of the residual prediction by the residual prediction model on the second device is determined by the service label prediction model on the first device, before iteratively optimizing the residual prediction model on the second device, it is necessary to first complete the training of the service label prediction model to ensure the correctness of the true value used for training the residual prediction model.

[0073] Exemplarily, before iteratively optimizing the residual prediction model on the second device, first initialize the service label prediction model; then obtain a batch of service label prediction model training sample service data and the second sample service label from the local sample data of the first device, input the service label prediction model training sample service data into the service label prediction model to perform service label prediction, obtain the third service label training prediction result, calculate the service label prediction model loss according to the difference between the third service label training prediction result and the second sample service label, determine whether the service label prediction model loss converges. If the service label prediction model loss converges, it is determined that the training of the service label prediction model is completed. If the service label prediction model loss does not converge, update the service label prediction model once according to the service label prediction model gradient calculated from the service label prediction model loss, and return to execute the step of obtaining the service label prediction model training sample service data and the second sample service label until the service label prediction model converges to obtain the trained service label prediction model.

[0074] Step S30, receive the training residual prediction value sent by the second device, where the training residual prediction value is obtained by the second device through the residual prediction model based on the second - party training sample service data of the alignment sample for residual prediction;

[0075] As an example, step S30 includes: after the samples are aligned, the second device can, based on the aligned sample ID, search for the second-party training sample service data corresponding to the aligned sample ID from the sample service data it owns; then input the second-party training sample service data into the residual prediction model deployed by itself for residual learning, and calculate the training residual prediction value based on the forward propagation algorithm; then send the training residual prediction value to the first device. Therefore, the first device can receive the training residual prediction value sent by the second device. It should be noted that the training residual prediction value also corresponds one-to-one with the aligned sample ID. Therefore, the training residual prediction value can be made to correspond one-to-one with the service label prediction residual determined on the first device based on the aligned sample ID.

[0076] Step S40: Based on the training residual prediction value and the true residual value, determine the gradient of the first residual prediction model, and input the training residual prediction value into the adversarial learning model for service label prediction to obtain the second service label training prediction result;

[0077] As an example, step S40 includes: after receiving the training residual prediction value sent by the second device, the training residual prediction value is used for calculating the gradient of the first residual prediction model on the one hand and for service label prediction on the other hand. Among them, the way the training residual prediction value is used for calculating the gradient of the first residual prediction model is to calculate the loss function of the residual prediction loss of the residual prediction model on the second device according to the difference value between the training residual prediction value and the true residual value, and then calculate the gradient of the loss function of the second device with respect to the training residual prediction value for the first residual prediction model; the way the training residual prediction value is used for service label prediction includes inputting the training residual prediction value into the adversarial learning model for service label prediction to obtain the second service label training prediction result.

[0078] In an implementable manner, the adversarial learning model may include a feature extractor, a fully connected layer, and an activation layer. The step of inputting the training residual prediction value into the adversarial learning model for service label prediction to obtain the second service label training prediction result includes: after inputting the training residual prediction value into the adversarial learning model, first perform feature extraction on the training residual prediction value based on the feature extractor to obtain the training residual prediction value features, and then the training residual prediction value features can be concatenated into a training residual prediction value feature vector or a training residual prediction value feature matrix, perform full connection on the training residual prediction value feature vector or the training residual prediction value feature matrix through the fully connected layer, and then activate the output of the fully connected layer through the activation function preset by the activation layer to obtain the second service label training prediction result.

[0079] Step S50: Iteratively update the adversarial learning model based on the first sample service label and the training prediction result of the second service label, and determine the gradient of the second residual prediction model;

[0080] As an example, step S50 includes: calculating the loss of the adversarial learning model according to the difference between the first sample service label and the training prediction result of the second service label; calculating the gradient of the adversarial learning model based on the loss of the adversarial learning model, performing one round of update on the adversarial learning model, and calculating the gradient of the second residual prediction model of the loss of the adversarial learning model with respect to the training residual prediction value.

[0081] Further, the step of iteratively updating the adversarial learning model based on the first sample service label and the training prediction result of the second service label, and determining the gradient of the second residual prediction model includes:

[0082] Step S51: Determine the loss of the adversarial learning model according to the difference between the first sample service label and the training prediction result of the second service label;

[0083] Step S52: Calculate the gradient of the adversarial learning model of the loss of the adversarial learning model with respect to the training prediction result of the second service label, iteratively update the adversarial learning model based on the gradient of the adversarial learning model, and calculate the gradient of the second residual prediction model of the loss of the adversarial learning model with respect to the training residual prediction value.

[0084] As an example, steps S51 - S52 include: calculating the loss of the adversarial learning model according to the difference between the first sample service label and the training prediction result of the second service label; taking the gradient of the loss of the adversarial learning model with respect to the training prediction result of the second service label to obtain the gradient of the adversarial learning model, performing one round of update on the adversarial learning model based on the gradient of the adversarial learning model; and taking the gradient of the loss of the adversarial learning model with respect to the training residual prediction value to obtain the gradient of the second residual prediction model.

[0085] Step S60: Aggregate the gradient of the first residual prediction model and the gradient of the second residual prediction model to obtain the target residual prediction model gradient, and send the target residual prediction model gradient to the second device for the second device to update the residual prediction model based on the target residual prediction model gradient.

[0086] In this embodiment, it should be noted that the gradient of the first residual prediction model is determined based on the residual prediction value and the true residual value fitted by the residual prediction model, and can optimize the residual prediction model more accurately. The gradient of the second residual prediction model can be aggregated with the gradient of the first output prediction model as an interference value to obtain the target residual prediction model gradient after interference processing. In this way, on the one hand, by decoupling the residual and the label, the difficulty for the second device to infer the service label information owned by the first device can be effectively increased. On the other hand, by using the adversarial learning model to interfere with the gradient information returned to the second device, the difficulty for the second device to infer the service label information owned by the first device can be further increased, so as to better protect the privacy of the first device.

[0087] As an example, step S60 includes: aggregating the gradient of the first residual prediction model and the gradient of the second residual prediction model to obtain the target residual prediction model gradient. Among them, the method of aggregating the gradient of the first residual prediction model and the gradient of the second residual prediction model can be addition operation, average value operation, weighted summation operation, weighted average operation, etc., and can be specifically determined according to actual needs. This embodiment does not limit this; then sending the target residual prediction model gradient to the second device for the second device to update the residual prediction model deployed by itself using the gradient descent method based on the received target residual prediction model gradient.

[0088] Further, the step of aggregating the gradient of the first residual prediction model and the gradient of the second residual prediction model to obtain the target residual prediction model gradient includes:

[0089] Step S61, obtaining the gradient weight corresponding to the second device, and determining the product of the gradient weight and the gradient of the second residual prediction model as the gradient adjustment value;

[0090] Step S62, determining the sum of the gradient of the first residual prediction model and the gradient adjustment value as the target residual prediction model gradient.

[0091] In this embodiment, it should be noted that since the adversarial learning model predicts business labels based on the training residual prediction values, the proportion of the gradient of the second residual prediction model output by the adversarial learning model in the gradient of the target residual prediction model is too large, which may instead increase the possibility for the second device to infer the business label information owned by the first device. Therefore, a balance point that can protect privacy and ensure model performance can be found by adjusting the gradient weight in advance, so as to determine the gradient weight corresponding to the second device, thereby achieving a trade-off between privacy protection and model performance. In an implementable manner, the gradient weight can be 0.02 - 0.1, such as 0.02, 0.05, 0.08, 0.1, etc.

[0092] As an example, steps S61 - S62 include: obtaining the gradient weight corresponding to the second device determined in advance, multiplying the gradient weight by the gradient of the second residual prediction model, determining the obtained product as the gradient adjustment value, adding the gradient of the first residual prediction model and the gradient adjustment value, and determining the obtained sum value as the gradient of the target residual prediction model.

[0093] In this way, one round of iterative update of the residual prediction model can be completed. By performing multiple rounds of iterative update on the residual prediction model until the preset federated training end condition is met. Among them, the preset federated training end condition can be that the residual prediction models in more than a preset number of the second devices participating in the federated learning converge, or reach the preset maximum number of iterations of the federated learning, or reach the preset maximum training time of the federated learning, etc. It can be specifically determined according to the actual situation, and this embodiment does not limit this.

[0094] Further, the step of determining the gradient of the first residual prediction model based on the training residual prediction value and the true residual value, and inputting the training residual prediction value into the adversarial learning model for business label prediction to obtain the second business label training prediction result includes:

[0095] Step B10, judging whether the current situation meets the federated learning training end condition according to the training residual prediction value and the true residual value;

[0096] Step B20, in the case of determining that the current situation does not meet the federated learning training end condition, determining the gradient of the first residual prediction model based on the training residual prediction value and the true residual value, and inputting the training residual prediction value into the adversarial learning model for business label prediction to obtain the second business label training prediction result;

[0097] After the step of aggregating the first residual prediction model gradient and the second residual prediction model gradient to obtain a target residual prediction model gradient and sending the target residual prediction model gradient to the second device for the second device to update the residual prediction model based on the target residual prediction model gradient, the method further includes:

[0098] Step B30: Return to execute the step of obtaining the first-party training sample service data and the first sample service label of the aligned sample.

[0099] In this embodiment, it should be noted that after receiving the training residual prediction value sent by the second device each time, it is possible to first determine whether the current condition for ending the federated training is satisfied. If the current condition for ending the federated training is not satisfied, it is necessary to further calculate the loss and gradient to iteratively optimize the residual prediction model. If the current condition for ending the federated training is satisfied, it means that the residual prediction model has been trained well.

[0100] The preset condition for ending the federated training may be that the residual prediction models in more than a preset number of the second devices participating in the federated learning converge, or reach the preset maximum number of iterations of the federated learning, or reach the preset maximum training time of the federated learning, etc. Specifically, it can be determined according to the actual situation, and this embodiment does not limit this.

[0101] As an example, the steps B10 - B30 include: judging whether the residual prediction models deployed on each second device converge according to the difference between the training residual prediction value corresponding to each second device and the true residual value corresponding to the training residual prediction value, and counting the number of converged residual prediction models. When it is detected that the number of converged residual prediction models exceeds a preset number threshold, it is determined that the current condition for the end of federated training is met. When it is detected that the number of converged residual prediction models does not exceed the preset number threshold, it can be determined that the current condition for the end of federated training is not met, or it can be comprehensively judged whether the current condition for the end of federated training is met in combination with the current number of iterations, the current training time, etc.; when it is determined that the current condition for the end of federated training is not met, the training residual prediction values are matched with the true residual values according to the aligned sample IDs, and according to the difference values between the mutually matched training residual prediction values and the true residual values, the loss function of the residual prediction loss of the residual prediction models on each second device is calculated, and then the first residual prediction model gradient of the loss function corresponding to each second device with respect to the training residual prediction value corresponding to it is calculated, and the first residual prediction model gradients are sent to the corresponding second devices respectively for each second device to update the residual prediction model deployed by itself based on the received first residual prediction model gradients by using the gradient descent method; and the training residual prediction values are input into the adversarial learning model for business label prediction to obtain the second business label training prediction result. Since the model training is not over, after sending the target residual prediction model gradient to the second device for the second device to update the residual prediction model based on the target residual prediction model gradient, the step of obtaining the first - party training sample service data and the first sample service label of the aligned sample can be returned to execute for the next round of iterative update until the condition for the end of federated training is met, and then the training of the adversarial learning model and the residual prediction model can be completed.

[0102] In an implementable manner, referring to Figure 2 , the vertical federated learning system includes a first device and N second devices. Each second device is communicatively connected to the first device. After sample alignment, each second device can perform residual prediction based on the second training sample service data of the aligned samples through the residual prediction model to obtain training residual prediction values R1, R2, R3, ……, R N , and send the training residual prediction values R1, R2, R3, ……, R N to the first device. After the first device receives the training residual prediction values R1, R2, R3, ……, R N , based on the training residual prediction values R1, R2, R3, ……, R Nand each of the true residual values, determine the first residual prediction model gradients G1’, G2’, G3’, ……, G N ’ corresponding to each of the second devices, and by using the training residual prediction values R1, R2, R3, ……, R N as inputs to the adversarial learning model, obtain the training prediction results of the second service labels. Based on the training prediction results of the second service labels and the first sample service labels, determine the second residual prediction model gradients G1”, G2”, G3”, ……, G N ” corresponding to each of the second devices. Aggregate the first residual prediction model gradients and the second residual prediction model gradients to obtain the target residual prediction model gradients G1, G2, G3, ……, G N and send the target residual prediction model gradients G1, G2, G3, ……, G N to their corresponding second devices respectively, so that each of the second devices can update the residual prediction models M1, M2, M3, ……, M N deployed by themselves based on the received target residual prediction model gradients.

[0103] In this embodiment, the vertical federated prediction optimization method is applied to a first device in a vertical federated learning system. A business label prediction model and an adversarial learning model are deployed on the first device. The vertical federated learning system further includes a second device on which a residual prediction model is deployed. First, sample alignment is performed with the second device to determine aligned samples, and the first-party training sample business data and the first sample business label of the aligned samples are obtained. The first business label training prediction result obtained by the trained business label prediction model for business label prediction based on the first-party training sample business data is also obtained. Based on the first sample business label and the first business label training prediction result, the true residual value is determined, realizing the determination of the true residual value generated by the trained business label prediction model for label prediction. Then, the training residual prediction value sent by the second device is received, where the training residual prediction value is obtained by the second device through the residual prediction model for residual prediction based on the second-party training sample business data of the aligned samples. In this way, the second-party training sample business data owned by the second device can be used for residual prediction, achieving the purpose of determining the residual prediction value. Then, based on the training residual prediction value and the true residual value, the first residual prediction model gradient is determined, and the training residual prediction value is input into the adversarial learning model for business label prediction to obtain the second business label training prediction result. The adversarial learning model is iteratively updated based on the first sample business label and the second business label training prediction result, and the second residual prediction model gradient is determined. The first residual prediction model gradient determined in this way can effectively improve the model performance of the residual prediction model deployed on the second device. However, in order to better protect the privacy of the label information of the first device, the adversarial learning model is used to perform label prediction based on the training residual prediction value, and then the second residual prediction model gradient is obtained. Then, by aggregating the first residual prediction model gradient and the second residual prediction model gradient, the target residual prediction model gradient is obtained, and the target residual prediction model gradient is sent to the second device for the second device to update the residual prediction model based on the target residual prediction model gradient, realizing the trade-off of the first residual prediction model gradient by the second residual prediction model gradient. In this way, the trade-off between label prediction accuracy and privacy protection can be achieved.In this way, during the entire training process, only the training residual prediction values and the gradients of the first residual prediction model are shared between the first device and the second device. For the second device, since the first device cannot know which sample features the second device uses to obtain the training residual prediction values, it is impossible to infer the second-party training sample service data owned by the second device. Therefore, privacy protection for the second device can be achieved. For the first device, since the residual is the difference between the first sample service label and the local model training prediction result, which reflects the model performance. Without knowing the first-party training sample service data, the first sample service label, and the service label prediction model, the second device cannot infer the label information or privacy data owned by the first device based only on the residual. Therefore, decoupling of the label information and the residual information can be achieved, thereby realizing privacy protection for the first device. Thus, it overcomes the technical defect that each participant in vertical federated learning collaborates with the label holder to train a vertical federated model, and then uses the vertical federated model for label prediction for the label holder. In this way, the participant can infer the label of the test sample of the label holder based on the locally trained model, thus bringing a serious privacy leakage risk. Moreover, compared with the service label prediction model trained only based on local data, vertical federated learning can use a variety of sample service data distributed on multiple second devices to more accurately predict the residual between the prediction result and the true result of the service label prediction model, thereby correcting the prediction result of the service label prediction model, that is, improving the accuracy of the finally output prediction result, and the requirements of both label prediction accuracy and privacy protection can be balanced through the adversarial learning model. Therefore, both the prediction accuracy can be improved and privacy leakage can be avoided. And since the privacy protection method of this application is in plain text throughout the process and does not involve encryption algorithms, the learning and prediction processes are very efficient. Therefore, it takes into account the efficiency, performance, and privacy of federated learning and has high practical and commercial value.

[0104] Embodiment 2

[0105] Furthermore, this application also provides a vertical federated prediction optimization method. In the second embodiment of this application, for the content that is the same as or similar to the above embodiment, reference can be made to the above introduction and will not be repeated hereinafter. On this basis, the vertical federated prediction optimization method is applied to the second device, and a residual prediction model is deployed on the second device; referring to Figure 3 , the vertical federated prediction optimization method includes the following steps:

[0106] Step C10, perform sample alignment with the first device to determine the aligned samples, and obtain the second-party training sample service data of the aligned samples;

[0107] The execution subject of the method in this embodiment can be a vertical federated prediction optimization device, or a vertical federated prediction optimization terminal device or server. In this embodiment, a vertical federated prediction optimization device is taken as an example. This vertical federated prediction optimization device can be integrated on terminal devices such as smartphones and computers with data processing functions.

[0108] In this embodiment, it should be noted that the vertical federated prediction optimization method provided in this embodiment is applied to any one of multiple second devices in a vertical federated learning system. A residual prediction model is deployed on the second device. The vertical federated learning system further includes a first device. For the second device, it only knows that a business label prediction model is deployed on the first device, but does not know that an adversarial learning model is also deployed on the first device. And for the second device, it will update the residual prediction model based on the target residual prediction model gradient returned by the first device, and cannot know that interference values are introduced into the target residual prediction model gradient.

[0109] As an example, the step C10 includes: first, initializing the residual prediction model, then aligning samples with the first device to determine the alignment sample IDs of at least one aligned sample, and searching for the second-party training sample business data corresponding to the alignment sample IDs from the sample business data owned by itself based on the alignment sample IDs.

[0110] Step C20, inputting the second-party training sample business data into the residual prediction model for residual prediction to obtain a training residual prediction value;

[0111] As an example, the step C20 includes: inputting the second-party training sample business data into the residual prediction model deployed by itself for residual learning, and calculating a training residual prediction value based on the forward propagation algorithm.

[0112] In an implementable manner, a boosting model can be initialized as the residual prediction model. The boosting model can be a three-layer fully connected neural network. The first layer is the input layer, and the dimension of the input layer is the same as the dimension of the data features. The third layer is the output layer, and the dimension of the output layer is the same as the dimension of the residuals. When performing residual prediction, first input each second-party training sample business data into the first layer, perform an inner product operation with the model parameters of the first layer to obtain the output result vector of the first layer; then input the output result vector of the first layer into the second layer, perform an inner product operation with the model parameters of the second layer to obtain the output result vector of the second layer; then input the output result vector of the second layer into the third layer, perform an inner product operation with the model parameters of the third layer to obtain the training residual prediction result. In this way, the training residual prediction result can be calculated based on the forward propagation algorithm by the residual prediction model.

[0113] Step C30: Send the training residual prediction value to the first device for the first device to determine the target residual prediction model gradient based on the training residual prediction value.

[0114] As an example, step C30 includes: sending the training residual prediction value to the first device for the first device to determine the target residual prediction model gradient based on the training residual prediction value, and the first device returns the target residual prediction model gradient to the second device. The method for the first device to determine the target residual prediction model gradient based on the training residual prediction value can refer to the specific content corresponding to steps S10 - S50 above, which will not be elaborated here. However, it should be noted that the second device cannot determine the specific generation method of the target residual prediction model gradient returned by the first device, thereby realizing the privacy protection of the first device.

[0115] Step C40: Receive the target residual prediction model gradient sent by the first device and update the residual prediction model based on the target residual prediction model gradient.

[0116] As an example, step C40 includes: receiving the target residual prediction model gradient sent by the first device and updating the locally deployed residual prediction model using the gradient descent method based on the received target residual prediction model gradient.

[0117] In this embodiment, for the second device, the second - party training sample service data also belongs to the user's privacy information. By performing residual prediction through the residual prediction model, only the residual information can be sent to the first device, realizing the decoupling of the residual information and the sample data. Without the first device knowing the second - party training sample service data, it is difficult for the second device to infer the privacy data owned by the second device only based on the residual information. Therefore, the privacy protection of the second device can be achieved.

[0118] Embodiment Three

[0119] Furthermore, the present application also provides a vertical federated prediction optimization method. In the third embodiment of the present application, the same or similar content as the above - mentioned embodiments can be referred to the above introduction and will not be elaborated later. On this basis, the vertical federated prediction optimization method is applied to the third device. Referring to Figure 4 , the vertical federated prediction optimization method includes the following steps:

[0120] Step D10: Obtain the first - party service data of the sample to be predicted and obtain the local service label prediction result obtained by the service label prediction model for business label prediction based on the first - party service data of the sample to be predicted.

[0121] In this embodiment, it should be noted that the vertical federated prediction optimization method is applied to the third device. The third device can be the first device. After completing the model training of the service label prediction model, the adversarial learning model, and the residual prediction model, the service label prediction can be directly performed locally. The third device can also be other electronic devices other than the first device. After using the first device to complete the model training of the service label prediction model, the adversarial learning model, and the residual prediction model, the trained service label prediction model is deployed on other electronic devices for service label prediction. It can be specifically determined according to the actual situation, and this embodiment does not limit this.

[0122] As an example, the step D10 includes: obtaining the first-party service data of the sample to be predicted of the sample to be predicted, and then inputting the first-party service data of the sample to be predicted into the trained service label prediction model to perform service label prediction, and obtaining the local service label prediction result.

[0123] In an implementable manner, the service label prediction model may include a feature extractor, a fully connected layer, and an activation layer. After obtaining the first-party service data of the sample to be predicted of the sample to be predicted, the first-party service data of the sample to be predicted can be first subjected to feature extraction based on the feature extractor to obtain the first-party sample features. Then, the first-party sample features can be concatenated into a first-party sample feature vector or a first-party sample feature matrix, and the first-party sample feature vector or the first-party sample feature matrix is fully connected through the fully connected layer. Then, the output of the fully connected layer is activated through the activation function preset by the activation layer to obtain the local service label prediction result.

[0124] Step D20: Align samples with at least one second device, so that for the target second device among the second devices that contains the second-party service data of the sample to be predicted corresponding to the sample to be predicted, the residual prediction is performed based on the second-party service data of the sample to be predicted through the residual prediction model deployed by itself, and the residual prediction result is obtained, where the residual prediction model is trained by using the vertical federated prediction optimization method as described above;

[0125] As an example, step D20 includes: the third device may send the ID of the sample to be predicted of the to-be-predicted sample to each of the second devices, so that each of the second devices can search for the second-party to-be-predicted sample service data corresponding to the ID of the to-be-predicted sample from the sample service data it owns, thereby achieving sample alignment with each of the second devices. The second device that can perform sample alignment with the third device can be determined as the target second device. When the target second device finds the second-party to-be-predicted sample service data, it inputs each of the second-party to-be-predicted sample service data into the trained residual prediction model deployed on its own side for residual prediction, obtains the residual prediction result, and then sends each of the training residual prediction results to the third device. When the second device cannot find the second-party to-be-predicted sample service data corresponding to the ID of the to-be-predicted sample, it may not need to perform the residual prediction task, or send a null value to the third device, or send a prompt message of no aligned sample to the third device. Among them, the training method of the residual prediction model can refer to the specific content corresponding to the above steps S10 - S50, which will not be elaborated here.

[0126] Step D30: Receive the residual prediction results sent by each of the target second devices, and aggregate the local service label prediction result and the residual prediction results to obtain a federated service label prediction result.

[0127] As an example, step D30 includes: receiving the residual prediction results sent by each of the target second devices, and aggregating the local service label prediction result and the residual prediction results to obtain a federated service label prediction result, where the way of aggregating the local service label prediction result and the residual prediction results can be addition operation, average operation, weighted summation operation, weighted average operation, etc.

[0128] In this embodiment, the model performance of the service label prediction model trained based on the local sample service data is limited, so the residual between the obtained local service label prediction result and the true value will be relatively large. By using the residual prediction models distributed on different second devices, the prediction result of the service label prediction model can be corrected by using the more diverse and rich sample service data owned by different second devices while ensuring the privacy of each party, effectively reducing the residual between the local service label prediction result and the true value, thereby obtaining a federated service label prediction result closer to the true value and improving the accuracy of service label prediction.

[0129] Embodiment 4

[0130] Furthermore, the embodiment of the present application also provides a vertical federated prediction optimization device, refer to Figure 5, the longitudinal federated prediction optimization device is applied to the first device in the longitudinal federated learning system. The business label prediction model and the adversarial learning model are deployed on the first device; the longitudinal federated learning system further includes a second device, and the residual prediction model is deployed on the second device; the longitudinal federated prediction optimization device includes:

[0131] The first alignment module 10 is used to align samples with the second device to determine aligned samples;

[0132] The first acquisition module 20 is used to acquire the first-party training sample business data and the first sample business label of the aligned samples, and acquire the first business label training prediction result obtained by the trained business label prediction model for business label prediction based on the first-party training sample business data. Based on the first sample business label and the first business label training prediction result, determine the true value of the residual;

[0133] The receiving module 30 is used to receive the training residual prediction value sent by the second device, where the training residual prediction value is obtained by the second device through the residual prediction model for residual prediction based on the second-party training sample business data of the aligned samples;

[0134] The business label prediction module 40 is used to determine the first residual prediction model gradient based on the training residual prediction value and the true value of the residual, and input the training residual prediction value into the adversarial learning model for business label prediction to obtain the second business label training prediction result;

[0135] The first update module 50 is used to iteratively update the adversarial learning model based on the first sample business label and the second business label training prediction result, and determine the second residual prediction model gradient;

[0136] The first aggregation module 60 is used to aggregate the first residual prediction model gradient and the second residual prediction model gradient to obtain the target residual prediction model gradient, and send the target residual prediction model gradient to the second device for the second device to update the residual prediction model based on the target residual prediction model gradient.

[0137] Furthermore, the first update module 50 is further used to:

[0138] Determine the adversarial learning model loss according to the difference between the first sample business label and the second business label training prediction result;

[0139] Calculate the adversarial learning model gradient of the adversarial learning model loss with respect to the training prediction result of the second service label, iteratively update the adversarial learning model based on the adversarial learning model gradient, and calculate the second residual prediction model gradient of the adversarial learning model loss with respect to the training residual prediction value.

[0140] Further, the service label prediction module 40 is further configured to:

[0141] Judge whether the current condition meets the end condition of federated learning training according to the training residual prediction value and the true residual value;

[0142] In the case where it is determined that the current condition does not meet the end condition of federated learning training, based on the training residual prediction value and the true residual value, determine the first residual prediction model gradient, and input the training residual prediction value into the adversarial learning model for service label prediction to obtain the second service label training prediction result;

[0143] After the operation of aggregating the first residual prediction model gradient and the second residual prediction model gradient to obtain the target residual prediction model gradient, and sending the target residual prediction model gradient to the second device for the second device to update the residual prediction model based on the target residual prediction model gradient, the vertical federated prediction optimization device further includes a return module, and the return module is used for:

[0144] Return to execute the step of obtaining the first-party training sample service data and the first sample service label of the aligned sample.

[0145] Further, before the operation of obtaining the first-party training sample service data and the first sample service label of the aligned sample, and obtaining the first service label training prediction result obtained by the trained service label prediction model for service label prediction based on the first-party training sample service data, and determining the true residual value based on the first sample service label and the first service label training prediction result, the vertical federated prediction optimization device further includes a local training module, and the local training module is used for:

[0146] Obtain the service label prediction model training sample service data and the second sample service label, and iteratively optimize the service label prediction model based on the service label prediction model training sample service data and the second sample service label to obtain the trained service label prediction model.

[0147] Further, the first aggregation module 60 is further configured to:

[0148] Obtain the gradient weight corresponding to the second device, and determine the gradient adjustment value as the product of the gradient weight and the second residual prediction model gradient;

[0149] Determine the sum of the first residual prediction model gradient and the gradient adjustment value as the target residual prediction model gradient.

[0150] The vertical federated prediction optimization device provided by the present invention adopts the vertical federated prediction optimization method in the above embodiment, and solves the technical problem of high privacy leakage risk in vertical federated learning in the related art. Compared with the related art, the advantages of the vertical federated prediction optimization device provided by the embodiments of the present invention are the same as those of the vertical federated prediction optimization method provided by the above embodiment, and other technical features in the vertical federated prediction optimization device are the same as the features disclosed in the method of the above embodiment, and will not be described in detail here.

[0151] Embodiment Five

[0152] Further, an embodiment of the present application further provides a vertical federated prediction optimization device, which is applied to a second device, and a residual prediction model is deployed on the second device; the vertical federated prediction optimization device includes:

[0153] A second alignment module, configured to perform sample alignment with a first device, determine aligned samples, and obtain second-party training sample service data of the aligned samples;

[0154] A residual prediction module, configured to input the second-party training sample service data into the residual prediction model for residual prediction to obtain a training residual prediction value;

[0155] A sending module, configured to send the training residual prediction value to the first device for the first device to determine a target residual prediction model gradient based on the training residual prediction value;

[0156] A second update module, configured to receive the target residual prediction model gradient sent by the first device, and update the residual prediction model based on the target residual prediction model gradient.

[0157] Embodiment Six

[0158] Further, an embodiment of the present application further provides a vertical federated prediction optimization device, which is applied to a third device, and the vertical federated prediction optimization device includes:

[0159] A second acquisition module, configured to acquire first-party to-be-predicted sample service data of a to-be-predicted sample, and acquire a local service label prediction result obtained by a service label prediction model performing service label prediction based on the first-party to-be-predicted sample service data;

[0160] A third alignment module, configured to perform sample alignment with at least one second device, so that for a target second device among the second devices that contains second-party to-be-predicted sample service data corresponding to the to-be-predicted sample, a residual prediction is performed based on the second-party to-be-predicted sample service data through a residual prediction model deployed by itself, and a residual prediction result is obtained, where the residual prediction model is trained by using the vertical federated prediction optimization method as described above;

[0161] A second aggregation module, configured to receive the residual prediction results sent by the target second devices, and aggregate the local service label prediction result and the residual prediction results to obtain a federated service label prediction result.

[0162] Embodiment 7

[0163] Furthermore, an embodiment of the present invention provides an electronic device, which includes: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and when the instructions are executed by the at least one processor, the at least one processor is enabled to execute the vertical federated prediction optimization method in the above embodiment.

[0164] Next, refer to Figure 6 , which shows a schematic structural diagram of an electronic device suitable for implementing the embodiments of the present disclosure. The electronic device in the embodiments of the present disclosure may include, but is not limited to, mobile terminals such as Bluetooth headsets, mobile phones, laptop computers, digital broadcast receivers, PDAs (Personal Digital Assistants), PADs (Tablet Computers), PMPs (Portable Multimedia Players), vehicle terminals (such as vehicle navigation terminals), etc., and fixed terminals such as digital TVs, desktop computers, etc. Figure 6 The electronic device shown is only an example and should not impose any limitation on the functions and usage scope of the embodiments of the present disclosure.

[0165] As Figure 6 shown, the electronic device may include a processing device (such as a central processing unit, a graphics processing unit, etc.), which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) or a program loaded from a storage device into a random access memory (RAM). In the RAM, various programs and arrays required for the operation of the electronic device are also stored. The processing device, the ROM, and the RAM are connected to each other through a bus. An input / output (I / O) interface is also connected to the bus.

[0166] Generally, the following systems can be connected to the I / O interface: input devices including, for example, a touch screen, a touchpad, a keyboard, a mouse, an image sensor, a microphone, an accelerometer, a gyroscope, etc.; output devices including, for example, a liquid crystal display (LCD), a speaker, a vibrator, etc.; storage devices including, for example, a magnetic tape, a hard disk, etc.; and a communication device. The communication device can allow the electronic device to communicate with other devices wirelessly or wiredly to exchange data. Although the figure shows an electronic device with various systems, it should be understood that it is not required to implement or have all the shown systems. Instead, more or fewer systems can be implemented or had.

[0167] In particular, according to an embodiment of the present disclosure, the process described above with reference to the flowchart can be implemented as a computer software program. For example, an embodiment of the present disclosure includes a computer program product that includes a computer program carried on a computer-readable medium, and the computer program includes program codes for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from a network through the communication device, or installed from a storage device, or installed from a ROM. When the computer program is executed by a processing device, the above-mentioned functions defined in the method of the embodiment of the present disclosure are executed.

[0168] The electronic device provided by the present invention adopts the vertical federated prediction optimization method in the above embodiment, and solves the technical problem of relatively high privacy leakage risk in vertical federated learning in the related art. Compared with the related art, the advantages of the electronic device provided by the embodiment of the present invention are the same as those of the vertical federated prediction optimization method provided by the above embodiment, and other technical features in the electronic device are the same as those disclosed in the above embodiment method, and will not be elaborated here.

[0169] It should be understood that each part of the present disclosure can be implemented by hardware, software, firmware, or a combination thereof. In the description of the above embodiments, specific features, structures, materials, or characteristics can be combined in a suitable manner in any one or more embodiments or examples.

[0170] The above is only the specific implementation manner of the present invention, but the protection scope of the present invention is not limited thereto. Any person skilled in the art can easily think of changes or substitutions within the technical scope disclosed by the present invention, and all should be covered by the protection scope of the present invention. Therefore, the protection scope of the present invention should be subject to the protection scope of the claims.

[0171] Embodiment VIII

[0172] Furthermore, this embodiment provides a computer-readable storage medium having computer-readable program instructions stored thereon for executing the vertical federated prediction optimization method in the above embodiment.

[0173] The computer-readable storage medium provided by the embodiment of the present invention may be, for example, a USB flash drive, but is not limited to an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination of the above. More specific examples of the computer-readable storage medium may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In this embodiment, the computer-readable storage medium may be any tangible medium that contains or stores a program, and this program can be used by or in combination with an instruction execution system, apparatus, or device. The program code contained on the computer-readable storage medium can be transmitted by any appropriate medium, including but not limited to: wires, optical cables, RF (radio frequency), etc., or any suitable combination of the above.

[0174] The above computer-readable storage medium may be included in an electronic device; or it may exist separately without being assembled into the electronic device.

[0175] The above computer-readable storage medium carries one or more programs. When the above one or more programs are executed by an electronic device, the electronic device is enabled to: obtain the first-party training sample service data and the first sample service label of the alignment sample, and obtain the first service label training prediction result obtained by the trained service label prediction model for service label prediction based on the first-party training sample service data. Based on the first sample service label and the first service label training prediction result, determine the residual true value; receive the training residual prediction value sent by the second device, where the training residual prediction value is obtained by the second device through the residual prediction model for residual prediction based on the second-party training sample service data of the alignment sample; based on the training residual prediction value and the residual true value, determine the first residual prediction model gradient, and input the training residual prediction value into the adversarial learning model for service label prediction to obtain the second service label training prediction result; based on the first sample service label and the second service label training prediction result, iteratively update the adversarial learning model, and determine the second residual prediction model gradient; aggregate the first residual prediction model gradient and the second residual prediction model gradient to obtain the target residual prediction model gradient, and send the target residual prediction model gradient to the second device for the second device to update the residual prediction model based on the target residual prediction model gradient.

[0176] Alternatively, the above computer-readable storage medium stores one or more programs, which, when executed by an electronic device, cause the electronic device to: perform sample alignment with a first device, determine aligned samples, and obtain second-party training sample service data of the aligned samples; input the second-party training sample service data into the residual prediction model to perform residual prediction, obtaining a training residual prediction value; send the training residual prediction value to the first device for the first device to determine a target residual prediction model gradient based on the training residual prediction value; and receive the target residual prediction model gradient sent by the first device and update the residual prediction model based on the target residual prediction model gradient.

[0177] Alternatively, the above computer-readable storage medium stores one or more programs, which, when executed by an electronic device, cause the electronic device to: obtain first-party service data of a sample to be predicted, and obtain a local service label prediction result obtained by a service label prediction model performing service label prediction based on the first-party service data of the sample to be predicted; perform sample alignment with at least one second device for a target second device among the second devices that contains second-party service data of the sample to be predicted corresponding to the sample to be predicted to perform residual prediction on the second-party service data of the sample to be predicted through a residual prediction model deployed by itself, obtaining a residual prediction result, where the residual prediction model is trained by using the vertical federated prediction optimization method as described above; receive the residual prediction results sent by the target second devices, and aggregate the local service label prediction result and the residual prediction results to obtain a federated service label prediction result.

[0178] Computer program code for performing the operations of the present disclosure may be written in one or more programming languages or combinations thereof. The programming languages include object-oriented programming languages such as Java, Smalltalk, C++, and also include conventional procedural programming languages such as the "C" language or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, executed as a stand-alone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., by using an Internet service provider to connect through the Internet).

[0179] The flowcharts and block diagrams in the accompanying drawings illustrate the possible architectures, functions, and operations of systems, methods, and computer program products according to various embodiments of the present invention. In this regard, each block in the flowchart or block diagram may represent a module, a segment of a program, or a part of code that contains one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions marked in the blocks may occur in a different order than that marked in the accompanying drawings. For example, two consecutive blocks shown may actually be executed substantially in parallel, and they may sometimes be executed in the reverse order, depending on the functions involved. It should also be noted that each block in the block diagram and / or flowchart, as well as combinations of blocks in the block diagram and / or flowchart, can be implemented by a dedicated hardware-based system that performs the specified functions or operations, or can be implemented by a combination of dedicated hardware and computer instructions.

[0180] The modules involved in the embodiments described in this disclosure can be implemented in software or in hardware. In this regard, the name of the module does not, in some cases, constitute a limitation on the unit itself.

[0181] The computer-readable storage medium provided by the present invention stores computer-readable program instructions for executing the above-mentioned vertical federated prediction optimization method, which solves the technical problem of relatively high privacy leakage risk in vertical federated learning in the related art. Compared with the related art, the advantages of the computer-readable storage medium provided by the embodiments of the present invention are the same as those of the vertical federated prediction optimization method provided by the above embodiments, and will not be elaborated herein.

[0182] Embodiment Nine

[0183] Furthermore, the present application also provides a computer program product, including a computer program, which when executed by a processor, implements the steps of the vertical federated prediction optimization method as described above.

[0184] The computer program product provided by the present application solves the technical problem of relatively high privacy leakage risk in vertical federated learning in the related art. Compared with the related art, the advantages of the computer program product provided by the embodiments of the present invention are the same as those of the vertical federated prediction optimization method provided by the above embodiments, and will not be elaborated herein.

[0185] The above are only the preferred embodiments of the present application, and do not limit the patent scope of the present application. Any equivalent structural or equivalent process transformation made using the specifications and drawings of the present application, or directly or indirectly applied in other related technical fields, shall be equally included in the patent scope of the present application.

Claims

1. A vertical federated prediction optimization method, characterized in that, The longitudinal federated prediction optimization method is applied to a first device in a longitudinal federated learning system. An adversarial learning model and a trained business label prediction model are deployed on the first device. The longitudinal federated learning system further includes a second device on which a residual prediction model is deployed. The longitudinal federated prediction optimization method includes the following steps: Perform sample alignment with the second device to determine aligned samples; Obtain the first-party training sample business data and the first sample business label of the aligned samples, and obtain the first business label training prediction result obtained by the trained business label prediction model for business label prediction based on the first-party training sample business data. Based on the first sample business label and the first business label training prediction result, determine the residual true value; Receive the training residual prediction value sent by the second device, where the training residual prediction value is obtained by the second device through the residual prediction model for residual prediction based on the second-party training sample business data of the aligned samples; Based on the training residual prediction value and the residual true value, determine the first residual prediction model gradient, and input the training residual prediction value into the adversarial learning model for business label prediction to obtain the second business label training prediction result; Iteratively update the adversarial learning model based on the first sample business label and the second business label training prediction result, and determine the second residual prediction model gradient; Aggregate the first residual prediction model gradient and the second residual prediction model gradient to obtain the target residual prediction model gradient, and send the target residual prediction model gradient to the second device for the second device to update the residual prediction model based on the target residual prediction model gradient.

2. The vertical federated prediction optimization method according to claim 1, characterized in that, The step of iteratively updating the adversarial learning model based on the first sample business label and the second business label training prediction result and determining the second residual prediction model gradient includes: Determine the adversarial learning model loss according to the difference between the first sample business label and the second business label training prediction result; Calculate the adversarial learning model gradient of the adversarial learning model loss with respect to the second business label training prediction result, iteratively update the adversarial learning model based on the adversarial learning model gradient, and calculate the second residual prediction model gradient of the adversarial learning model loss with respect to the training residual prediction value.

3. The vertical federated prediction optimization method according to claim 1, characterized in that, The step of determining the first residual prediction model gradient based on the training residual prediction value and the residual true value, inputting the training residual prediction value into the adversarial learning model for business label prediction, and obtaining the second business label training prediction result includes: Judge whether the current condition meets the end condition of federated learning training according to the training residual prediction value and the residual true value; In the case of determining that the current condition does not meet the end condition of federated learning training, determine the first residual prediction model gradient based on the training residual prediction value and the residual true value, and input the training residual prediction value into the adversarial learning model for business label prediction to obtain the second business label training prediction result; After the step of aggregating the gradients of the first residual prediction model and the gradients of the second residual prediction model to obtain the target residual prediction model gradients, and sending the target residual prediction model gradients to the second device for the second device to update the residual prediction model based on the target residual prediction model gradients, the method further includes: Return to execute the step of obtaining the first-party training sample service data and the first sample service label of the aligned sample.

4. The vertical federated prediction optimization method according to claim 1, characterized in that, Before the step of obtaining the first-party training sample service data and the first sample service label of the aligned sample, and obtaining the first service label training prediction result obtained by the trained service label prediction model for predicting the service label based on the first-party training sample service data, and determining the true value of the residual based on the first sample service label and the first service label training prediction result, the method further includes: Obtain the service label prediction model training sample service data and the second sample service label, and iteratively optimize the service label prediction model based on the service label prediction model training sample service data and the second sample service label to obtain the trained service label prediction model.

5. The vertical federated prediction optimization method according to claim 1, characterized in that, The step of aggregating the gradients of the first residual prediction model and the gradients of the second residual prediction model to obtain the target residual prediction model gradients includes: Obtain the gradient weight corresponding to the second device, and determine the product of the gradient weight and the gradients of the second residual prediction model as the gradient adjustment value; Determine the sum of the gradients of the first residual prediction model and the gradient adjustment value as the target residual prediction model gradients.

6. A vertical federated prediction optimization method, characterized in that, The vertical federated prediction optimization method is applied to a second device, and a residual prediction model is deployed on the second device; the vertical federated prediction optimization method includes the following steps: Perform sample alignment with a first device to determine the aligned sample, and obtain the second-party training sample service data of the aligned sample; Input the second-party training sample service data into the residual prediction model to perform residual prediction to obtain the training residual prediction value; Send the training residual prediction value to the first device for the first device to determine the target residual prediction model gradients based on the training residual prediction value; Receive the target residual prediction model gradients sent by the first device, and update the residual prediction model based on the target residual prediction model gradients.

7. A vertical federated prediction optimization method, characterized in that, The vertical federated prediction optimization method is applied to a third device, and includes the following steps: Obtain the first-party to-be-predicted sample service data of the to-be-predicted sample, and obtain the local service label prediction result obtained by the service label prediction model for predicting the service label based on the first-party to-be-predicted sample service data; Perform sample alignment with at least one second device, so that for each target second device among the second devices that includes the second-party to-be-predicted sample service data corresponding to the to-be-predicted sample, perform residual prediction on the second-party to-be-predicted sample service data through the residual prediction model deployed on its own side to obtain the residual prediction result, where the residual prediction model is trained by using the vertical federated prediction optimization method according to any one of claims 1-6; Receive the residual prediction results sent by each of the target second devices, aggregate the local service label prediction results and the residual prediction results to obtain a federated service label prediction result.

8. A vertical federated prediction optimization device, characterized in that, The vertical federated prediction optimization device is applied to a first device in a vertical federated learning system. A service label prediction model and an adversarial learning model are deployed on the first device. The vertical federated learning system further includes a second device on which a residual prediction model is deployed. The vertical federated prediction optimization device includes: A first alignment module for aligning samples with the second device to determine aligned samples. A first acquisition module for acquiring the first-party training sample service data and the first sample service label of the aligned samples, and acquiring the first service label training prediction result obtained by the trained service label prediction model for service label prediction based on the first-party training sample service data. Based on the first sample service label and the first service label training prediction result, determine the true value of the residual. A receiving module for receiving the training residual prediction value sent by the second device, where the training residual prediction value is obtained by the second device through the residual prediction model for residual prediction based on the second-party training sample service data of the aligned samples. A service label prediction module for determining the first residual prediction model gradient based on the training residual prediction value and the true value of the residual, and inputting the training residual prediction value into the adversarial learning model for service label prediction to obtain a second service label training prediction result. A first update module for iteratively updating the adversarial learning model based on the first sample service label and the second service label training prediction result, and determining the second residual prediction model gradient. A first aggregation module for aggregating the first residual prediction model gradient and the second residual prediction model gradient to obtain a target residual prediction model gradient, and sending the target residual prediction model gradient to the second device for the second device to update the residual prediction model based on the target residual prediction model gradient.

9. An electronic device, characterized in that, The electronic device includes: At least one processor; and, A memory communicatively connected to at least one of the processors; wherein, The memory stores instructions executable by at least one of the processors. The instructions are executed by at least one of the processors so that at least one of the processors can execute the steps of the vertical federated prediction optimization method according to any one of claims 1 to 7.

10. A storage medium, characterized in that, The storage medium is a computer-readable storage medium. A program for implementing the vertical federated prediction optimization method is stored on the computer-readable storage medium. The program for implementing the vertical federated prediction optimization method is executed by a processor to implement the steps of the vertical federated prediction optimization method according to any one of claims 1 to 7.

11. A product, the product being a computer program product, including a computer program, characterized in that, When the computer program is executed by a processor, it implements the steps of the vertical federated prediction optimization method according to any one of claims 1 to 7.

Images