Longitudinal federal predictive optimization method, device, medium and computer program product

Abstract

The application discloses a longitudinal federal prediction optimization method, device, medium and computer program product, comprising: extracting a to-be-predicted sample, and generating a first-party model prediction result corresponding to the to-be-predicted sample and an intermediate sample feature corresponding to the to-be-predicted sample based on a locally iteratively trained target prediction model; sending the intermediate sample feature to a second device, so that the second device jointly performs model prediction on the intermediate sample feature and an ID matching sample corresponding to the to-be-predicted sample based on a longitudinal federal residual error promotion model, and obtains a second-party model prediction result; obtaining a first-party model weight, and receiving the second-party model prediction result and a second-party model weight sent by the second device; and based on the first-party model weight and the second-party model weight, weighting and aggregating the first-party model prediction result and the second-party model prediction result to obtain a target federal prediction result. The application solves the technical problem of low overall sample prediction accuracy of longitudinal federal prediction.

Description

Technical Field

[0001] This application relates to the field of artificial intelligence technology in financial technology (Fintech), and more particularly to a vertical federated prediction optimization method, apparatus, medium, and computer program product. Background Technology

[0002] With the continuous development of fintech, especially internet fintech, more and more technologies (such as distributed systems and artificial intelligence) are being applied in the financial field. However, the financial industry is also placing higher demands on technology, such as on the distribution of tasks to be completed.

[0003] With the continuous development of computer software and artificial intelligence computing, the application of artificial intelligence technology is becoming increasingly widespread. Currently, existing vertical federated prediction models require vertical federated learning modeling based on aligned samples among the participants. After modeling, the vertical federated prediction model is distributed across the participants, with each participant holding only a portion of the model. Therefore, in vertical federated prediction scenarios, for aligned samples, the predictor typically performs vertical federated prediction using aligned samples distributed across other participants to achieve accurate sample prediction. However, for unaligned samples, the predictor needs to use a separate local model for prediction. Thus, while the predictor can achieve higher accuracy in federated prediction for aligned samples, it cannot perform sample prediction based on the complete model for unaligned samples, resulting in lower overall sample prediction accuracy. Therefore, existing vertical federated prediction methods suffer from the problem of low overall sample prediction accuracy. Summary of the Invention

[0004] The main purpose of this application is to provide a vertical federated forecasting optimization method, device, medium, and computer program product, which aims to solve the technical problem of low overall sample forecasting accuracy in the prior art.

[0005] To achieve the above objectives, this application provides a vertical federated prediction optimization method, which is applied to a first device, and includes:

[0006] Extract the sample to be predicted, and obtain the intermediate sample features generated by the feature extractor of the target prediction model for the sample to be predicted, and the first-party model prediction result generated by the target prediction model for the sample to be predicted, wherein the target prediction model is obtained by local iterative training of the first device;

[0007] The intermediate sample features are sent to the second device, which then performs model prediction on the intermediate sample features and the ID matching samples corresponding to the sample to be predicted based on the longitudinal federated residual boosting model, and obtains the second-party model prediction result.

[0008] Obtain the first-party model weights corresponding to the target prediction model, and receive the second-party model prediction results and the second-party model weights corresponding to the longitudinal federated residual improvement model sent by the second device;

[0009] Based on the weights of the first-party model and the weights of the second-party model, the prediction results of the first-party model and the prediction results of the second-party model are weighted and aggregated to obtain the target federated prediction result.

[0010] To achieve the above objectives, this application provides a vertical federated prediction optimization method, which is applied to a second device, and includes:

[0011] Receive intermediate sample features sent by the first device, and search for samples whose IDs match the intermediate sample features;

[0012] Based on the longitudinal federated residual enhancement model, the model prediction is jointly performed on the features of the ID matching samples and the intermediate samples to obtain the second-party model prediction results.

[0013] Obtain the second-party model weights corresponding to the longitudinal federated residual enhancement model, and send the second-party model prediction results and the second-party model weights to the first device, so that the first device can generate a target federated prediction result based on the first-party model prediction results generated by the target prediction model for the sample to be predicted corresponding to the ID matching sample, the first-party model weights corresponding to the target prediction model, the second-party model prediction results and the second-party model weights, wherein the target prediction model is obtained by the first device through local iterative training.

[0014] To achieve the above objectives, this application provides a vertical federated learning modeling optimization method, which is applied to a first device, and includes:

[0015] Obtain the initial weights of the first-party model, and extract the training samples and the corresponding training sample labels;

[0016] Obtain intermediate training sample features generated by the feature extractor of the target prediction model to be trained, based on the training samples.

[0017] Based on the training sample labels, the training model prediction results corresponding to the intermediate training sample features, and the first-party initial model weights, the target prediction model is obtained by iteratively optimizing the target prediction model to be trained by calculating the first-party model prediction loss corresponding to the target prediction model to be trained.

[0018] The training sample labels, intermediate training sample features, and the first-party model prediction loss are sent to the second device so that the second device can calculate the second-party model prediction loss and optimize the training residual enhancement model based on the residual loss calculated by the second-party model prediction loss and the first-party model prediction loss to obtain the vertical federated residual enhancement model.

[0019] To achieve the above objectives, this application provides a vertical federated learning modeling optimization method, which is applied to a second device. The vertical federated learning modeling optimization method includes:

[0020] Obtain the initial weights of the second-party model, and receive the intermediate training sample features, training sample labels, and first-party model prediction loss sent by the first device;

[0021] Obtain training sample ID matching samples, and improve the model based on the residual to be trained. Perform model prediction on the training sample ID matching samples and the features of the intermediate training samples to obtain the prediction result of the second-party training model.

[0022] Based on the training sample labels, the second-party initial model weights, and the prediction results of the second-party trained model, the prediction loss of the second-party model is calculated.

[0023] Based on the residual loss generated by the prediction loss of the first-party model and the prediction loss of the second-party model, the residual boosting model to be trained is iteratively optimized to obtain the longitudinal federated residual boosting model.

[0024] This application also provides a vertical federated prediction optimization apparatus, wherein the vertical federated prediction optimization apparatus is a virtual apparatus and is applied to a first device, and the vertical federated prediction optimization apparatus includes:

[0025] The model prediction module is used to extract the sample to be predicted, and obtain the intermediate sample features generated by the feature extractor of the target prediction model for the sample to be predicted, and the first-party model prediction result generated by the target prediction model for the sample to be predicted, wherein the target prediction model is obtained by the first device through local iterative training.

[0026] The sending module is used to send the intermediate sample features to the second device, so that the second device can perform model prediction on the intermediate sample features and the ID matching samples corresponding to the sample to be predicted based on the longitudinal federated residual boosting model, and obtain the second-party model prediction result;

[0027] The receiving module is used to obtain the first-party model weights corresponding to the target prediction model, and to receive the second-party model prediction results and the second-party model weights corresponding to the longitudinal federated residual improvement model sent by the second device.

[0028] The weighted aggregation module is used to perform weighted aggregation on the prediction results of the first model and the prediction results of the second model based on the weights of the first model and the second model to obtain the target federated prediction result.

[0029] This application also provides a vertical federated prediction optimization apparatus, which is a virtual apparatus and is applied to a second device. The vertical federated prediction optimization apparatus includes:

[0030] The receiving and searching module is used to receive intermediate sample features sent by the first device and search for samples whose IDs match the intermediate sample features.

[0031] The model prediction module is used to perform model prediction on the features of the ID matching samples and the intermediate samples based on the longitudinal federated residual enhancement model, and obtain the second-party model prediction results.

[0032] The sending module is used to obtain the second-party model weights corresponding to the longitudinal federated residual enhancement model, and send the second-party model prediction results and the second-party model weights to the first device, so that the first device can generate a target federated prediction result based on the first-party model prediction results generated by the target prediction model for the sample to be predicted corresponding to the ID matching sample, the first-party model weights corresponding to the target prediction model, the second-party model prediction results and the second-party model weights, wherein the target prediction model is obtained by the first device through local iterative training.

[0033] This application also provides a vertical federated learning modeling optimization device, which is a virtual device and is applied to a first device. The vertical federated learning modeling optimization device includes:

[0034] The first acquisition module is used to acquire the initial model weights of the first party and extract the training samples and the training sample labels corresponding to the training samples;

[0035] The second acquisition module is used to acquire the intermediate training sample features generated by the feature extractor of the target prediction model to be trained for the training samples.

[0036] The iterative optimization module is used to iteratively optimize the target prediction model to be trained by calculating the first-party model prediction loss corresponding to the target prediction model to be trained, based on the training sample labels, the training model prediction results corresponding to the intermediate training sample features, and the first-party initial model weights, to obtain the target prediction model.

[0037] The sending module is used to send the training sample labels, the intermediate training sample features, and the first-party model prediction loss to the second device, so that the second device can calculate the second-party model prediction loss and optimize the training residual enhancement model based on the residual loss calculated by the second-party model prediction loss and the first-party model prediction loss to obtain the vertical federated residual enhancement model.

[0038] This application also provides a vertical federated learning modeling optimization device, which is a virtual device and is applied to a second device. The vertical federated learning modeling optimization device includes:

[0039] The receiving module is used to obtain the initial model weights of the second party and to receive the intermediate training sample features, training sample labels and the prediction loss of the first party model sent by the first device.

[0040] The model prediction module is used to obtain training sample ID matching samples, and improve the model based on the residual to be trained, and jointly perform model prediction on the training sample ID matching samples and the features of the intermediate training samples to obtain the prediction result of the second-party training model.

[0041] The calculation module is used to calculate the prediction loss of the second-party model based on the training sample labels, the weights of the second-party initial model, and the prediction results of the second-party trained model.

[0042] The iterative optimization module is used to iteratively optimize the training residual boosting model based on the residual loss generated by the prediction loss of the first-party model and the prediction loss of the second-party model, so as to obtain the longitudinal federated residual boosting model.

[0043] This application also provides a vertical federated prediction optimization device, which is a physical device. The vertical federated prediction optimization device includes: a memory, a processor, and a program of the vertical federated prediction optimization method stored in the memory and executable on the processor. When the program of the vertical federated prediction optimization method is executed by the processor, it can implement the steps of the vertical federated prediction optimization method as described above.

[0044] This application also provides a vertical federated learning modeling optimization device, which is a physical device. The vertical federated learning modeling optimization device includes: a memory, a processor, and a program of the vertical federated learning modeling optimization method stored in the memory and executable on the processor. When the program of the vertical federated learning modeling optimization method is executed by the processor, it can implement the steps of the vertical federated learning modeling optimization method as described above.

[0045] This application also provides a medium, which is a readable storage medium, on which a program implementing the vertical federated prediction optimization method is stored. When the program of the vertical federated prediction optimization method is executed by a processor, it implements the steps of the vertical federated prediction optimization method as described above.

[0046] This application also provides a medium, which is a readable storage medium, on which a program implementing a longitudinal federated learning modeling optimization method is stored. When the program is executed by a processor, it implements the steps of the longitudinal federated learning modeling optimization method as described above.

[0047] This application also provides a computer program product, including a computer program that, when executed by a processor, implements the steps of the vertical federated prediction optimization method as described above.

[0048] This application also provides a computer program product, including a computer program that, when executed by a processor, implements the steps of the longitudinal federated learning modeling optimization method as described above.

[0049] This application provides a vertical federated prediction optimization method, device, medium, and computer program product. Compared to the existing technology that uses aligned samples from other participants for vertical federated prediction in a vertical federated prediction scenario to accurately predict samples, and unaligned samples for local prediction based on a locally held portion of the vertical federated prediction model, this application first extracts the sample to be predicted and obtains intermediate sample features generated by the feature extractor of the target prediction model for the sample to be predicted, as well as the first-party model prediction result generated by the target prediction model for the sample to be predicted. The target prediction model is locally iterated by the first device. The target prediction model, trained iteratively locally, is thus achieved, enabling local prediction of the samples to be predicted. Therefore, for samples that are not aligned, the first device can independently and accurately predict the samples based on the target prediction model (which serves as the complete model). Furthermore, the intermediate sample features are sent to the second device, which then uses a vertical federated residual boosting model to jointly perform model prediction on the intermediate sample features and the ID matching samples corresponding to the samples to be predicted. This provides more decision-making basis for the second device to perform model prediction based on the vertical federated residual boosting model, improving the accuracy of the second-party model prediction results generated by the second device. Specifically, due to the vertical federated... The residual enhancement model is obtained by the second device based on the vertical federated common samples, combining the model prediction loss of the target prediction model in the first device on the vertical federated common samples, the intermediate common sample features corresponding to the vertical federated common samples, and the corresponding sample labels, and performing residual learning based on vertical federated learning with the first device. Then, for ID-matching samples aligned with the sample to be predicted, the second device performs model prediction on the ID-matching samples and intermediate sample features based on the vertical federated residual enhancement model to generate more accurate residual enhancement information corresponding to the sample to be predicted, i.e., the second-party model prediction result. It then obtains the first-party model weights corresponding to the target prediction model and receives the second-party model prediction result sent by the second device. The first and second model prediction results are weighted and aggregated based on the weights of the first and second models, corresponding to the vertical federated residual enhancement model, to obtain the target federated prediction result. Therefore, for a sample to be predicted that is an aligned sample between the first and second devices, the first device can utilize the more accurate residual enhancement information (second model prediction result) generated by the second device based on the vertical federated residual enhancement model for the ID matching samples and intermediate sample features aligned with the sample to be predicted. This improves the accuracy of the first model prediction result output by the target prediction model, enabling vertical federated prediction of the sample to be predicted based on more accurate residual enhancement information.Therefore, this method achieves the goal of accurately predicting unaligned samples locally and independently based on the complete model, while performing vertical federated predictions on aligned samples using residual enhancement information that yields higher accuracy. This overcomes the technical limitation where, when performing more accurate federated predictions on aligned samples, the predictor cannot perform sample predictions on unaligned samples based on the complete model, resulting in lower overall sample prediction accuracy. Thus, it improves the overall sample prediction accuracy of vertical federated predictions. Attached Figure Description

[0050] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application.

[0051] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, for those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0052] Figure 1 This is a flowchart illustrating the first embodiment of the vertical federated prediction optimization method of this application;

[0053] Figure 2 This is a flowchart illustrating the second embodiment of the vertical federated prediction optimization method of this application;

[0054] Figure 3 This is a flowchart illustrating the first embodiment of the longitudinal federated learning modeling optimization method of this application;

[0055] Figure 4 This is a flowchart illustrating the second embodiment of the longitudinal federated learning modeling optimization method of this application;

[0056] Figure 5 This is a schematic diagram of the device structure of the hardware operating environment involved in the vertical federated prediction optimization method in this application embodiment;

[0057] Figure 6 This is a schematic diagram of the device structure of the hardware operating environment involved in the vertical federated learning modeling optimization method in the embodiments of this application.

[0058] The purpose, features, and advantages of this application will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. Detailed Implementation

[0059] It should be understood that the specific embodiments described herein are for illustrative purposes only and are not intended to limit the scope of this application.

[0060] This application provides a vertical federated prediction optimization method. In the first embodiment of the vertical federated prediction optimization method of this application, refer to... Figure 1 The vertical federated prediction optimization method is applied to the first device, and the vertical federated prediction optimization method includes:

[0061] Step S10: Extract the sample to be predicted, and obtain the intermediate sample features generated by the feature extractor of the target prediction model for the sample to be predicted, and the first-party model prediction result generated by the target prediction model for the sample to be predicted, wherein the target prediction model is obtained by local iterative training of the first device.

[0062] In this embodiment, it should be noted that the vertical federated prediction optimization method is applied to a vertical federated learning scenario, where both the first device and the second device are participants in the scenario. The first device has samples with labeled samples, while the second device has samples without labeled samples. The first device is the predictor, performing the prediction task, while the second device is the auxiliary data provider, providing residual enhancement information to the first device to improve the accuracy of the prediction results generated by the first device.

[0063] In one feasible approach, the target prediction model can be a multi-layer neural network or a deep factorization machine model, used to classify the samples to be predicted. For example, assuming the output of the last fully connected layer of the target prediction model is z, and the activation function is the sigmoid function, then the classification result P = sigmoid(z), where P is the probability that the sample to be predicted belongs to a preset sample category.

[0064] Specifically, in step S10, the sample to be predicted is extracted, and features are extracted from the sample to be predicted based on the feature extractor in the target prediction model to obtain intermediate sample features. Then, based on the classifier in the target prediction model, the intermediate sample features are fully connected to obtain the target fully connected layer output. Then, the target fully connected layer output is converted into a first-party model prediction result through a preset activation function, wherein the first-party model prediction result can be a classification probability.

[0065] Prior to the steps of obtaining intermediate sample features generated by the feature extractor of the target prediction model for the sample to be predicted, and the first-party model prediction result generated by the target prediction model for the sample to be predicted, the vertical federated prediction optimization method further includes:

[0066] Step A10: Obtain the initial weights of the first-party model and extract the training samples and the training sample labels corresponding to the training samples;

[0067] In this embodiment, it should be noted that the number of training samples is at least 1, the training sample label is the identifier of the training sample, and the initial model weight of the first party is the initial value of the model weight representing the prediction accuracy of the model, which is preset in the first device. The model weight can be set to the logarithm of the ratio of the number of correctly classified samples to the number of misclassified samples; optionally, the initial value of the model weight can be set to 1.

[0068] Step A20: Obtain intermediate training sample features generated by the feature extractor of the target prediction model to be trained for the training samples;

[0069] In this embodiment, based on the feature extractor in the target prediction model to be trained, features are extracted from the training samples to generate intermediate training sample features.

[0070] Step A30: Based on the training sample labels, the training model prediction results corresponding to the intermediate training sample features, and the first-party initial model weights, the target prediction model is iteratively optimized by calculating the first-party model prediction loss corresponding to the target prediction model to be trained, and the target prediction model is obtained.

[0071] In this embodiment, it should be noted that the iterative training process of the target prediction model to be trained includes multiple iteration rounds, wherein each iteration round requires iterative training based on a preset number of training samples.

[0072] Specifically, in step A30, based on the classifier and preset activation function in the target prediction model to be trained, the features of the intermediate training samples are converted into prediction results of the training model. Then, based on the training samples, the prediction results of the training model, and the initial weights of the first-party model, the prediction loss of the first-party model is calculated to determine whether the prediction loss of the first-party model has converged. If the prediction loss of the first-party model has converged, the target prediction model to be trained is used as the target prediction model. If the prediction loss of the first-party model has not converged, based on the gradient calculated from the prediction loss of the first-party model, the target prediction model to be trained is updated using a preset model optimization method. The initial weights of the first-party model are also updated based on the prediction results of the training model and the training sample labels. The process then returns to the execution step: extracting the training samples and their corresponding training sample labels for the next iteration. The preset model optimization method includes gradient descent and gradient ascent, etc. The loss function for calculating the prediction loss of the first-party model based on the training samples, the prediction results of the training model, and the initial weights of the first-party model is as follows:

[0073]

[0074] Among them, L A (θ A ,α A ,X A Y) represents the prediction loss of the first-party model, and N represents the prediction loss of the second-party model. A θ is the number of training samples in one iteration. A For the target prediction model to be trained, α A X represents the initial model weights of the first party. A For N A The training sample set consists of N training samples, and Y is N. A The label set consisting of the training sample labels corresponding to each training sample, y i Let x be the label of the i-th training sample in one iteration. A,i Let be the feature of the i-th training sample in one iteration.

[0075] Further, after step A30, the first device counts the number of correctly classified samples and the number of misclassified samples during the iterative training of the target prediction model, obtaining the first-party correctly classified sample count and the first-party misclassified sample count. Then, by calculating the ratio of the first-party correctly classified sample count to the first-party misclassified sample count, the first-party model weight is generated. The specific process for calculating the first-party model weight is as follows:

[0076]

[0077] Where, α A Let A be the weight of the first-party model, B be the number of correctly classified samples by the first-party model, and B be the number of incorrectly classified samples by the first-party model.

[0078] Step A40: The training sample labels, intermediate training sample features, and the first-party model prediction loss are sent to the second device. The second device calculates the second-party model prediction loss based on the residual boosting model to be trained, the training sample ID matching sample corresponding to the training sample, the intermediate training sample features, the sample labels, and the obtained second-party initial model weights. Based on the residual loss calculated by the second-party model prediction loss and the first-party model prediction loss, the second device optimizes the residual boosting model to be trained to obtain the vertical federated residual boosting model.

[0079] In this embodiment, it should be noted that the initial model weight of the second party is the initial value of the model weight representing the prediction accuracy of the model, which is preset in the second device. The model weight can be set to the logarithm of the ratio of the number of correctly classified samples to the number of misclassified samples. Optionally, the initial value of the model weight can be set to 1.

[0080] Specifically, in step A40, the training sample label, the corresponding first-party model prediction loss, the corresponding training sample, and the corresponding intermediate training sample features for each training sample during the iterative training of the target prediction model are sent to the second device. The second device then extracts the training sample ID and, based on the training sample ID, finds the ID-matching sample corresponding to the sample to be predicted. The second device then performs model prediction by inputting the training sample ID-matching sample into the residual boosting model to be trained, obtaining the second-party training model prediction result corresponding to the training sample ID-matching sample. The second device then concatenates the training sample ID-matching sample and the corresponding intermediate training sample features to perform feature enhancement on the training sample ID-matching sample, obtaining a training feature-enhanced sample. The model prediction is then performed on the training feature-enhanced sample based on the residual boosting model to be trained, obtaining the second-party model prediction loss. Finally, the second-party model prediction loss is calculated based on the second-party training model prediction result corresponding to the training sample ID-matching sample, the corresponding training sample label, and the obtained second-party initial model weights.

[0081] The process of calculating the prediction loss of the second-party model can be referred to the process of calculating the prediction loss of the first-party model by the first device, and will not be repeated here. Then, based on the prediction losses of the first-party model and the second-party model, the residual loss is calculated, and it is determined whether the residual loss has converged. If the residual loss converges, the residual boosting model to be trained is used as the longitudinal federated residual boosting model; if the residual loss has not converged, the residual boosting model to be trained is updated based on the gradient calculated by the residual loss, and the second-party initial model weights are updated based on the prediction results of the second-party training model corresponding to the training sample ID and the corresponding sample labels, and the execution step is returned: the second device extracts the training sample ID and performs the next iteration. The loss function used by the second device to calculate the residual loss is as follows:

[0082]

[0083] Where, L(θ) B ,α B ,X B Y) is the residual loss, N C θ represents the number of training sample ID matching samples in one iteration. B For the residual boosting model to be trained, α B X represents the initial model weights of the second party. B For N C The training sample set consists of N training sample ID matching samples, where Y is N. C The set of labels consisting of the training sample labels corresponding to the training sample IDs of the matching samples. Let i be the label of the i-th training sample in one iteration. The features of the sample matching the ID of the i-th training sample in one iteration. For N C The first-party model prediction loss corresponding to each training sample ID matching sample. The intermediate training sample features are the matching samples of the ID of the i-th training sample in one iteration.

[0084] Furthermore, the second device counts the number of correctly classified samples and the number of misclassified samples during the iterative training of the longitudinal federated residual boosting model, obtaining the second-party correctly classified sample count and the second-party misclassified sample count. Then, by calculating the ratio of the second-party correctly classified sample count to the second-party misclassified sample count, the second-party model weights are generated. The specific process of the second device generating the second-party model weights is similar to the process of the first device generating the first-party model weights described above, and will not be repeated here.

[0085] Step S20: The intermediate sample features are sent to the second device, so that the second device can perform model prediction on the intermediate sample features and the ID matching samples corresponding to the sample to be predicted based on the longitudinal federated residual boosting model, and obtain the second-party model prediction result.

[0086] In this embodiment, it should be noted that the vertical federated residual enhancement model is obtained by the second device performing residual learning based on vertical federated learning with the first device, using the vertical federated common samples, the model prediction loss of the target prediction model in the first device on the vertical federated common samples, the intermediate common sample features corresponding to the vertical federated common samples, and the corresponding sample labels. The specific process of the second device performing residual learning based on vertical federated learning with the first device to obtain the vertical federated residual enhancement model can be referred to steps A10 to A30, and will not be repeated here. The vertical federated common samples are samples in the second device whose IDs are aligned with those in the first device, that is, training sample ID matching samples corresponding to the training samples in the first device.

[0087] Specifically, the intermediate sample features are sent to the second device, which then concatenates the ID matching sample corresponding to the sample to be predicted with the corresponding intermediate sample features to enhance the ID matching sample and obtain a feature-enhanced sample. The feature-enhanced sample corresponding to the sample to be predicted is then input into the vertical federated residual enhancement model to perform model prediction on the feature-enhanced sample and obtain the second-party model prediction result. The second device then sends the second-party model prediction result and the second-party model weights corresponding to the vertical federated residual enhancement model to the first device.

[0088] Prior to the step of sending the intermediate sample features to the second device, the longitudinal federated prediction optimization method further includes:

[0089] Step B10: Send the ID of the sample to be predicted corresponding to the sample to be predicted to the second device so that the second device can find the ID matching sample corresponding to the sample ID to be predicted.

[0090] In this embodiment, it should be noted that the sample ID to be predicted is the sample ID of the sample to be predicted.

[0091] Step B20: If a search failure message is received from the second device, the prediction result of the first-party model is used as the target prediction result.

[0092] In this embodiment, if a lookup failure message is received from the second device, the prediction result of the first-party model is used as the target prediction result. Specifically, if a lookup failure message is received from the second device, it proves that the sample to be predicted is not an aligned sample between the first and second devices, and the prediction result of the first-party model is used as the target prediction result to achieve the purpose of independently predicting the sample to be predicted based on the target prediction model as a complete model.

[0093] Step B30: If no search failure information is received from the second device, then the following step is executed: send the intermediate sample features to the second device.

[0094] In this embodiment, if the search failure information sent by the second device is not received, the following steps are performed: the intermediate sample features are sent to the second device to obtain the residual improvement information with higher accuracy calculated based on the intermediate sample features sent by the second device, wherein the residual improvement information is the second-party model prediction result output by the longitudinal federated residual improvement model in the second device.

[0095] Step S30: Obtain the first-party model weights corresponding to the target prediction model, and receive the second-party model prediction results and the second-party model weights corresponding to the longitudinal federated residual improvement model sent by the second device;

[0096] In this embodiment, it should be noted that the weight of the first-party model is obtained by the first device by calculating the ratio of the number of correctly classified samples to the number of incorrectly classified samples of the first-party model during the iterative training process of the target prediction model.

[0097] Step S40: Based on the weights of the first-party model and the weights of the second-party model, perform weighted aggregation on the prediction results of the first-party model and the prediction results of the second-party model to obtain the target federated prediction result.

[0098] In this embodiment, specifically, based on the weights of the first-party model and the weights of the second-party model, the prediction results of the first-party model and the prediction results of the second-party model are weighted and aggregated using preset aggregation rules to obtain the target federated prediction result. The preset aggregation rules include summation and averaging, thereby achieving the goal of improving the accuracy of the first device's prediction of the sample to be predicted by utilizing the residual enhancement information generated by the second device.

[0099] It should be noted that since the residual enhancement information is generated by the vertical federated residual enhancement model based on the feature enhancement samples, and the feature enhancement samples are generated by concatenating the ID matching samples and the corresponding intermediate sample features of the sample to be predicted, the purpose of enhancing the ID matching samples based on the intermediate sample features in the first device is achieved. This makes the input of the vertical federated residual enhancement model have more feature information, and the vertical federated residual enhancement model has more decision-making basis for generating residual enhancement information. The vertical federated residual enhancement model can output residual enhancement information with higher accuracy. Therefore, the higher accuracy residual enhancement information generated by the second device is further improved on the basis of using residual enhancement information to improve the accuracy of the first device in predicting the sample to be predicted as the alignment sample.

[0100] Furthermore, the target prediction model can be set as a binary classification model for use as a recommendation model. That is, by performing binary classification on the sample to be predicted, it determines whether to recommend the item corresponding to the sample to be predicted or whether to recommend the item to the user corresponding to the sample to be predicted. Since the embodiments of this application achieve the goal of accurately predicting unaligned samples locally based on the complete model when performing vertical federated prediction based on residual improvement information with higher accuracy on aligned samples, the overall sample prediction accuracy of vertical federated prediction is improved, and the overall recommendation accuracy of the recommendation model is improved.

[0101] Furthermore, existing vertical federated prediction models require collaboration among all participants in the vertical federated learning process to perform model predictions. If any participant experiences data loss or service outage, it becomes impossible to predict samples based on the complete model and sample data, thus affecting the accuracy of sample predictions. However, in the embodiments of this application, since the target prediction model is held solely by the first device, even if the second device experiences data loss or service outage, the first device can still rely on the target prediction model, which serves as the complete model, to perform sample predictions independently. This improves the accuracy of sample predictions when participants in the vertical federated learning process experience data loss or service outages.

[0102] This application provides a vertical federated prediction optimization method. Compared to existing technologies where, for misaligned samples in vertical federated prediction scenarios, the predictor performs local prediction based on a locally held portion of the vertical federated prediction model, this application first extracts the sample to be predicted and obtains intermediate sample features generated by the feature extractor of the target prediction model for the sample to be predicted, as well as the first-party model prediction result generated by the target prediction model for the sample to be predicted. The target prediction model is obtained through local iterative training by the first device, thus achieving the goal of predicting the sample to be predicted locally based on the locally trained target prediction model. Therefore, for misaligned samples to be predicted, the first device can independently and accurately predict the sample to be predicted based on the target prediction model as a complete model.Furthermore, the intermediate sample features are sent to the second device, which then performs model prediction on the intermediate sample features and the ID matching samples corresponding to the sample to be predicted based on the vertical federated residual boosting model. This provides more decision-making basis for the second device to perform model prediction based on the vertical federated residual boosting model, improving the accuracy of the second-party model prediction results generated by the second device. The vertical federated residual boosting model is obtained by the second device based on vertical federated common samples, combined with the model prediction loss of the target prediction model on the vertical federated common samples in the first device, the intermediate common sample features corresponding to the vertical federated common samples, and the corresponding sample labels, through residual learning based on vertical federated learning with the first device. Therefore, for ID matching samples aligned with the sample to be predicted, the second device can generate more accurate residual boosting information corresponding to the sample to be predicted by performing model prediction on the ID matching samples and intermediate sample features based on the vertical federated residual boosting model, i.e., the second-party model prediction result. This allows the second device to obtain the first-party model weights corresponding to the target prediction model and receive the second-party model prediction results and the vertical federated residual boosting model sent by the second device. The second-party model weights corresponding to the residual enhancement model are used, and the prediction results of the first-party model and the second-party model are weighted and aggregated based on the weights of the first-party model and the second-party model to obtain the target federated prediction result. Therefore, for the sample to be predicted that is an aligned sample between the first device and the second device, the first device can use the higher accuracy residual enhancement information (second-party model prediction result) generated by the second device based on the vertical federated residual enhancement model for the ID matching sample and intermediate sample features aligned with the sample to be predicted to improve the accuracy of the first-party model prediction result output by the target prediction model. This achieves vertical federated prediction of the sample to be predicted based on the higher accuracy residual enhancement information. It realizes the purpose of accurately predicting unaligned samples locally based on the complete model when performing vertical federated prediction of aligned samples based on the higher accuracy residual enhancement information. This overcomes the technical defect that the predictor cannot perform sample prediction of unaligned samples based on the complete model when performing higher accuracy federated prediction of aligned samples, which makes the overall sample prediction accuracy lower. This improves the overall sample prediction accuracy of vertical federated prediction.

[0103] Furthermore, referring to Figure 2 In another embodiment of this application, the vertical federated prediction optimization method is applied to a second device, and the vertical federated prediction optimization method includes:

[0104] Step C10: Receive intermediate sample features sent by the first device, and search for samples whose IDs match the intermediate sample features;

[0105] In this embodiment, it should be noted that the intermediate sample features are generated by the first device based on the feature extractor in the target prediction model to extract features from the sample to be predicted, and the ID matching sample is a sample in the second device whose sample ID is consistent with the sample ID of the sample to be predicted corresponding to the intermediate sample features.

[0106] Specifically, in step C10, the system receives intermediate sample features generated by the feature extractor based on the target prediction model from the first device, as well as the prediction sample ID corresponding to the prediction sample sent by the first device, and then searches for ID matching samples based on the prediction sample ID.

[0107] The vertical federated prediction optimization method further includes, after the step of finding ID matching samples, the following:

[0108] Step D10: If the search is successful, proceed to the following step: Based on the longitudinal federated residual boosting model, perform model prediction on the features of the ID matching sample and the intermediate sample to obtain the second-party model prediction result;

[0109] In this embodiment, if the search is successful, it proves that the second device possesses the aligned sample corresponding to the sample to be predicted, i.e., the ID matching sample. Then, the following steps are executed: Based on the vertical federated residual enhancement model, model prediction is performed on the ID matching sample and the intermediate sample features to obtain the second-party model prediction result. Based on the vertical federated residual enhancement model, the ID matching sample corresponding to the sample to be predicted, and the corresponding intermediate sample features, more accurate residual enhancement information corresponding to the sample to be predicted is generated. The residual enhancement information is the second-party model prediction result. This residual enhancement information is then sent to the first device, allowing the first device to improve the accuracy of its sample prediction results for the sample to be predicted based on the more accurate residual enhancement information.

[0110] Step D20: If the search fails, the search failure information is fed back to the first device, so that after receiving the search failure information, the first device can use the first-party model prediction result generated by the target prediction model for the sample to be predicted as the target prediction result.

[0111] In this embodiment, if the search fails, it proves that the second device does not have an aligned sample corresponding to the sample to be predicted, and then sends a search failure message back to the first device. After receiving the search failure message, the first device can directly use the first-party model prediction result generated by the target prediction model for the sample to be predicted as the target prediction result, so as to achieve the purpose of independently predicting the unaligned sample.

[0112] Step C20: Based on the longitudinal federated residual boosting model, perform model prediction on the features of the ID matching samples and the intermediate samples to obtain the second-party model prediction results;

[0113] In this embodiment, it should be noted that the vertical federated residual enhancement model is obtained by the second device performing residual learning based on vertical federated learning with the first device, using the second device based on vertical federated common samples, combining the model prediction loss of the target prediction model in the first device on the vertical federated common samples, the intermediate common sample features corresponding to the vertical federated common samples, and the corresponding sample labels. The specific process of the second device performing residual learning based on vertical federated learning with the first device to obtain the vertical federated residual enhancement model can be referred to steps A10 to A40 above, and will not be repeated here.

[0114] Specifically, in step C20, based on the intermediate sample features, the ID matching sample is feature-enhanced to obtain a feature-enhanced sample. Then, the feature-enhanced sample is input into the longitudinal federated residual enhancement model to perform model prediction and generate a second-party model prediction result.

[0115] The step of performing model prediction on the ID matching samples and the intermediate sample features based on the longitudinal federated residual boosting model to obtain the second-party model prediction result includes:

[0116] Step C21: Concatenate the features of the ID matching sample and the intermediate sample to obtain the feature-enhanced sample;

[0117] In this embodiment, the features of the ID matching sample and the intermediate sample are concatenated to enhance the features of the ID matching sample based on the features of the intermediate sample, thereby obtaining a feature-enhanced sample.

[0118] In another feasible approach, the ID matching sample and the intermediate sample features can be weighted and concatenated to enhance the features of the ID matching sample based on the intermediate sample features, thereby obtaining a feature-enhanced sample.

[0119] Step C22: Based on the longitudinal federated residual enhancement model, perform model prediction on the feature-enhanced samples to obtain the prediction results of the second-party model.

[0120] Specifically, in step C22, the feature-enhanced samples are input into the vertical federated residual boosting model to perform data processing. This data processing includes convolution, pooling, and fully connected layers, thereby obtaining the output of the last fully connected layer in the vertical federated residual boosting model. Then, using a preset activation function, the fully connected layer output is converted into a second-party model prediction result.

[0121] Prior to the step of performing model prediction on the features of the ID matching samples and the intermediate samples based on the vertical federated residual enhancement model to obtain the second-party model prediction result, the vertical federated prediction optimization method further includes:

[0122] Step E10: Obtain the initial weights of the second-party model and receive the intermediate training sample features, training sample labels, and first-party model prediction loss sent by the first device. The first-party model prediction loss is calculated by the first device based on the first-party model prediction result of the target prediction model on the training sample corresponding to the training sample ID and the training sample label. The intermediate training sample features are obtained by the first device based on the feature extractor of the target prediction model for feature extraction of the training sample.

[0123] In this embodiment, it should be noted that the specific process of the first device generating the first-party model prediction loss and intermediate training sample features can refer to the specific content of steps A10 to A30 above, and will not be repeated here.

[0124] Additionally, it should be noted that the first device needs to send the training sample ID, the corresponding training sample label, the corresponding first-party model prediction loss, and the corresponding intermediate training sample features of all training samples corresponding to the target prediction model during the iterative training process to the second device.

[0125] Step E20: Obtain training sample ID matching samples, and based on the residual improvement model to be trained, perform model prediction on the training sample ID matching samples and the intermediate training sample features to obtain the second-party training model prediction result.

[0126] Specifically, in step E20, training sample ID matching samples are extracted, and then the training sample ID matching samples are concatenated with the intermediate training sample features to perform feature enhancement on the training sample ID matching samples based on the intermediate training sample features, thereby obtaining training feature-enhanced samples. Then, by inputting the training feature-enhanced samples into the residual enhancement model to be trained, model prediction is performed to obtain the second-party training model prediction result.

[0127] Step E30: Calculate the second-party model prediction loss based on the training sample labels, the second-party initial model weights, and the prediction results of the second-party trained model.

[0128] In this embodiment, the second device calculates the loss function for the prediction loss of the second-party model based on the training sample labels, the initial weights of the second-party model, and the prediction results of the second-party trained model, as follows:

[0129]

[0130] Where, L(θ) B ,α B ,X B Y) is the residual loss, N C θ represents the number of training sample ID matching samples in one iteration. B For the residual boosting model to be trained, α B X represents the initial model weights of the second party. B For N C The training sample set consists of N training sample ID matching samples, where Y is N. C The set of labels consisting of the training sample labels corresponding to the training sample IDs of the matching samples. Let i be the label of the i-th training sample in one iteration. The features of the sample matching the ID of the i-th training sample in one iteration. For N C The first-party model prediction loss corresponding to each training sample ID matching sample. The intermediate training sample features are the matching samples of the ID of the i-th training sample in one iteration.

[0131] Step E40: Based on the residual loss generated by the prediction loss of the first-party model and the prediction loss of the second-party model, iteratively optimize the training residual enhancement model to obtain the longitudinal federated residual enhancement model.

[0132] Specifically, in step E40, residual loss is calculated based on the prediction loss of the first-party model and the prediction loss of the second-party model, and then it is determined whether the residual loss has converged. If the residual boosting model converges, the residual boosting model to be trained is used as the longitudinal federated residual boosting model; if the residual loss has not converged, the residual boosting model to be trained is updated based on the gradient calculated by the residual loss using a preset model optimization method, and the second-party initial model weights are updated based on the prediction results of the second-party training model corresponding to the training sample ID matching sample and the corresponding training sample label, and the process returns to the execution step: obtain the training sample ID matching sample and perform the next iteration. The specific process of calculating the residual loss based on the prediction loss of the first-party model and the prediction loss of the second-party model can be referred to the specific content in step A40 above, and will not be repeated here.

[0133] Furthermore, the second device counts the number of correctly classified samples and the number of misclassified samples in the residual boosting model during iterative training, obtaining the second-party correctly classified sample count and the second-party misclassified sample count. Then, by calculating the ratio of the second-party correctly classified sample count to the second-party misclassified sample count, the second-party model weights are generated. The specific process of the second device generating the second-party model weights is similar to the process of the first device generating the first-party model weights, and will not be repeated here.

[0134] Step C30: Obtain the second-party model weights corresponding to the longitudinal federated residual enhancement model, and send the second-party model prediction results and the second-party model weights to the first device, so that the first device can generate a target federated prediction result based on the first-party model prediction results generated by the target prediction model for the sample to be predicted corresponding to the ID matching sample, the first-party model weights corresponding to the target prediction model, the second-party model prediction results, and the second-party model weights, wherein the target prediction model is obtained by the first device through local iterative training.

[0135] In this embodiment, it should be noted that the specific process by which the first device generates the first-party model prediction result for the sample to be predicted corresponding to the training sample ID matching sample based on the target prediction model can be referred to the specific steps in step S10, and will not be repeated here.

[0136] Specifically, the second-party model weights corresponding to the longitudinal federated residual enhancement model are obtained, and the prediction results and weights of the second-party model are sent to the first device. The first device then performs weighted aggregation on the prediction results of the first-party model and the second-party model based on the first-party model weights and the second-party model weights corresponding to the target prediction model, according to a preset aggregation rule, to obtain the target federated prediction result. This achieves higher accuracy residual enhancement information based on the second device, optimizes the first-party model prediction results of the samples to be predicted in the first device, and improves the accuracy of the first device's sample prediction results for the samples to be predicted.

[0137] This application provides a vertical federated prediction optimization method. Compared to existing technologies where the predictor performs local predictions based on a locally held portion of the vertical federated prediction model for misaligned samples in a vertical federated prediction scenario, this application first receives intermediate sample features sent by a first device and finds ID-matching samples corresponding to the intermediate sample features. Based on a vertical federated residual enhancement model, it performs model prediction on the ID-matching samples and the intermediate sample features to obtain a second-party model prediction result. This achieves the goal of generating more accurate residual enhancement information for the sample to be predicted using the intermediate sample features sent by the first device. Then, it obtains the second-party model weights corresponding to the vertical federated residual enhancement model and sends the second-party model prediction result and the second-party model weights to the first device. The first device then generates a target federated prediction result based on the first-party model prediction result generated by the target prediction model for the sample to be predicted corresponding to the ID-matching sample, the first-party model weights corresponding to the target prediction model, the second-party model prediction result, and the second-party model weights. The target prediction model is obtained through local iterative training by the first device. This enables the optimization of the first-party model prediction results generated by the first device using the more accurate residual enhancement information generated by the second device for aligned samples, thereby generating target federated prediction results with higher sample prediction accuracy. Furthermore, since the target prediction model is obtained through local iterative training by the first device, for unaligned samples, the first device can also perform sample prediction locally on its own based on the target prediction model, which serves as a complete model. This overcomes the technical defect that the predictor cannot perform sample prediction based on the complete model for unaligned samples when performing more accurate federated prediction for aligned samples, resulting in lower overall sample prediction accuracy. This improves the overall sample prediction accuracy of vertical federated prediction.

[0138] Furthermore, referring to Figure 3In another embodiment of this application, a vertical federated learning modeling optimization method is also provided. This vertical federated learning modeling optimization method is applied to a first device and includes:

[0139] Step F10: Obtain the initial weights of the first-party model and extract the training samples and the corresponding training sample labels;

[0140] In this embodiment, it should be noted that the vertical federated learning modeling optimization method is applied to a vertical federated learning scenario, where both the first device and the second device are participants in the scenario. The first device has samples with labeled samples, while the second device has samples without labeled samples. The first device acts as the predictor, used to construct a prediction model, while the second device acts as the auxiliary data provider, used to construct a vertical federated residual boosting model that provides residual boosting information to the first device, thereby improving the accuracy of the prediction results generated by the prediction model in the first device.

[0141] Additionally, it should be noted that the number of training samples is at least one, the training sample label is the identifier of the training sample, and the initial model weight of the first party is an initial value of the model weight representing the prediction accuracy of the model, preset in the first device. The model weight can be set to the logarithm of the ratio of the number of correctly classified samples to the number of misclassified samples. Optionally, the initial value of the model weight can be set to 1.

[0142] Step F20: Obtain intermediate training sample features generated by the feature extractor of the target prediction model to be trained for the training samples.

[0143] In this embodiment, features are extracted from the training samples based on the feature extractor in the target prediction model to be trained, thereby obtaining intermediate training sample features.

[0144] Step F30: Based on the training sample labels, the training model prediction results corresponding to the intermediate training sample features, and the first-party initial model weights, the target prediction model is iteratively optimized by calculating the first-party model prediction loss corresponding to the target prediction model to be trained, and the target prediction model is obtained.

[0145] Specifically, in step F30, the intermediate training sample features are fully connected through the classifier in the target prediction model to be trained, obtaining the output of the fully connected training layer. Then, the output of the fully connected training layer is converted into the prediction result of the training model through a preset activation function. Then, the prediction loss of the first-party model is calculated based on the training samples, the prediction result of the training model, and the weights of the first-party initial model. Then, it is determined whether the prediction loss of the first-party model has converged. If the prediction loss of the first-party model has converged, the target prediction model to be trained is used as the target prediction model. If the prediction loss of the first-party model has not converged, the target prediction model to be trained is updated based on the gradient calculated by the prediction loss of the first-party model through a preset model optimization method, and the weights of the first-party initial model are updated based on the prediction result of the training model and the training sample labels. Then, the process returns to the execution step: extract the training samples and the training sample labels corresponding to the training samples, and perform the next iteration. The preset model optimization method includes gradient descent and gradient ascent, etc. The calculation process of calculating the prediction loss of the first-party model based on the training samples, the prediction results of the training model and the initial weights of the first-party model can be referred to the specific content of steps A10 to A30 above, and will not be repeated here.

[0146] The step of obtaining the target prediction model by iteratively optimizing the target prediction model based on the training sample labels, the training model prediction results corresponding to the intermediate training sample features, and the first-party initial model weights includes:

[0147] Step F31: Based on the classifier in the target prediction model to be trained, convert the features of the intermediate training samples into prediction results of the training model;

[0148] In this embodiment, based on the classifier in the target prediction model to be trained, the features of the intermediate training samples are fully connected to obtain the output of the fully connected training layer. Then, the output of the fully connected training layer is converted into the prediction result of the training model through a preset activation function.

[0149] Step F32: Calculate the first-party model prediction loss based on the training sample labels, the training model prediction results, and the first-party initial model weights;

[0150] In this embodiment, it should be noted that the specific calculation process of step F32 can be referred to the content of step A30, and will not be repeated here.

[0151] Step F33: Update the initial weights of the first-party model based on the prediction results of the training model and the labels of the training samples;

[0152] Specifically, in step F33, based on the prediction results of the training model and the labels of the training samples, the number of currently misclassified samples and the number of currently correctly classified samples corresponding to the residual improvement model to be trained are updated. Then, the first-party initial model weights are recalculated by calculating the ratio of the current number of correctly classified samples to the current number of misclassified samples. The process of recalculating the first-party initial model weights can be referred to in detail to the process of the first device calculating the first-party model weights after step A30, and will not be repeated here.

[0153] Step F34: Based on the prediction loss of the first-party model and the updated weights of the first-party initial model, iteratively optimize the target prediction model to be trained to obtain the target prediction model.

[0154] Specifically, in step F34, it is determined whether the prediction loss of the first-party model has converged. If the prediction loss of the first-party model has converged, the target prediction model to be trained is used as the target prediction model, and the updated first-party initial model weights are used as the first-party model weights. If the prediction loss of the first-party model has not converged, the target prediction model to be trained is updated based on the gradient calculated by the prediction loss of the first-party model using a preset model optimization method, and the execution steps are returned: extracting training samples and the training sample labels corresponding to the training samples, and performing the next iteration based on the updated target prediction model to be trained and the updated first-party initial model weights.

[0155] The longitudinal federated learning modeling optimization method further includes, after the step of iteratively optimizing the target prediction model by calculating the first-party model prediction loss corresponding to the target prediction model to be trained based on the training sample labels, the training model prediction results corresponding to the intermediate training sample features, and the first-party initial model weights, to obtain the target prediction model:

[0156] Step G10: Obtain the number of correctly classified first-party samples and the number of incorrectly classified first-party samples corresponding to the target prediction model;

[0157] In this embodiment, it should be noted that the number of correctly classified samples by the first party is the number of training samples in which the output classification label of the training sample and the corresponding training sample label are consistent during the iterative training process of the target prediction model, and the number of incorrectly classified samples by the first party is the number of training samples in which the output classification label of the training sample and the corresponding training sample label are inconsistent during the iterative training process of the target prediction model.

[0158] Step G20: The first-party model weights are generated by calculating the ratio of the number of correctly classified samples to the number of incorrectly classified samples.

[0159] In this embodiment, the specific calculation formula for step G20 is as follows:

[0160]

[0161] Where, α A Let A be the weight of the first-party model, B be the number of correctly classified samples by the first-party model, and B be the number of incorrectly classified samples by the first-party model.

[0162] Step F40: The training sample labels, the intermediate training sample features, and the first-party model prediction loss are sent to the second device so that the second device can calculate the second-party model prediction loss and optimize the training residual enhancement model based on the residual loss calculated by the second-party model prediction loss and the first-party model prediction loss to obtain the vertical federated residual enhancement model.

[0163] In this embodiment, it should be noted that the second-party model prediction loss is calculated based on the training residual enhancement model to be trained, the training sample ID matching sample corresponding to the training sample, the intermediate training sample features, the sample label, and the obtained second-party initial model weights.

[0164] Specifically, during the iterative training of the target prediction model, the training sample labels, corresponding first-party model prediction losses, corresponding training sample IDs, and corresponding intermediate training sample features for all training samples are sent to the second device. The second device then extracts the training sample IDs and, based on these IDs, finds matching samples. It inputs the training feature enhancement samples, which are the same as the matching samples and the intermediate training sample features, into the residual enhancement model to be trained to perform model prediction, obtaining the second-party training model prediction result. Then, based on the second-party training model prediction result corresponding to the matching samples, the corresponding training sample labels, and the obtained second-party initial model weights, the second-party model prediction loss is calculated. The second device then calculates the residual loss based on the first-party model prediction loss and the second-party model prediction loss, and iteratively optimizes the residual enhancement model to be trained based on the residual loss to obtain the vertical federated residual enhancement model. The specific process of the second device constructing the vertical federated residual enhancement model can be found in steps A10 to A40, and will not be repeated here.

[0165] The method further includes the following steps after sending the training sample labels, intermediate training sample features, and the first-party model prediction loss to the second device, so that the second device can calculate the second-party model prediction loss based on the residual boosting model to be trained, the training sample ID matching sample corresponding to the training sample, the intermediate training sample features, and the sample labels, and optimize the residual boosting model to be trained based on the residual loss calculated by the second-party model prediction loss and the first-party model prediction loss to obtain the vertical federated residual boosting model:

[0166] Step H10: Extract the sample to be predicted, and obtain the intermediate sample features generated by the feature extractor of the target prediction model for the sample to be predicted, and the first-party model prediction result generated by the target prediction model for the sample to be predicted.

[0167] In this embodiment, specifically, a sample to be predicted is extracted, and features are extracted from the sample to be predicted based on the feature extractor in the target prediction model to obtain intermediate sample features. Then, based on the classifier in the target prediction model, the intermediate sample features are fully connected to obtain the target fully connected layer output. Then, the target fully connected layer output is converted into a first-party model prediction result through a preset activation function, wherein the first-party model prediction result can be a classification probability.

[0168] Step H20: The intermediate sample features are sent to the second device, so that the second device can perform model prediction on the intermediate sample features and the ID matching samples corresponding to the sample to be predicted based on the longitudinal federated residual boosting model, and obtain the second-party model prediction result.

[0169] In this embodiment, specifically, the intermediate sample features are sent to the second device, and then the second device concatenates the ID matching sample corresponding to the sample to be predicted and the corresponding intermediate sample features to perform feature enhancement on the ID matching sample to obtain a feature-enhanced sample. Then, by inputting the feature-enhanced sample corresponding to the sample to be predicted into the vertical federated residual enhancement model, the model prediction is performed on the feature-enhanced sample to obtain the second-party model prediction result. Finally, the second device sends the second-party model prediction result and the second-party model weights corresponding to the vertical federated residual enhancement model to the first device.

[0170] Step H30: Receive the second-party model prediction results and the second-party model weights corresponding to the longitudinal federated residual improvement model sent by the second device;

[0171] Step H40: Based on the first-party model weights and the second-party model weights corresponding to the target prediction model, the prediction results of the first-party model and the prediction results of the second-party model are weighted and aggregated to obtain the target federated prediction result.

[0172] In this embodiment, specifically, based on the weights of the first-party model and the weights of the second-party model, the prediction results of the first-party model and the prediction results of the second-party model are weighted and aggregated using preset aggregation rules to obtain the target federated prediction result. The preset aggregation rules include summation and averaging, thereby achieving the goal of improving the accuracy of the first device's prediction of the sample to be predicted by utilizing the residual enhancement information generated by the second device.

[0173] This application provides a method for optimizing vertical federated learning modeling. Specifically, it obtains the initial weights of a first-party model and extracts training samples and their corresponding labels. Then, it obtains intermediate training sample features generated by a feature extractor of the target prediction model for the training samples. Based on the training sample labels, the prediction results of the training model corresponding to the intermediate training sample features, and the initial weights of the first-party model, it iteratively optimizes the target prediction model by calculating the prediction loss of the first-party model corresponding to the target prediction model, thus obtaining the target prediction model. This achieves the goal of building a complete target prediction model locally on a first device. The training sample labels, the intermediate training sample features, and the prediction loss of the first-party model are then sent to a second device for calculating the prediction loss of the second-party model. Based on the residual loss calculated from the second-party model prediction loss and the first-party model prediction loss, the residual boosting model for the target prediction model is optimized, resulting in a vertical federated residual boosting model, thus realizing residual learning based on vertical federated learning. By leveraging the features of intermediate training samples corresponding to the training samples in the first device, the vertical federated residual enhancement model constructed in the second device expands the feature dimension of the training sample ID matching samples in the second device, resulting in higher prediction accuracy. Therefore, based on the target prediction model in the first device and the vertical federated residual enhancement model in the second device, it is possible to perform accurate sample prediction for unaligned samples locally, based on the complete model, while performing vertical federated prediction on aligned samples using more accurate residual enhancement information. This lays the foundation for overcoming the technical limitation that while the predictor performs more accurate federated prediction on aligned samples, it cannot perform sample prediction for unaligned samples based on the complete model, leading to lower overall sample prediction accuracy.

[0174] Furthermore, referring to Figure 4In another embodiment of this application, a vertical federated learning modeling optimization method is also provided. This vertical federated learning modeling optimization method is applied to a second device and includes:

[0175] Step R10: Obtain the initial weights of the second-party model and receive the intermediate training sample features, training sample labels and the prediction loss of the first-party model sent by the first device.

[0176] In this embodiment, it should be noted that the first-party model prediction loss is calculated by the first device based on the first-party model prediction result of the target prediction model on the training sample corresponding to the training sample ID and the training sample label. The intermediate training sample features are obtained by the first device through feature extraction of the training samples based on the feature extractor of the target prediction model. The specific process of the first device generating the first-party model prediction loss and intermediate training sample features can be referred to the specific content of steps A10 to A30, and will not be repeated here.

[0177] Specifically, the system obtains the initial weights of the second-party model and receives the training sample labels, the corresponding first-party model prediction loss, and the corresponding intermediate training sample features of all training samples in the iterative training process of the target prediction model sent by the first device.

[0178] Step R20: Obtain training sample ID matching samples, and based on the residual improvement model to be trained, perform model prediction on the training sample ID matching samples and the intermediate training sample features to obtain the second-party training model prediction result.

[0179] In this embodiment, specifically, the training sample ID is extracted, and the training sample ID matching sample corresponding to the training sample ID is found. Then, the training sample ID matching sample and the intermediate training sample features are concatenated to perform feature enhancement on the training sample ID matching sample based on the intermediate training sample features, thereby obtaining a training feature-enhanced sample. The training feature-enhanced sample is then input into the residual enhancement model to be trained to perform model prediction, thereby obtaining the second-party training model prediction result.

[0180] Step R30: Calculate the second-party model prediction loss based on the training sample labels, the second-party initial model weights, and the prediction results of the second-party trained model.

[0181] In this embodiment, it should be noted that the specific process of the second device calculating the second-party model prediction loss in step R30 can be referred to the specific content in step E30, and will not be repeated here.

[0182] Step R40: Based on the residual loss generated by the prediction loss of the first-party model and the prediction loss of the second-party model, iteratively optimize the training residual enhancement model to obtain the longitudinal federated residual enhancement model.

[0183] In this embodiment, specifically, based on the prediction loss of the first-party model and the prediction loss of the second-party model, a residual loss is calculated, and then it is determined whether the residual loss has converged. If the residual boosting model converges, the residual boosting model to be trained is used as the vertical federated residual boosting model; if the vertical federated residual boosting model has not converged, based on the gradient calculated by the residual loss, the residual boosting model to be trained is updated using a preset model optimization method, and based on the prediction result of the second-party training model corresponding to the training sample ID matching sample and the corresponding sample label, the weights of the second-party initial model are updated, and the execution step is returned: obtain the training sample ID matching sample and perform the next iteration. The specific process of calculating the residual loss based on the prediction loss of the first-party model and the prediction loss of the second-party model can be found in step A40, and will not be repeated here.

[0184] The longitudinal federated learning modeling optimization method further includes, after the step of iteratively optimizing the trainable residual boosting model based on the residual loss generated by the first-party model prediction loss and the second-party model prediction loss to obtain the longitudinal federated residual boosting model:

[0185] Step T10: Obtain the number of correctly classified samples and the number of incorrectly classified samples corresponding to the longitudinal federated residual boosting model.

[0186] In this embodiment, it should be noted that the number of correctly classified samples in the second party is the number of training sample ID matching samples whose output classification label and corresponding training sample label are consistent during the iterative training process of the vertical federated residual boosting model; the number of incorrectly classified samples in the second party is the number of training sample ID matching samples whose output classification label and corresponding training sample label are inconsistent during the iterative training process of the vertical federated residual boosting model.

[0187] Step T20: The second-party model weights are generated by calculating the ratio of the number of correctly classified samples to the number of incorrectly classified samples.

[0188] In this embodiment, the specific calculation formula for step T20 is as follows:

[0189]

[0190] Where, α AHere, A represents the number of correctly classified samples by the second-party model, and B represents the number of incorrectly classified samples by the second-party model.

[0191] The method for vertical federated learning modeling optimization, following the step of iteratively optimizing the trainable residual boosting model based on the residual loss generated by the first-party model prediction loss and the second-party model prediction loss to obtain the vertical federated residual boosting model, further includes:

[0192] Step Y10: Receive intermediate sample features sent by the first device, and search for samples whose IDs match the intermediate sample features;

[0193] In this embodiment, specifically, the system receives intermediate sample features generated by the feature extractor based on the target prediction model from the first device, as well as the prediction sample ID corresponding to the prediction sample sent by the first device, and then searches for ID-matching samples based on the prediction sample ID.

[0194] Step Y20: Based on the longitudinal federated residual boosting model, perform model prediction on the ID matching samples and the intermediate sample features to obtain the second-party model prediction result;

[0195] In this embodiment, based on the features of the intermediate samples, feature enhancement is performed on the ID matching samples to obtain feature-enhanced samples. These feature-enhanced samples are then input into the longitudinal federated residual enhancement model to perform model prediction, generating a second-party model prediction result. The specific implementation method for feature enhancement of the ID matching samples can be found in step C21, and will not be repeated here.

[0196] Step Y30: The second-party model prediction result and the second-party model weights corresponding to the longitudinal federated residual enhancement model are sent to the first device, so that the first device can generate a target federated prediction result based on the first-party model prediction result generated by the target prediction model for the sample to be predicted corresponding to the ID matching sample, the first-party model weights corresponding to the target prediction model, the second-party model prediction result, and the second-party model weights. The target prediction model is obtained by the first device through local iterative training.

[0197] In this embodiment, it should be noted that the specific process by which the first device generates the first-party model prediction result for the sample to be predicted corresponding to the training sample ID matching sample based on the target prediction model can be referred to the specific steps in step S10, and will not be repeated here.

[0198] Specifically, the second-party model weights corresponding to the longitudinal federated residual enhancement model are obtained, and the prediction results and weights of the second-party model are sent to the first device. The first device then performs weighted aggregation on the prediction results of the first-party model and the second-party model based on the first-party model weights and the second-party model weights corresponding to the target prediction model, according to a preset aggregation rule, to obtain the target federated prediction result. This achieves higher accuracy residual enhancement information based on the second device, optimizes the first-party model prediction results of the samples to be predicted in the first device, and improves the accuracy of the first device's sample prediction results for the samples to be predicted.

[0199] This application provides a method for optimizing vertical federated learning modeling. Specifically, it involves obtaining initial second-party model weights and receiving intermediate training sample features, training sample labels, and a first-party model prediction loss from a first device. The first-party model prediction loss is calculated by the first device based on the first-party model prediction result of the target prediction model on the training sample corresponding to the training sample with the matching training sample ID, and the training sample label. The intermediate training sample features are obtained by the first device through feature extraction on the training sample using the feature extractor of the target prediction model. This process yields training sample ID matching samples. Based on the residual boosting model to be trained, model prediction is performed jointly on the training sample ID matching samples and the intermediate training sample features to obtain the second-party training model prediction result. Based on the training sample label, the second-party initial model weights, and the second-party training model prediction result, the second-party model prediction loss is calculated. Then, based on the residual loss generated by the first-party model prediction loss and the second-party model prediction loss, the residual boosting model to be trained is iteratively optimized to obtain the vertical federated residual boosting model. This method achieves the goal of leveraging the features of intermediate training samples from the first device to enhance the features of training sample ID matching samples corresponding to training samples in the second device, and jointly constructing a vertical federated residual enhancement model based on residual learning using the first device. Based on this model, for aligned samples, the second device can generate more accurate residual enhancement information to optimize the first-party model prediction results generated by the first device, thereby generating a target federated prediction result with higher sample prediction accuracy. This lays the foundation for overcoming the technical limitation that while the predictor can achieve higher accuracy in federated predictions for aligned samples, it cannot perform sample predictions based on the complete model for unaligned samples, resulting in lower overall sample prediction accuracy.

[0200] Reference Figure 5 , Figure 5 This is a schematic diagram of the device system of the hardware operating environment involved in the embodiments of this application.

[0201] like Figure 5 As shown, the vertical federated prediction optimization device may include: a processor 1001, such as a CPU, a memory 1005, and a communication bus 1002. The communication bus 1002 is used to establish communication between the processor 1001 and the memory 1005. The memory 1005 may be high-speed RAM or stable non-volatile memory, such as disk storage. Optionally, the memory 1005 may also be a storage device independent of the aforementioned processor 1001.

[0202] Optionally, the vertical federated prediction optimization device may also include a rectangular user interface, a network interface, a camera, RF (Radio Frequency) circuitry, sensors, audio circuitry, a WiFi module, etc. The rectangular user interface may include a display screen and an input submodule such as a keyboard; optionally, the rectangular user interface may also include a standard wired interface or a wireless interface. The network interface may optionally include a standard wired interface or a wireless interface (such as a Wi-Fi interface).

[0203] Those skilled in the art will understand that Figure 5 The vertical federated forecasting optimization device system shown in the figure does not constitute a limitation on the vertical federated forecasting optimization device, which may include more or fewer components than shown, or combine certain components, or have different component arrangements.

[0204] like Figure 5 As shown, the memory 1005, as a computer storage medium, may include an operating system, a network communication module, and a vertical federated prediction optimization program. The operating system is a program that manages and controls the hardware and software resources of the vertical federated prediction optimization device, supporting the operation of the vertical federated prediction optimization program and other software and / or programs. The network communication module is used to enable communication between the various components within the memory 1005, as well as communication with other hardware and software in the vertical federated prediction optimization system.

[0205] exist Figure 5 In the vertical federated prediction optimization device shown, the processor 1001 is used to execute the vertical federated prediction optimization program stored in the memory 1005 to implement the steps of the vertical federated prediction optimization method described above.

[0206] The specific implementation of the vertical federated prediction optimization device in this application is basically the same as the embodiments of the above-described vertical federated prediction optimization method, and will not be repeated here.

[0207] Reference Figure 6 , Figure 6 This is a schematic diagram of the device system of the hardware operating environment involved in the embodiments of this application.

[0208] like Figure 6 As shown, the vertical federated learning modeling optimization device may include: a processor 1001, such as a CPU, a memory 1005, and a communication bus 1002. The communication bus 1002 is used to establish communication between the processor 1001 and the memory 1005. The memory 1005 may be a high-speed RAM or a stable, non-volatile memory, such as a disk drive. Optionally, the memory 1005 may also be a storage device independent of the aforementioned processor 1001.

[0209] Optionally, the longitudinal federated learning modeling optimization device may also include a rectangular user interface, a network interface, a camera, RF (Radio Frequency) circuitry, sensors, audio circuitry, a WiFi module, etc. The rectangular user interface may include a display screen and an input submodule such as a keyboard; optionally, the rectangular user interface may also include a standard wired interface or a wireless interface. The network interface may optionally include a standard wired interface or a wireless interface (such as a Wi-Fi interface).

[0210] Those skilled in the art will understand that Figure 6 The longitudinal federated learning modeling optimization apparatus system shown in the figure does not constitute a limitation on the longitudinal federated learning modeling optimization apparatus, and may include more or fewer components than shown, or combine certain components, or have different component arrangements.

[0211] like Figure 6 As shown, the memory 1005, which serves as a computer storage medium, may include an operating system, a network communication module, and a vertical federated learning modeling optimization program. The operating system is a program that manages and controls the hardware and software resources of the vertical federated learning modeling optimization device, supporting the operation of the vertical federated learning modeling optimization program and other software and / or programs. The network communication module is used to enable communication between the various components within the memory 1005, as well as communication with other hardware and software in the vertical federated learning modeling optimization system.

[0212] exist Figure 6 In the vertical federated learning modeling optimization device shown, the processor 1001 is used to execute the vertical federated learning modeling optimization program stored in the memory 1005 to implement the steps of the vertical federated learning modeling optimization method described above.

[0213] The specific implementation of the vertical federated learning modeling optimization device in this application is basically the same as the embodiments of the above-mentioned vertical federated learning modeling optimization method, and will not be repeated here.

[0214] This application embodiment also provides a vertical federated prediction optimization apparatus, which is applied to a first device, and the vertical federated prediction optimization apparatus includes:

[0215] The model prediction module is used to extract the sample to be predicted, and obtain the intermediate sample features generated by the feature extractor of the target prediction model for the sample to be predicted, and the first-party model prediction result generated by the target prediction model for the sample to be predicted, wherein the target prediction model is obtained by the first device through local iterative training.

[0216] The sending module is used to send the intermediate sample features to the second device, so that the second device can perform model prediction on the intermediate sample features and the ID matching samples corresponding to the sample to be predicted based on the vertical federated residual boosting model, and obtain the second-party model prediction result. The vertical federated residual boosting model is obtained by the second device performing residual learning based on vertical federated common samples, combining the model prediction loss of the target prediction model in the first device on the vertical federated common samples, the intermediate common sample features corresponding to the vertical federated common samples, and the corresponding sample labels, with the first device.

[0217] The receiving module is used to obtain the first-party model weights corresponding to the target prediction model, and to receive the second-party model prediction results and the second-party model weights corresponding to the longitudinal federated residual improvement model sent by the second device.

[0218] The weighted aggregation module is used to perform weighted aggregation on the prediction results of the first model and the prediction results of the second model based on the weights of the first model and the second model to obtain the target federated prediction result.

[0219] Optionally, the vertical federated prediction optimization device is further used for:

[0220] The ID of the sample to be predicted corresponding to the sample to be predicted is sent to the second device so that the second device can find the ID matching sample corresponding to the sample ID to be predicted;

[0221] If a search failure message is received from the second device, the prediction result of the first-party model is used as the target prediction result.

[0222] If no search failure message is received from the second device, then the following step is performed: send the intermediate sample features to the second device.

[0223] Optionally, the vertical federated prediction optimization device is further used for:

[0224] Obtain the initial weights of the first-party model, and extract the training samples and the corresponding training sample labels;

[0225] Obtain intermediate training sample features generated by the feature extractor of the target prediction model to be trained, based on the training samples.

[0226] Based on the training sample labels, the training model prediction results corresponding to the intermediate training sample features, and the first-party initial model weights, the target prediction model is obtained by iteratively optimizing the target prediction model to be trained by calculating the first-party model prediction loss corresponding to the target prediction model to be trained.

[0227] The training sample labels, intermediate training sample features, and the first-party model prediction loss are sent to the second device, so that the second device can calculate the second-party model prediction loss based on the residual boosting model to be trained, the training sample ID matching sample corresponding to the training sample, the intermediate training sample features, the sample labels, and the obtained second-party initial model weights. Based on the residual loss calculated by the second-party model prediction loss and the first-party model prediction loss, the second device optimizes the residual boosting model to be trained to obtain the vertical federated residual boosting model.

[0228] The specific implementation of the vertical federated prediction optimization device in this application is basically the same as the embodiments of the vertical federated prediction optimization method described above, and will not be repeated here.

[0229] This application embodiment also provides a vertical federated prediction optimization apparatus, which is applied to a second device, and the vertical federated prediction optimization apparatus includes:

[0230] The receiving and searching module is used to receive intermediate sample features sent by the first device and search for samples whose IDs match the intermediate sample features.

[0231] The model prediction module is used to perform model prediction on the ID matching samples and the intermediate sample features based on the vertical federated residual boosting model to obtain the second-party model prediction result. The vertical federated residual boosting model is obtained by the second device through residual learning based on vertical federated common samples, combined with the model prediction loss of the target prediction model in the first device on the vertical federated common samples, the intermediate common sample features corresponding to the vertical federated common samples, and the corresponding sample labels, and the first device.

[0232] The sending module is used to obtain the second-party model weights corresponding to the longitudinal federated residual enhancement model, and send the second-party model prediction results and the second-party model weights to the first device, so that the first device can generate a target federated prediction result based on the first-party model prediction results generated by the target prediction model for the sample to be predicted corresponding to the ID matching sample, the first-party model weights corresponding to the target prediction model, the second-party model prediction results and the second-party model weights, wherein the target prediction model is obtained by the first device through local iterative training.

[0233] Optionally, the model prediction module is further configured to:

[0234] The features of the ID matching sample and the intermediate sample are concatenated to obtain the feature-enhanced sample;

[0235] Based on the aforementioned longitudinal federated residual enhancement model, model prediction is performed on the feature-enhanced samples to obtain the prediction results of the second-party model.

[0236] Optionally, the vertical federated prediction optimization device is further used for:

[0237] If the search is successful, proceed to the following steps: Based on the longitudinal federated residual boosting model, perform model prediction on the features of the ID matching sample and the intermediate sample to obtain the second-party model prediction result;

[0238] If the search fails, a search failure message is sent back to the first device, so that the first device, after receiving the search failure message, can use the first-party model prediction result generated by the target prediction model for the sample to be predicted as the target prediction result.

[0239] Optionally, the vertical federated prediction optimization device is further used for:

[0240] The system obtains the initial weights of the second-party model and receives the intermediate training sample features, training sample labels, and first-party model prediction loss sent by the first device. The first-party model prediction loss is calculated by the first device based on the first-party model prediction result of the target prediction model on the training sample corresponding to the training sample ID and the training sample label. The intermediate training sample features are obtained by the first device based on the feature extractor of the target prediction model for feature extraction of the training sample.

[0241] Obtain training sample ID matching samples, and improve the model based on the residual to be trained. Perform model prediction on the training sample ID matching samples and the features of the intermediate training samples to obtain the prediction result of the second-party training model.

[0242] Based on the training sample labels, the second-party initial model weights, and the prediction results of the second-party trained model, the prediction loss of the second-party model is calculated.

[0243] Based on the residual loss generated by the prediction loss of the first-party model and the prediction loss of the second-party model, the residual boosting model to be trained is iteratively optimized to obtain the longitudinal federated residual boosting model.

[0244] The specific implementation of the vertical federated prediction optimization device in this application is basically the same as the embodiments of the vertical federated prediction optimization method described above, and will not be repeated here.

[0245] This application embodiment also provides a vertical federated learning modeling optimization device, which is applied to a first device, and the vertical federated learning modeling optimization device includes:

[0246] The first acquisition module is used to acquire the initial model weights of the first party and extract the training samples and the training sample labels corresponding to the training samples;

[0247] The second acquisition module is used to acquire the intermediate training sample features generated by the feature extractor of the target prediction model to be trained for the training samples.

[0248] The iterative optimization module is used to iteratively optimize the target prediction model to be trained by calculating the first-party model prediction loss corresponding to the target prediction model to be trained, based on the training sample labels, the training model prediction results corresponding to the intermediate training sample features, and the first-party initial model weights, to obtain the target prediction model.

[0249] The sending module is used to send the training sample labels, the intermediate training sample features, and the first-party model prediction loss to the second device, so that the second device can calculate the second-party model prediction loss and optimize the training residual enhancement model based on the residual loss calculated from the second-party model prediction loss and the first-party model prediction loss, thereby obtaining a vertical federated residual enhancement model. The second-party model prediction loss is calculated based on the training residual enhancement model to be trained, the training sample ID matching sample corresponding to the training sample, the intermediate training sample features, the sample labels, and the obtained second-party initial model weights.

[0250] Optionally, the iterative optimization module is further configured to:

[0251] Based on the classifier in the target prediction model to be trained, the features of the intermediate training samples are converted into prediction results of the training model;

[0252] Based on the training sample labels, the training model prediction results, and the first-party initial model weights, calculate the first-party model prediction loss;

[0253] Based on the prediction results of the training model and the labels of the training samples, update the weights of the first-party initial model;

[0254] Based on the prediction loss of the first-party model and the updated weights of the first-party initial model, the target prediction model to be trained is iteratively optimized to obtain the target prediction model.

[0255] Optionally, the longitudinal federated learning modeling optimization device is further used for:

[0256] Obtain the number of correctly classified first-party samples and the number of incorrectly classified first-party samples corresponding to the target prediction model;

[0257] The first-party model weights are generated by calculating the ratio of the number of correctly classified samples to the number of incorrectly classified samples.

[0258] Optionally, the longitudinal federated learning modeling optimization device is further used for:

[0259] Extract the sample to be predicted, and obtain the intermediate sample features generated by the feature extractor of the target prediction model for the sample to be predicted, and the first-party model prediction result generated by the target prediction model for the sample to be predicted.

[0260] The intermediate sample features are sent to the second device, which then performs model prediction on the intermediate sample features and the ID matching samples corresponding to the sample to be predicted based on the longitudinal federated residual boosting model, and obtains the second-party model prediction result.

[0261] Receive the second-party model prediction results and the second-party model weights corresponding to the longitudinal federated residual enhancement model sent by the second device;

[0262] Based on the first-party model weights and the second-party model weights corresponding to the target prediction model, the prediction results of the first-party model and the prediction results of the second-party model are weighted and aggregated to obtain the target federated prediction result.

[0263] The specific implementation of the vertical federated learning modeling optimization device in this application is basically the same as the embodiments of the above-mentioned vertical federated learning modeling optimization method, and will not be repeated here.

[0264] This application embodiment also provides a vertical federated learning modeling optimization device, which is applied to a second device, and the vertical federated learning modeling optimization device includes:

[0265] The receiving module is used to obtain the initial weights of the second-party model and receive the intermediate training sample features, training sample labels, and first-party model prediction loss sent by the first device. The first-party model prediction loss is calculated by the first device based on the first-party model prediction result of the target prediction model on the training sample corresponding to the training sample ID and the training sample label. The intermediate training sample features are obtained by the first device through feature extraction of the training samples based on the feature extractor of the target prediction model.

[0266] The model prediction module is used to obtain training sample ID matching samples, and improve the model based on the residual to be trained, and jointly perform model prediction on the training sample ID matching samples and the features of the intermediate training samples to obtain the prediction result of the second-party training model.

[0267] The calculation module is used to calculate the prediction loss of the second-party model based on the training sample labels, the weights of the second-party initial model, and the prediction results of the second-party trained model.

[0268] The iterative optimization module is used to iteratively optimize the training residual boosting model based on the residual loss generated by the prediction loss of the first-party model and the prediction loss of the second-party model, so as to obtain the longitudinal federated residual boosting model.

[0269] Optionally, the longitudinal federated learning modeling optimization device is further used for:

[0270] Obtain the number of correctly classified samples and the number of incorrectly classified samples corresponding to the longitudinal federated residual boosting model;

[0271] The second-party model weights are generated by calculating the ratio of the number of correctly classified samples to the number of incorrectly classified samples.

[0272] Optionally, the longitudinal federated learning modeling optimization device is further used for:

[0273] Receive intermediate sample features sent by the first device, and search for samples whose IDs match the intermediate sample features;

[0274] Based on the vertical federated residual enhancement model, the model prediction is jointly performed on the features of the ID matching samples and the intermediate samples to obtain the second-party model prediction results.

[0275] The second-party model prediction result and the second-party model weights corresponding to the longitudinal federated residual enhancement model are sent to the first device so that the first device can generate a target federated prediction result based on the first-party model prediction result generated by the target prediction model for the sample to be predicted corresponding to the ID matching sample, the first-party model weights corresponding to the target prediction model, the second-party model prediction result, and the second-party model weights. The target prediction model is obtained by the first device through local iterative training.

[0276] The specific implementation of the vertical federated learning modeling optimization device in this application is basically the same as the embodiments of the above-mentioned vertical federated learning modeling optimization method, and will not be repeated here.

[0277] This application provides a medium that is a readable storage medium, and the readable storage medium stores one or more programs, which can be executed by one or more processors to implement the steps of the vertical federated prediction optimization method described in any of the above claims.

[0278] The specific implementation of the readable storage medium in this application is basically the same as the embodiments of the above-described vertical federated prediction optimization method, and will not be repeated here.

[0279] This application provides a medium that is a readable storage medium, and the readable storage medium stores one or more programs, which can be executed by one or more processors to implement the steps of the longitudinal federated learning modeling optimization method described in any of the above claims.

[0280] The specific implementation of the readable storage medium in this application is basically the same as the embodiments of the above-described vertical federated learning modeling optimization method, and will not be repeated here.

[0281] This application provides a computer program product, which includes one or more computer programs. The one or more computer programs can be executed by one or more processors to implement the steps of the vertical federated prediction optimization method described in any of the above claims.

[0282] The specific implementation of the computer program product in this application is basically the same as the embodiments of the above-described vertical federated prediction optimization method, and will not be repeated here.

[0283] This application provides a computer program product, which includes one or more computer programs. The one or more computer programs can be executed by one or more processors to implement the steps of the longitudinal federated learning modeling optimization method described in any of the above claims.

[0284] The specific implementation of the computer program product in this application is basically the same as the embodiments of the above-described vertical federated learning modeling and optimization method, and will not be repeated here.

[0285] The above are merely preferred embodiments of this application and do not limit the patent scope of this application. Any equivalent structural or procedural transformations made using the content of this application's specification and drawings, or direct or indirect applications in other related technical fields, are similarly included within the patent scope of this application.

Claims

1. A vertical federated prediction optimization method, characterized in that, Applied to the first device, the longitudinal federated prediction optimization method includes: Extract the sample to be predicted, and obtain the intermediate sample features generated by the feature extractor of the target prediction model for the sample to be predicted, and the first-party model prediction result generated by the target prediction model for the sample to be predicted, wherein the target prediction model is obtained by local iterative training of the first device; The intermediate sample features are sent to the second device, which then performs model prediction on the intermediate sample features and the ID matching samples corresponding to the sample to be predicted based on the longitudinal federated residual boosting model, and obtains the second-party model prediction result. Obtain the first-party model weights corresponding to the target prediction model, and receive the second-party model prediction results and the second-party model weights corresponding to the longitudinal federated residual improvement model sent by the second device; Based on the weights of the first-party model and the weights of the second-party model, the prediction results of the first-party model and the prediction results of the second-party model are weighted and aggregated to obtain the target federated prediction result; The vertical federated residual enhancement model is obtained by the second device based on the vertical federated common samples, combined with the model prediction loss of the target prediction model in the first device on the vertical federated common samples, the intermediate common sample features corresponding to the vertical federated common samples, and the corresponding sample labels, and by performing residual learning based on vertical federated learning with the first device. Before the steps of obtaining intermediate sample features generated by the feature extractor of the target prediction model for the sample to be predicted, and the first-party model prediction result generated by the target prediction model for the sample to be predicted, the vertical federated prediction optimization method further includes: Obtain the initial weights of the first-party model, and extract the training samples and the corresponding training sample labels; Obtain intermediate training sample features generated by the feature extractor of the target prediction model to be trained by extracting features from the training samples; Based on the training sample labels, the training model prediction results corresponding to the intermediate training sample features, and the first-party initial model weights, the target prediction model is obtained by iteratively optimizing the target prediction model to be trained by calculating the first-party model prediction loss corresponding to the target prediction model to be trained. The training sample labels, intermediate training sample features, and the first-party model prediction loss are sent to the second device, so that the second device can calculate the second-party model prediction loss based on the residual boosting model to be trained, the training sample ID matching sample corresponding to the training sample, the intermediate training sample features, the sample labels, and the obtained second-party initial model weights. Based on the residual loss calculated by the second-party model prediction loss and the first-party model prediction loss, the second device optimizes the residual boosting model to be trained to obtain the vertical federated residual boosting model.

2. The vertical federated prediction optimization method as described in claim 1, characterized in that, Prior to the step of sending the intermediate sample features to the second device, the longitudinal federated prediction optimization method further includes: The ID of the sample to be predicted corresponding to the sample to be predicted is sent to the second device so that the second device can find the ID matching sample corresponding to the sample ID to be predicted; If a search failure message is received from the second device, the prediction result of the first-party model is used as the target prediction result. If no search failure message is received from the second device, then the following step is performed: send the intermediate sample features to the second device.

3. A vertical federated prediction optimization method, characterized in that, Applied to the second device, the longitudinal federated prediction optimization method includes: Receive intermediate sample features sent by the first device, and search for samples whose IDs match the intermediate sample features; Based on the longitudinal federated residual enhancement model, the model prediction is jointly performed on the features of the ID matching samples and the intermediate samples to obtain the second-party model prediction results. Obtain the second-party model weights corresponding to the longitudinal federated residual enhancement model, and send the second-party model prediction results and the second-party model weights to the first device, so that the first device can generate a target federated prediction result based on the first-party model prediction results generated by the target prediction model for the sample to be predicted corresponding to the ID matching sample, the first-party model weights corresponding to the target prediction model, the second-party model prediction results and the second-party model weights, wherein the target prediction model is obtained by the first device through local iterative training; The vertical federated residual enhancement model is obtained by the second device based on the vertical federated common samples, combined with the model prediction loss of the target prediction model in the first device on the vertical federated common samples, the intermediate common sample features corresponding to the vertical federated common samples, and the corresponding sample labels, and by performing residual learning based on vertical federated learning with the first device. Before the step of performing model prediction on the features of the ID matching samples and the intermediate samples based on the vertical federated residual enhancement model to obtain the second-party model prediction result, the vertical federated prediction optimization method further includes: The system obtains the initial weights of the second-party model and receives the intermediate training sample features, training sample labels, and first-party model prediction loss sent by the first device. The first-party model prediction loss is calculated by the first device based on the first-party model prediction result of the target prediction model on the training sample corresponding to the training sample ID and the training sample label. The intermediate training sample features are obtained by the first device based on the feature extractor of the target prediction model for feature extraction of the training sample. Obtain training sample ID matching samples, and improve the model based on the residual to be trained. Perform model prediction on the training sample ID matching samples and the features of the intermediate training samples to obtain the prediction result of the second-party training model. Based on the training sample labels, the second-party initial model weights, and the prediction results of the second-party trained model, the prediction loss of the second-party model is calculated. Based on the residual loss generated by the prediction loss of the first-party model and the prediction loss of the second-party model, the residual boosting model to be trained is iteratively optimized to obtain the longitudinal federated residual boosting model.

4. The vertical federated prediction optimization method as described in claim 3, characterized in that, The step of performing model prediction on the features of the ID matching samples and the intermediate samples based on the longitudinal federated residual boosting model to obtain the second-party model prediction result includes: The features of the ID matching sample and the intermediate sample are concatenated to obtain the feature-enhanced sample; Based on the aforementioned longitudinal federated residual enhancement model, model prediction is performed on the feature-enhanced samples to obtain the prediction results of the second-party model.

5. The vertical federated prediction optimization method as described in claim 3, characterized in that, Following the step of finding ID-matching samples, the vertical federated prediction optimization further includes: If the search is successful, proceed to the following steps: Based on the longitudinal federated residual boosting model, perform model prediction on the features of the ID matching sample and the intermediate sample to obtain the second-party model prediction result; If the search fails, a search failure message is sent back to the first device, so that the first device, after receiving the search failure message, can use the first-party model prediction result generated by the target prediction model for the sample to be predicted as the target prediction result.

6. A longitudinal federated learning modeling optimization method, characterized in that, Applied to the first device, the longitudinal federated learning modeling optimization method includes: Obtain the initial weights of the first-party model, and extract the training samples and the corresponding training sample labels; Obtain intermediate training sample features generated by the feature extractor of the target prediction model to be trained by extracting features from the training samples; Based on the training sample labels, the training model prediction results corresponding to the intermediate training sample features, and the first-party initial model weights, the target prediction model is obtained by iteratively optimizing the target prediction model to be trained by calculating the first-party model prediction loss corresponding to the target prediction model to be trained. The training sample labels, intermediate training sample features, and the first-party model prediction loss are sent to the second device so that the second device can calculate the second-party model prediction loss and optimize the training residual enhancement model based on the residual loss calculated by the second-party model prediction loss and the first-party model prediction loss to obtain the vertical federated residual enhancement model. The second-party model prediction loss is calculated based on the residual improvement model to be trained, the training sample ID matching sample corresponding to the training sample, the intermediate training sample features, the sample label, and the obtained second-party initial model weights. The step of obtaining the target prediction model by calculating the first-party model prediction loss corresponding to the target prediction model to be trained, based on the training sample labels, the training model prediction results corresponding to the intermediate training sample features, and the first-party initial model weights, includes: Based on the classifier in the target prediction model to be trained, the features of the intermediate training samples are converted into prediction results of the training model; Based on the training sample labels, the training model prediction results, and the first-party initial model weights, calculate the first-party model prediction loss; Based on the prediction results of the training model and the labels of the training samples, update the weights of the first-party initial model; Based on the prediction loss of the first-party model and the updated weights of the first-party initial model, the target prediction model to be trained is iteratively optimized to obtain the target prediction model.

7. The vertical federated learning modeling optimization method as described in claim 6, characterized in that, After the step of iteratively optimizing the target prediction model by calculating the first-party model prediction loss corresponding to the target prediction model to be trained based on the training sample labels, the training sample features corresponding to the intermediate training sample features, and the first-party initial model weights, to obtain the target prediction model, the vertical federated learning modeling optimization method further includes: Obtain the number of correctly classified first-party samples and the number of incorrectly classified first-party samples corresponding to the target prediction model; The first-party model weights are generated by calculating the ratio of the number of correctly classified samples to the number of incorrectly classified samples.

8. The longitudinal federated learning modeling optimization method as described in claim 6, characterized in that, After the steps of sending the training sample labels, the intermediate training sample features, and the first-party model prediction loss to the second device for the second device to calculate the second-party model prediction loss, and optimizing the residual boosting model to be trained based on the residual loss calculated by the second-party model prediction loss and the first-party model prediction loss to obtain the vertical federated residual boosting model, the vertical federated learning modeling optimization method further includes: Extract the sample to be predicted, and obtain the intermediate sample features generated by the feature extractor of the target prediction model for the sample to be predicted, and the first-party model prediction result generated by the target prediction model for the sample to be predicted. The intermediate sample features are sent to the second device, which then performs model prediction on the intermediate sample features and the ID matching samples corresponding to the sample to be predicted based on the longitudinal federated residual boosting model, and obtains the second-party model prediction result. Receive the second-party model prediction results and the second-party model weights corresponding to the longitudinal federated residual enhancement model sent by the second device; Based on the first-party model weights and the second-party model weights corresponding to the target prediction model, the prediction results of the first-party model and the prediction results of the second-party model are weighted and aggregated to obtain the target federated prediction result.

9. A longitudinal federated learning modeling optimization method, characterized in that, Applied to a second device, the longitudinal federated learning modeling optimization method includes: Obtain the initial weights of the second-party model, and receive the intermediate training sample features, training sample labels, and first-party model prediction loss sent by the first device; Obtain training sample ID matching samples, and improve the model based on the residual to be trained. Perform model prediction on the training sample ID matching samples and the features of the intermediate training samples to obtain the prediction result of the second-party training model. Based on the training sample labels, the second-party initial model weights, and the prediction results of the second-party trained model, the prediction loss of the second-party model is calculated. Based on the residual loss generated by the first-party model prediction loss and the second-party model prediction loss, the training residual enhancement model is iteratively optimized to obtain the vertical federated residual enhancement model; Wherein, the first-party model prediction loss is calculated by the first device based on the first-party model prediction result on the training sample corresponding to the training sample ID matching sample based on the target prediction model and the training sample label, and the intermediate training sample features are obtained by the first device based on the feature extractor of the target prediction model for the training sample; After the step of iteratively optimizing the trainable residual boosting model based on the residual loss generated by the first-party model prediction loss and the second-party model prediction loss to obtain the vertical federated residual boosting model, the vertical federated learning modeling optimization method further includes: Obtain the number of correctly classified samples and the number of incorrectly classified samples corresponding to the longitudinal federated residual boosting model; The second-party model weights are generated by calculating the ratio of the number of correctly classified samples to the number of incorrectly classified samples.

10. The longitudinal federated learning modeling optimization method as described in claim 9, characterized in that, After the step of iteratively optimizing the trainable residual boosting model based on the residual loss to obtain the vertical federated residual boosting model, the vertical federated learning modeling optimization method further includes: Receive intermediate sample features sent by the first device, and search for samples whose IDs match the intermediate sample features; Based on the vertical federated residual enhancement model, the model prediction is jointly performed on the features of the ID matching samples and the intermediate samples to obtain the second-party model prediction results. The second-party model prediction result and the second-party model weights corresponding to the longitudinal federated residual enhancement model are sent to the first device so that the first device can generate a target federated prediction result based on the first-party model prediction result generated by the target prediction model for the sample to be predicted corresponding to the ID matching sample, the first-party model weights corresponding to the target prediction model, the second-party model prediction result, and the second-party model weights. The target prediction model is obtained by the first device through local iterative training.

11. A vertical federated prediction optimization device, characterized in that, The vertical federated prediction optimization device includes: a memory, a processor, and a program stored in the memory for implementing the vertical federated prediction optimization method. The memory is used to store programs that implement the vertical federated prediction optimization method; The processor is configured to execute a program that implements the vertical federated prediction optimization method to implement the steps of the vertical federated prediction optimization method as described in any one of claims 1 to 5.

12. A longitudinal federated learning modeling optimization device, characterized in that, The vertical federated learning modeling optimization device includes: a memory, a processor, and a program stored in the memory for implementing the vertical federated learning modeling optimization method. The memory is used to store programs that implement the longitudinal federated learning modeling optimization method; The processor is configured to execute a program that implements the longitudinal federated learning modeling optimization method, thereby implementing the steps of the longitudinal federated learning modeling optimization method as described in any one of claims 6 to 10.

13. A medium, said medium being a readable storage medium, characterized in that, The readable storage medium stores a program implementing the vertical federated prediction optimization method, which is executed by a processor to implement the steps of the vertical federated prediction optimization method as described in any one of claims 1 to 5.

14. A medium, said medium being a readable storage medium, characterized in that, The readable storage medium stores a program that implements the longitudinal federated learning modeling optimization method, which is executed by a processor to implement the steps of the longitudinal federated learning modeling optimization method as described in any one of claims 6 to 10.

15. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the steps of the longitudinal federated prediction optimization method as described in any one of claims 1 to 5, or the steps of the longitudinal federated learning modeling optimization method as described in any one of claims 6 to 10.

Images