Federal xgboost algorithm prediction method and device for low-bandwidth scenarios

Abstract

Embodiments of the present application disclose a low-bandwidth scene federated xgboost algorithm prediction method and device, applied to a multi-party computing system, the multi-party computing system comprising P computing parties and a model owner party of an xgboost model, P being an integer greater than 1; the method comprises: each of the P computing parties performing prediction based on respective original data and the xgboost model, obtaining P prediction results, each prediction result comprising a sparse prediction matrix and a prediction vector; the model owner party restoring the P prediction results into dense prediction vectors, obtaining P dense prediction vectors; the model owner party intersecting the P dense prediction vectors, obtaining a final prediction vector, and determining a target prediction value according to the final prediction vector. The embodiments of the present application can improve the prediction efficiency of the federated xgboost algorithm in a low-bandwidth environment.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of privacy computing or the technical field of computer, in particular to a federated xgboost algorithm prediction method in a low-bandwidth scenario and a related device. BACKGROUND

[0002] The emergence of federated learning technology breaks the concept that modeling personnel must analyze and model data centrally, providing a cross-domain distributed computing and modeling idea.

[0003] The existing federated XGBOOST prediction scheme needs to communicate multiple times between multiple computing nodes. In the prediction process, when encountering an intermediate tree node that is not a feature value of the local party, the local prediction needs to be stopped. After removing the samples that have reached the intermediate node, the tree node numbers corresponding to the remaining samples are sent to the next computing party for prediction. Therefore, when the feature values used by the intermediate nodes are uniformly distributed among different computing parties, almost every intermediate tree node needs to be predicted. Moreover, the xgboost model is generally integrated by multiple trees, and the number of communications and the cumulative communication volume are very large, causing waste of computing resources and communication resources, which cannot meet the needs of a low-bandwidth scenario. Therefore, how to improve the prediction efficiency of the federated xgboost algorithm in a low-bandwidth environment is an urgent problem to be solved. SUMMARY

[0004] The embodiments of the present application provide a federated xgboost algorithm prediction method and device in a low-bandwidth scenario, which can improve the prediction efficiency of the federated xgboost algorithm in a low-bandwidth environment.

[0005] In a first aspect, the embodiments of the present application provide a federated xgboost algorithm prediction method in a low-bandwidth scenario, applied to a multi-party computing system, the multi-party computing system including P computing parties and a model owner party of an xgboost model, P being an integer greater than 1; the method comprising:

[0006] predicting, by each of the P computing parties, based on respective original data and the xgboost model, to obtain P prediction results, each prediction result including a sparse prediction matrix and a prediction vector;

[0007] restoring, by the model owner party, the P prediction results into dense prediction vectors to obtain P dense prediction vectors;

[0008] intersecting, by the model owner party, the P dense prediction vectors to obtain a final prediction vector, and determining a target prediction value according to the final prediction vector.

[0009] In a second aspect, an embodiment of the present application provides a federated xgboost algorithm prediction device in a low-bandwidth scenario, applied to a multi-party computing system, the multi-party computing system including P computing parties and a model owner party of an xgboost model, P being an integer greater than 1; the device includes a prediction unit, a recovery unit, and a determination unit, wherein

[0010] The prediction unit is configured to perform prediction by each of the P computing parties based on respective original data and the xgboost model, to obtain P prediction results, each prediction result including a sparse prediction matrix and a prediction vector.

[0011] The recovery unit is configured to recover the P prediction results into dense prediction vectors by the model owner party, to obtain P dense prediction vectors.

[0012] The determination unit is configured to intersect the P dense prediction vectors by the model owner party, to obtain a final prediction vector, and to determine a target prediction value according to the final prediction vector.

[0013] In a third aspect, an embodiment of the present application provides an electronic device, including a processor, a memory, a communication interface, and one or more programs, wherein the one or more programs are stored in the memory and configured to be executed by the processor, and the programs include instructions for performing the steps in the first aspect of the embodiments of the present application.

[0014] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program for electronic data exchange, and the computer program causes a computer to perform some or all of the steps described in the first aspect of the embodiments of the present application.

[0015] In a fifth aspect, an embodiment of the present application provides a computer program product, which includes a non-transitory computer-readable storage medium storing a computer program, and the computer program is operable to cause a computer to perform some or all of the steps described in the first aspect of the embodiments of the present application. The computer program product can be a software installation package.

[0016] By implementing the embodiments of the present application, the following beneficial effects are achieved:

[0017] It can be seen that the federated xgboost algorithm prediction method and device in the low-bandwidth scenario described in the embodiments of the present application are applied to a multi-party computing system, the multi-party computing system includes P computing parties and a model owner party of an xgboost model, P is an integer greater than 1, each computing party in the P computing parties performs prediction based on respective original data and the xgboost model, P prediction results are obtained, each prediction result includes a sparse prediction matrix and a prediction vector, the model owner party restores the P prediction results into dense prediction vectors to obtain P dense prediction vectors, the model owner party intersects the P dense prediction vectors to obtain a final prediction vector, and a target prediction value is determined according to the final prediction vector. Not only does it not need to communicate in the process of traversing the whole tree node, but also makes the overall prediction matrix sparse, which can reduce the communication amount and thus reduce the overall prediction time, while meeting the low-bandwidth communication scenario, and thus the prediction efficiency of the federated xgboost algorithm in the low-bandwidth environment can be improved. BRIEF DESCRIPTION OF DRAWINGS

[0018] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the drawings needed to be used in the embodiments or the prior art description will be briefly introduced. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can be obtained by those skilled in the art without creative labor.

[0019] Figure 1 is an architecture schematic diagram of a multi-party computing system for implementing a federated xgboost algorithm prediction method in a low-bandwidth scenario provided by the embodiments of the present application;

[0020] Figure 2 is a flowchart of a federated xgboost algorithm prediction method in a low-bandwidth scenario provided by the embodiments of the present application;

[0021] Figure 3 is a flowchart of another federated xgboost algorithm prediction method in a low-bandwidth scenario provided by the embodiments of the present application;

[0022] Figure 4 is a structural schematic diagram of an electronic device provided by the embodiments of the present application;

[0023] Figure 5 is a functional unit composition block diagram of a federated xgboost algorithm prediction device in a low-bandwidth scenario provided by the embodiments of the present application. DETAILED DESCRIPTION

[0024] In the following, the technical solutions in the embodiments of the present application will be described clearly and completely in conjunction with the drawings in the embodiments of the present application, so that those skilled in the art can better understand the present application. Obviously, the described embodiments are only some of the embodiments of the present application, but not all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative work fall within the scope of the present application.

[0025] The terms "first", "second", and the like in the specification of the present application and the above drawings are used to distinguish different objects, and are not used to describe a particular order. In addition, the terms "include" and "have" and any variations thereof are intended to cover non-exclusive inclusion. For example, a process, method, system, product or device including a series of steps or units is not limited to the listed steps or units, but can optionally include other steps or units not listed or can optionally include other steps or units inherent to the process, method, product or device.

[0026] In this document, reference to "an embodiment" means that a particular feature, structure, or characteristic described in connection with the embodiment can be included in at least one embodiment of the present application. The appearance of the phrase in various places in the specification does not necessarily all refer to the same embodiment, nor does it necessarily refer to a particular embodiment that is independent of or alternative to other embodiments. It is explicitly and implicitly understood that the embodiments described herein can be combined with other embodiments.

[0027] The computing node described in the embodiments of the present application, i.e. the computing party or the model owner party, can be an electronic device, which can include a smartphone (such as an Android phone, an iOS phone, a Windows Phone, etc.), a tablet computer, a palm computer, a car recorder, a server, a notebook computer, a mobile Internet device (MID) or a wearable device (such as a smart watch, a Bluetooth headset), etc. The above are only examples and are not exhaustive, and the electronic device can include but is not limited to the above electronic devices. The electronic device can also be a cloud server, or the electronic device can also be a computer cluster. In the embodiments of the present application, the result party and the sending party can be the above electronic devices.

[0028] The embodiments of the present application will be described in detail below.

[0029] Please refer to Figure 1 , Figure 1An architecture diagram of a multi-party computing system for a federated xgboost algorithm prediction method in a low-bandwidth scenario is provided in embodiments of the present application, as shown in the figure, the multi-party computing system can include P computing parties and a model owner party of an xgboost model, P is an integer greater than 1; the following functions can be implemented based on the multi-party computing system:

[0030] Each of the P computing parties performs prediction based on respective raw data and the xgboost model to obtain P prediction results, each prediction result including a sparse prediction matrix and a prediction vector;

[0031] The model owner party restores the P prediction results to dense prediction vectors to obtain P dense prediction vectors;

[0032] The model owner party intersects the P dense prediction vectors to obtain a final prediction vector, and determines a target prediction value according to the final prediction vector.

[0033] Optionally, the step of each of the P computing parties performing prediction based on respective raw data and the xgboost model to obtain P prediction results includes:

[0034] The computing party i obtains raw data of the computing party i, the computing party i being any of the P computing parties;

[0035] The xgboost model is loaded in the computing party i, and the xgboost model is locally traversed to obtain a prediction result of the computing party i, the raw data including multiple samples, in the traversal process of any sample, when an intermediate tree node of a non-party-specific feature is encountered, the sample is allowed to access both left and right child nodes.

[0036] Optionally, the step of locally traversing the xgboost model to obtain a prediction result of the computing party i includes:

[0037] The xgboost model is locally traversed to obtain a preliminary prediction result of each sample in the raw data, and multiple preliminary prediction results are obtained;

[0038] Any of the multiple preliminary prediction results is randomly selected as a prediction flag vector;

[0039] Each of the multiple preliminary prediction results is subtracted from the prediction flag vector to obtain multiple intermediate prediction results;

[0040] A sparse prediction matrix is determined according to the multiple intermediate prediction results;

[0041] The sparse prediction matrix and the prediction flag vector are determined as the prediction result of the i-th computing party.

[0042] Optionally, the determining a target prediction value according to the final prediction vector comprises:

[0043] The final prediction vector is operated with each leaf node weight of the xgboost model to obtain the target prediction value.

[0044] Optionally, the following functions can also be implemented:

[0045] A target bandwidth parameter is obtained by the model owner party.

[0046] Each of the P computing parties performs prediction based on respective raw data and the xgboost model when the target bandwidth parameter is less than a preset threshold to obtain P prediction results.

[0047] To solve the defects in the related art, please refer to Figure 2 , Figure 2 is a flowchart of a low-bandwidth scenario federated xgboost algorithm prediction method provided by the present application; applied to a multi-party computing system as shown in Figure 1 The multi-party computing system includes P computing parties and a model owner party of an xgboost model, and P is an integer greater than 1; as shown in the figure, the low-bandwidth scenario federated xgboost algorithm prediction method includes:

[0048] 201, each of the P computing parties performs prediction based on respective raw data and the xgboost model to obtain P prediction results, and each prediction result includes a sparse prediction matrix and a prediction vector.

[0049] In a specific implementation, each computing party loads raw data to be predicted, each computing party locally calculates prediction, when the split feature of the intermediate node of the tree model adopted is provided by other parties, at this time, prediction is performed to the left and right child nodes (i.e., accessing the left and right child nodes), when the leaf node of the tree model is reached, prediction is stopped, and a Boolean value is marked to the reached node, and a group of Boolean vectors are obtained to represent which leaf nodes the sample can finally access. That is, the prediction vector can include a Boolean vector.

[0050] In the present application, each of the P computing parties can perform prediction based on respective raw data and the xgboost model to obtain P prediction results, and each prediction result includes a sparse prediction matrix and a prediction vector.

[0051] In the process of local prediction of the computing node, when an intermediate tree node of a non-party feature is encountered, the sample can be allowed to access both left and right child nodes first, so that the sample will access more than one leaf node, thereby generating a set of Boolean prediction vectors, and during the traversal of the entire tree node, no communication is required.

[0052] Of course, without sparse compression, each sample corresponding to a set of Boolean prediction vectors will result in a larger matrix, and sparse compression is required before being sent to the model owner, which can greatly reduce the communication amount of xgboost prediction in vertical federated learning. Since in the case of large data volume (the data volume is much larger than the number of leaf nodes), a large amount of data of the same tree falls on the same leaf node, the overall prediction matrix can be made sparse by subtracting the prediction vectors, which can reduce the communication amount, thereby reducing the overall prediction time, and at the same time, the low-bandwidth communication scenario can be met.

[0053] Optionally, the step 201 of obtaining P prediction results by each of the P computing parties based on respective original data and the xgboost model can include the following steps:

[0054] 11. Obtain the original data of the computing party i by the computing party i, the computing party i being any of the P computing parties;

[0055] 12. Load the original data in the xgboost model, locally traverse the xgboost model, and obtain the prediction result of the computing party i, the original data including a plurality of samples, and in the traversal process of any sample, when an intermediate tree node of a non-party feature is encountered, the sample is allowed to access both left and right child nodes first.

[0056] In a specific implementation, taking the computing party i as an example, the computing party i being any of the P computing parties, the original data of the computing party i can be obtained by the computing party i, the original data is loaded in the xgboost model, the xgboost model is locally traversed, and the prediction result of the computing party i is obtained. In the process of local prediction of the computing node, when an intermediate tree node of a non-party feature is encountered, the sample can be allowed to access both left and right child nodes first, so that the sample will access more than one leaf node, thereby generating a set of Boolean prediction vectors, and during the traversal of the entire tree node, no communication is required.

[0057] Further, the step 11 of locally traversing the xgboost model to obtain the prediction result of the computing party i can include the following steps:

[0058] 111、locally traversing the xgboost model to obtain a preliminary prediction result of each sample in the original data, and obtaining a plurality of preliminary prediction results;

[0059] 112、randomly selecting any one of the plurality of preliminary prediction results as a prediction flag vector;

[0060] 113、subtracting each of the plurality of preliminary prediction results from the prediction flag vector to obtain a plurality of intermediate prediction results;

[0061] 114、determining a sparse prediction matrix according to the plurality of intermediate prediction results;

[0062] 115、determining the sparse prediction matrix and the prediction flag vector as the prediction result of the calculation party i.

[0063] In a specific implementation, the xgboost model is locally traversed to obtain a preliminary prediction result of each sample in the original data, and a plurality of preliminary prediction results are obtained. Any one of the plurality of preliminary prediction results is randomly selected as a prediction flag vector. Each of the plurality of preliminary prediction results is subtracted from the prediction flag vector to obtain a plurality of intermediate prediction results. A sparse prediction matrix is determined according to the plurality of intermediate prediction results. The sparse prediction matrix and the prediction flag vector are determined as the prediction result of the calculation party i. Without sparse compression, each sample corresponds to a group of Boolean prediction vectors, which will cause the entire matrix to become large. Sparse compression is required before being sent to the model owner. Because in the case of large data volume (the data volume is much larger than the number of leaf nodes), a large amount of data of the same tree falls on the same leaf node. Therefore, by subtracting the prediction vectors, the overall prediction matrix can become sparse. This can reduce the communication volume, thereby reducing the overall prediction time, and at the same time, the low-bandwidth communication scenario can be met.

[0064] That is, in a specific implementation, the prediction results of each tree can be tiled together to obtain a prediction result of a certain sample, which is taken as a prediction vector, that is, as a prediction flag vector. The prediction vectors of all samples are subtracted from the prediction flag vector to obtain a sparse prediction matrix of the overall sample. After sparse storage, the sparse prediction matrix and the prediction flag vector are sent to the model owner.

[0065] 202、recovering the P prediction results into dense prediction vectors by the model owner, and obtaining P dense prediction vectors.

[0066] In this embodiment of the application, the sparse prediction vectors of each computing party can be received, and then the sparse prediction vectors can be restored to dense prediction vectors, that is, the sparse prediction vectors of each party are added to the flag vector to obtain the original prediction vector of the computing party itself.

[0067] 203. The P dense prediction vectors are intersected by the model owner to obtain the final prediction vector, and the target prediction value is determined based on the final prediction vector.

[0068] In this embodiment, after the model owner receives the prediction vectors from each computing party, they perform intersection to calculate the leaf nodes that the sample can actually access. The weight of the leaf node that is finally accessed is the sample prediction probability.

[0069] In this embodiment, the number of communication times for xgboost prediction in longitudinal federated learning can be greatly reduced, that is, the communication overhead is significantly reduced by a single communication. Furthermore, an efficient sparse optimization algorithm is used to compress the intermediate prediction values, which further reduces the amount of communication and the I / O overhead of the computing nodes, thereby improving the prediction efficiency of the federated xgboost algorithm in low-bandwidth environments.

[0070] In this embodiment, during the local prediction process of the computation node, when encountering an intermediate tree node that is not a feature of the local node, the sample is first allowed to visit both its left and right child nodes. This ensures that the sample visits more than one leaf node, generating a set of Boolean prediction vectors. Furthermore, no communication is required during the traversal of the entire tree. However, without sparse compression, each sample corresponding to a set of Boolean prediction vectors would result in a large matrix. Sparse compression is necessary before sending the vectors to the model owner. In large-scale data scenarios (where the data volume is far greater than the number of leaf nodes), a large number of data points on the same tree will fall on the same leaf nodes. Therefore, subtracting the prediction vectors makes the overall prediction matrix sparser, reducing communication overhead and thus the overall prediction time, while also meeting the requirements of low-bandwidth communication scenarios. After the model owner receives the prediction vectors from each computation node, they perform intersection calculations to determine the leaf nodes that the sample can actually access. The weight of the last accessed leaf node is the sample's prediction probability.

[0071] For example, Figure 3 As shown, the steps of a single prediction using the federated XGBoost algorithm can include the computation node's own process, the prediction result sending process, and the model owner's process.

[0072] The local computation node process is as follows: Each computation node loads data locally, traverses the model locally, and obtains a local prediction vector. Specifically, each computation node loads the original data to be predicted, and each computation node performs local prediction. When the splitting feature used by the middle node of the tree model is provided by another party, prediction is performed simultaneously on its left and right child nodes (i.e., visiting the left and right child nodes). When the leaf node of the tree model is reached, the prediction stops, and the nodes reached are marked with Boolean values. A set of Boolean vectors is obtained to represent which leaf nodes the sample may visit in the end, and the local prediction result, i.e., the local prediction vector, is obtained.

[0073] The process of sending prediction results is as follows: The sparse prediction matrix is ​​optimized, and then the sparse prediction matrix and prediction flag vector are sent. Specifically, the prediction results of each tree are flattened together, and a random sample is selected as the prediction vector, i.e., the prediction flag vector. This prediction flag vector is subtracted from the prediction vectors of all samples to obtain the sparse prediction matrix of the entire sample. After sparse storage, the sparse prediction matrix and prediction flag vector are sent to the model owner node.

[0074] The model owner process is as follows: The dense prediction vectors of each computational party are recovered in the model owner's side. The intersection of the dense prediction vectors of each party is obtained to get the final prediction vector. The prediction vector is multiplied by the weight of the model leaf node to obtain the prediction value. Specifically, the sparse prediction vectors of each computational party are received; the sparse prediction vectors are restored to dense prediction vectors, and a flag vector is added to obtain the original prediction vector of the computational party itself; the intersection of the prediction vectors of each computational party is obtained, and multiplied by the weight of the leaf node to obtain the prediction result of the overall sample.

[0075] Optionally, the following steps may also be included:

[0076] A1. Obtain the target bandwidth parameters;

[0077] A2. When the target bandwidth parameter is less than a preset threshold, perform the step of predicting P prediction results by each of the P calculation methods based on its own original data and the xgboost model.

[0078] The preset threshold can be set in advance or be the system default. The target bandwidth parameter can be obtained. When the target bandwidth parameter is less than the preset threshold, step 201 is executed, which is the prediction method of the federated xgboost algorithm in low bandwidth scenarios, effectively improving the prediction efficiency of the vertical federated xgboost algorithm.

[0079] In this embodiment of the application, without affecting the prediction accuracy and security of the federated learning model, a federated xgboost algorithm prediction method based on low bandwidth scenarios is designed, which effectively improves the prediction efficiency of the vertical federated xgboost algorithm.

[0080] As can be seen, the federated XGBoost algorithm prediction method for low-bandwidth scenarios described in this application embodiment is applied to a multi-party computation system, which includes P computation parties and a model owner of the XGBoost model, where P is an integer greater than 1. Each of the P computation parties performs predictions based on its own original data and the XGBoost model, obtaining P prediction results. Each prediction result includes a sparse prediction matrix and a prediction vector. The model owner restores the P prediction results into dense prediction vectors, obtaining P dense prediction vectors. The model owner intersects the P dense prediction vectors to obtain the final prediction vector, and the target prediction value is determined based on the final prediction vector. This method not only eliminates the need for communication during the traversal of the entire tree nodes but also makes the overall prediction matrix sparse, thus reducing communication overhead and overall prediction time. It also meets the requirements of low-bandwidth communication scenarios, thereby improving the prediction efficiency of the federated XGBoost algorithm in low-bandwidth environments.

[0081] Consistent with the above embodiments, please refer to Figure 4 , Figure 4 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. As shown in the figure, the electronic device includes a processor, a memory, a communication interface, and one or more programs. The one or more programs are stored in the memory and configured to be executed by the processor. In this embodiment, the electronic device is applied to a multi-party computing system, which includes P computing parties and a model owner of the XGBoost model, where P is an integer greater than 1. The program includes instructions for performing the following steps:

[0082] Each of the P computational methods performs a prediction based on its own original data and the xgboost model to obtain P prediction results, each prediction result including a sparse prediction matrix and a prediction vector.

[0083] The P prediction results are restored into dense prediction vectors by the model owner, resulting in P dense prediction vectors;

[0084] The model owner intersects the P dense prediction vectors to obtain the final prediction vector, and the target prediction value is determined based on the final prediction vector.

[0085] Optionally, in obtaining P prediction results by each of the P calculation methods based on its respective original data and the xgboost model, the above procedure includes instructions for performing the following steps:

[0086] The original data of computation method i is obtained through computation method i, where computation method i is any one of the P computation methods;

[0087] The original data is loaded into the xgboost model, and the xgboost model is traversed locally to obtain the prediction result of the calculation method i. The original data includes multiple samples. During the traversal of any sample, when an intermediate tree node that is not a feature of this method is encountered, the sample is first allowed to visit both its left and right child nodes.

[0088] Further, optionally, in the process of locally traversing the xgboost model to obtain the prediction results of computation method i, the above procedure includes instructions for performing the following steps:

[0089] The xgboost model is traversed locally to obtain preliminary prediction results for each sample in the original data, resulting in multiple preliminary prediction results.

[0090] Randomly select any one of the multiple preliminary prediction results as the prediction flag vector;

[0091] Subtract the prediction flag vector from each of the multiple preliminary prediction results to obtain multiple intermediate prediction results;

[0092] A sparse prediction matrix is ​​determined based on the multiple intermediate prediction results;

[0093] The sparse prediction matrix and the prediction flag vector are determined as the prediction result of the calculation method i.

[0094] Optionally, in determining the target predicted value based on the final predicted vector, the above procedure includes instructions for performing the following steps:

[0095] The final prediction vector is calculated by combining it with the weights of each leaf node of the XGBoost model to obtain the target prediction value.

[0096] Optionally, the above procedure may also include instructions for performing the following steps:

[0097] Obtain the target bandwidth parameters;

[0098] When the target bandwidth parameter is less than a preset threshold, the step of performing predictions by each of the P calculation methods based on its own original data and the xgboost model to obtain P prediction results is executed.

[0099] As can be seen, the electronic device described in this application embodiment is applied to a multi-party computing system. The multi-party computing system includes P computing parties and a model owner of the XGBoost model, where P is an integer greater than 1. Each of the P computing parties makes predictions based on its own original data and the XGBoost model, obtaining P prediction results. Each prediction result includes a sparse prediction matrix and a prediction vector. The model owner restores the P prediction results into dense prediction vectors, obtaining P dense prediction vectors. The model owner intersects the P dense prediction vectors to obtain the final prediction vector, and the target prediction value is determined based on the final prediction vector. This not only eliminates the need for communication during the traversal of the entire tree nodes, but also makes the overall prediction matrix sparse, thus reducing communication volume and overall prediction time. It also meets the requirements of low-bandwidth communication scenarios, thereby improving the prediction efficiency of the federated XGBoost algorithm in low-bandwidth environments.

[0100] Figure 5 This is a functional unit block diagram of a federated XGBoost algorithm prediction device 500 for low-bandwidth scenarios, as described in this application embodiment. It is applied to a multi-party computation system, which includes P computation parties and a model owner party for the XGBoost model, where P is an integer greater than 1. The device 500 includes: a prediction unit 501, a recovery unit 502, and a determination unit 503.

[0101] The prediction unit 501 is used to make predictions based on their respective original data and the xgboost model through each of the P calculation methods to obtain P prediction results, each prediction result including a sparse prediction matrix and a prediction vector.

[0102] The recovery unit 502 is used to recover the P prediction results into dense prediction vectors through the model owner, thereby obtaining P dense prediction vectors.

[0103] The determining unit 503 is used to intersect the P dense prediction vectors through the model owner to obtain the final prediction vector, and to determine the target prediction value based on the final prediction vector.

[0104] Optionally, in the process of obtaining P prediction results by each of the P calculation methods based on its own original data and the xgboost model, the prediction unit 501 is specifically used for:

[0105] The original data of computation method i is obtained through computation method i, where computation method i is any one of the P computation methods;

[0106] The original data is loaded into the xgboost model, and the xgboost model is traversed locally to obtain the prediction result of the calculation method i. The original data includes multiple samples. During the traversal of any sample, when an intermediate tree node that is not a feature of this method is encountered, the sample is first allowed to visit both its left and right child nodes.

[0107] Optionally, in the process of locally traversing the xgboost model to obtain the prediction result of computation method i, the prediction unit 501 is specifically used for:

[0108] The xgboost model is traversed locally to obtain preliminary prediction results for each sample in the original data, resulting in multiple preliminary prediction results.

[0109] Randomly select any one of the multiple preliminary prediction results as the prediction flag vector;

[0110] Subtract the prediction flag vector from each of the multiple preliminary prediction results to obtain multiple intermediate prediction results;

[0111] A sparse prediction matrix is ​​determined based on the multiple intermediate prediction results;

[0112] The sparse prediction matrix and the prediction flag vector are determined as the prediction result of the calculation method i.

[0113] Optionally, in determining the target predicted value based on the final predicted vector, the determining unit 503 is specifically used for:

[0114] The final prediction vector is calculated by combining it with the weights of each leaf node of the XGBoost model to obtain the target prediction value.

[0115] Optionally, the device 500 is further specifically used for:

[0116] Obtain the target bandwidth parameters;

[0117] When the target bandwidth parameter is less than a preset threshold, the step of performing predictions by each of the P calculation methods based on its own original data and the xgboost model to obtain P prediction results is executed.

[0118] As can be seen, the federated XGBoost algorithm prediction device for low-bandwidth scenarios described in this application embodiment is applied to a multi-party computation system, which includes P computation parties and a model owner of the XGBoost model, where P is an integer greater than 1. Each of the P computation parties performs predictions based on its own original data and the XGBoost model, obtaining P prediction results. Each prediction result includes a sparse prediction matrix and a prediction vector. The model owner restores the P prediction results into dense prediction vectors, obtaining P dense prediction vectors. The model owner intersects the P dense prediction vectors to obtain the final prediction vector, and the target prediction value is determined based on the final prediction vector. This not only eliminates the need for communication during the traversal of the entire tree nodes, but also makes the overall prediction matrix sparse, thus reducing communication volume and overall prediction time. It also meets the requirements of low-bandwidth communication scenarios, thereby improving the prediction efficiency of the federated XGBoost algorithm in low-bandwidth environments.

[0119] It is understood that the functions of each program module of the federated xgboost algorithm prediction device in the low bandwidth scenario of this embodiment can be specifically implemented according to the methods in the above method embodiments. The specific implementation process can be referred to the relevant descriptions in the above method embodiments, and will not be repeated here.

[0120] This application also provides a computer storage medium storing a computer program for electronic data interchange, which causes a computer to perform some or all of the steps of any of the methods described in the above method embodiments, wherein the computer includes an electronic device.

[0121] This application also provides a computer program product, which includes a non-transitory computer-readable storage medium storing a computer program operable to cause a computer to perform some or all of the steps of any of the methods described in the above method embodiments. The computer program product may be a software installation package, and the computer may include an electronic device.

[0122] It should be noted that, for the sake of simplicity, the foregoing method embodiments are all described as a series of actions. However, those skilled in the art should understand that this application is not limited to the described order of actions, as some steps may be performed in other orders or simultaneously according to this application. Furthermore, those skilled in the art should also understand that the embodiments described in the specification are preferred embodiments, and the actions and modules involved are not necessarily essential to this application.

[0123] In the above embodiments, the descriptions of each embodiment have different focuses. For parts not described in detail in a certain embodiment, please refer to the relevant descriptions in other embodiments.

[0124] In the several embodiments provided in this application, it should be understood that the disclosed apparatus can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of the units described above is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between devices or units may be electrical or other forms.

[0125] The units described above as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0126] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0127] If the integrated units described above are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage device (CMD). Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a memory and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned memory includes various media capable of storing program code, such as USB flash drives, read-only memory (ROM), random access memory (RAM), portable hard drives, magnetic disks, or optical disks.

[0128] Those skilled in the art will understand that all or part of the steps in the various methods of the above embodiments can be implemented by a program instructing related hardware. The program can be stored in a computer-readable storage medium, which may include: flash drive, read-only memory (ROM), random access memory (RAM), disk or optical disk, etc.

[0129] The embodiments of this application have been described in detail above. Specific examples have been used to illustrate the principles and implementation methods of this application. The description of the above embodiments is only for the purpose of helping to understand the method and core ideas of this application. At the same time, for those skilled in the art, there will be changes in the specific implementation methods and application scope based on the ideas of this application. Therefore, the content of this specification should not be construed as a limitation of this application.

Claims

1. A federated XGBoost algorithm prediction method for low-bandwidth scenarios, characterized in that, The method is applied to a multi-party computation system, which includes P computation parties and a model owner party of an XGBoost model, where P is an integer greater than 1; the method includes: Each of the P computational methods performs a prediction based on its own original data and the xgboost model to obtain P prediction results, each prediction result including a sparse prediction matrix and a prediction vector. The P prediction results are restored into dense prediction vectors by the model owner, resulting in P dense prediction vectors; The P dense prediction vectors are intersected by the model owner to obtain the final prediction vector, and the target prediction value is determined based on the final prediction vector. Wherein, the step of obtaining P prediction results by each of the P calculation methods based on its own original data and the xgboost model includes: The original data of computation method i is obtained through computation method i, where computation method i is any one of the P computation methods; The original data is loaded into the xgboost model, and the xgboost model is traversed locally to obtain the prediction result of the calculation method i. The original data includes multiple samples. During the traversal of any sample, when an intermediate tree node that is not a feature of this method is encountered, the sample is first allowed to visit both its left and right child nodes. The local traversal of the xgboost model to obtain the prediction result of computation method i includes: The xgboost model is traversed locally to obtain preliminary prediction results for each sample in the original data, resulting in multiple preliminary prediction results. Randomly select any one of the multiple preliminary prediction results as the prediction flag vector; Subtract the prediction flag vector from each of the multiple preliminary prediction results to obtain multiple intermediate prediction results; A sparse prediction matrix is ​​determined based on the multiple intermediate prediction results; The sparse prediction matrix and the prediction flag vector are determined as the prediction result of the calculation method i.

2. The method according to claim 1, characterized in that, Determining the target predicted value based on the final predicted vector includes: The final prediction vector is calculated by combining it with the weights of each leaf node of the XGBoost model to obtain the target prediction value.

3. The method according to claim 1, characterized in that, The method further includes: Obtain the target bandwidth parameters; When the target bandwidth parameter is less than a preset threshold, the step of performing predictions by each of the P calculation methods based on its own original data and the xgboost model to obtain P prediction results is executed.

4. A federated XGBoost algorithm prediction device for low-bandwidth scenarios, characterized in that, The device is applied to a multi-party computation system, which includes P computational parties and a model owner party of the XGBoost model, where P is an integer greater than 1; the device includes: a prediction unit, a recovery unit, and a determination unit, wherein, The prediction unit is used to make predictions based on the original data and the xgboost model by each of the P calculation methods to obtain P prediction results, each prediction result including a sparse prediction matrix and a prediction vector. The recovery unit is used to recover the P prediction results into dense prediction vectors through the model owner, thereby obtaining P dense prediction vectors; The determining unit is used to intersect the P dense prediction vectors through the model owner to obtain the final prediction vector, and to determine the target prediction value based on the final prediction vector. Specifically, in the process of obtaining P prediction results by each of the P calculation methods based on its own original data and the xgboost model, the prediction unit is used for: The original data of computation method i is obtained through computation method i, where computation method i is any one of the P computation methods; The original data is loaded into the xgboost model, and the xgboost model is traversed locally to obtain the prediction result of the calculation method i. The original data includes multiple samples. During the traversal of any sample, when an intermediate tree node that is not a feature of this method is encountered, the sample is first allowed to visit both its left and right child nodes. Specifically, in the process of locally traversing the xgboost model to obtain the prediction result of computation method i, the prediction unit is used for: The xgboost model is traversed locally to obtain preliminary prediction results for each sample in the original data, resulting in multiple preliminary prediction results. Randomly select any one of the multiple preliminary prediction results as the prediction flag vector; Subtract the prediction flag vector from each of the multiple preliminary prediction results to obtain multiple intermediate prediction results; A sparse prediction matrix is ​​determined based on the multiple intermediate prediction results; The sparse prediction matrix and the prediction flag vector are determined as the prediction result of the calculation method i.

5. An electronic device, characterized in that, It includes a processor and a memory, the memory being used to store one or more programs and configured to be executed by the processor, the programs including instructions for performing the steps of the method as described in any one of claims 1-3.

6. A computer-readable storage medium, characterized in that, A computer program for storing electronic data interchange is provided, wherein the computer program causes a computer to perform the method as described in any one of claims 1-3.

Images