Multi-party vertical gbdt secure bucket aggregation method based on secret sharing and masking

Abstract

The application provides a multi-party longitudinal GBDT secure bucket aggregation method based on secret sharing and masking, comprising: obtaining a plurality of gradient vector segment values of a plurality of nodes and an indicator vector of a feature owner node; and obtaining sample gradient and segment value of any node by using the plurality of gradient vector segment values and the indicator vector; wherein the plurality of nodes comprise a plurality of passive party nodes, one active party node and one auxiliary party node, and any node in the plurality of passive party nodes is the feature owner node. The application effectively avoids the expensive homomorphic encryption overhead by proposing the multi-party longitudinal GBDT secure bucket aggregation calculation method, and can easily expand to the multi-participation scene by using the mask to protect the original data, thereby significantly improving the performance of the privacy protection GBDT training on the distributed large-scale data set.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of information security, and particularly relates to a multi-party longitudinal GBDT secure bucket aggregation method based on secret sharing and masking. BACKGROUND

[0002] Gradient Boosting Decision Trees (GBDT) is a widely used method in the field of machine learning, which is favored for its excellent performance in tasks such as fraud detection, online advertising recommendation, and risk management, as well as its strong model interpretability. However, with the increasing demand for data privacy protection, the traditional GBDT training method faces severe challenges, especially in the context of multi-party data collaboration. To solve this problem, multi-party computation protocols are introduced into the training process of GBDT, enabling multiple data holders to collaboratively train GBDT models without disclosing their private data sets. This joint modeling approach not only effectively safeguards data privacy, but also enhances the generalization ability and prediction accuracy of the model by integrating multi-party data.

[0003] In the context of federated learning or privacy protection, the secure bucket aggregation protocol is one of the key technologies for implementing privacy-protected joint training of GBDT. This protocol addresses the risk of data privacy leakage in traditional GBDT training by securely aggregating gradient information from each participant. Specifically, GBDT needs to find the optimal feature split point to minimize the loss function in each iteration. Traditional methods require calculating the loss function value for all possible split points of each feature based on first-order and second-order gradients, which is computationally expensive when dealing with large data sets. To address this issue, the secure bucket aggregation protocol divides feature values into multiple intervals (referred to as "buckets" or "bins") and calculates the first-order and second-order gradient sums within each bucket, thereby significantly reducing computational complexity. In addition, by combining encryption technology or differential privacy mechanisms, the protocol protects data privacy while achieving efficient feature split point search. This approach provides important support for the application of GBDT in privacy scenarios.

[0004] The 23-year literature "Squirrel: A scalable secure Two-Party computation framework for training gradient boosting decision tree" obtains a secure and practical two-party longitudinal decision tree secure bucket aggregation scheme by utilizing hybrid encryption primitives, which overall improves the efficiency of privacy-protected machine learning. However, it relies on computationally expensive homomorphic encryption (HE) schemes to achieve communication efficiency.

[0005] The 24-year paper "NodeGuard: A Highly Efficient Two-Party Computation Framework for Training Large-Scale Gradient Boosting Decision Tree" uses a new key bucket aggregation protocol to globally optimize communication and computational complexity during training, achieving efficiency advantages. This solution focuses on the vertical federated learning setting, in which two participants and The same samples with different feature spaces, i.e. hold , hold , at the same time, let Holding label data To ensure privacy, node split information during training is only available to the party with the best split candidate. Assume that the number of samples is , for any node , each participant holds Shard ,in Each participant wishes to calculate each feature Each bucket The gradient and. In addition, only the feature owner Knowing the indicator vector , the indicator vector consists only of 0 and 1, which is used to indicate the location of the sample. If a sample belongs to the feature Bucket , then the node's The corresponding element in is 1, otherwise it is 0. Then, the indicator vector and the gradient vector are inner-producted to obtain the feature Barrel The sum of the sample gradients. Specifically, in the offline phase, a random number is generated by a trusted third party or a two-party multiplication protocol. and ,calculate And split it into pieces, and Send to , and Send to . Online stage, calculate and send it to , calculate and send to .Then, and The features can be calculated locally according to the data mastered Upper bucket Gradient and slice However, it is only applicable to the case of two participants, and therefore has a limited range of application. SUMMARY

[0006] The application provides a multi-party longitudinal GBDT safe bucket aggregation method based on secret sharing and masking, to solve the defect that the GBDT safe bucket aggregation in the prior art is not applicable to the multi-party scenario.

[0007] In a first aspect, the application provides a multi-party longitudinal GBDT safe bucket aggregation method based on secret sharing and masking, comprising:

[0008] Obtaining a plurality of gradient vector slice values of a plurality of nodes and an indication vector of a feature owner node;

[0009] Using the plurality of gradient vector slice values and the indication vector, obtaining a sample gradient and slice value of any node;

[0010] Wherein, the plurality of nodes includes a plurality of passive party nodes, an active party node and an auxiliary party node, and any node in the plurality of passive party nodes is the feature owner node.

[0011] According to the multi-party longitudinal GBDT safe bucket aggregation method based on secret sharing and masking provided by the application, before obtaining the plurality of gradient vector slice values of the plurality of nodes and the indication vector of the feature owner node, further comprising:

[0012] Determining that the plurality of passive party nodes and the active party node participate in model training, and the auxiliary party node assists in model training;

[0013] Obtaining the number of samples, and the active party node calculates the total gradient vector according to the label value, the prediction value and the number of samples;

[0014] The active party node shares the total gradient vector to the plurality of passive party nodes after slicing, and sends the total gradient vector to the auxiliary party node after removing the gradient vector slice value corresponding to the feature owner node;

[0015] According to the number of samples, determining the indication vector of the feature owner node.

[0016] According to the multi-party longitudinal GBDT safe bucket aggregation method based on secret sharing and masking provided by the application, the feature owner node generates a random number vector using a random number generator, the number of random number vectors is equal to the number of samples, and the sum of the indication vector and the random number vector is calculated;

[0017] The feature owner node sends the sum of the indication vector and the random number vector to the rest of the passive nodes and the active node respectively, and sends the random number vector to the auxiliary node.

[0018] According to the multi-party longitudinal GBDT secure bucket aggregation method based on secret sharing and masking provided by the application, the multiple nodes excluding the feature owner node respectively multiply the sum of the indication vector and the random number vector by the gradient vector slice value of each node to obtain the gradient and slice of each node.

[0019] The feature owner node multiplies the indication vector by the gradient vector slice value of the feature owner node to obtain the gradient and slice of the feature owner node.

[0020] According to the multi-party longitudinal GBDT secure bucket aggregation method based on secret sharing and masking provided by the application, the auxiliary node multiplies the sum of the random number vector negative number and the gradient vector slice value of the multiple nodes excluding the feature owner node to obtain the gradient and slice of the auxiliary node.

[0021] The gradient and slice of the auxiliary node are secret shared to the multiple passive nodes and the active node.

[0022] According to the multi-party longitudinal GBDT secure bucket aggregation method based on secret sharing and masking provided by the application, the method further comprises:

[0023] The gradient and slice of the auxiliary node, the gradient and slice of each node, and the gradient and slice of the feature owner node are added to obtain the product of the indication vector and the total gradient vector, and the correctness proof is completed.

[0024] According to the multi-party longitudinal GBDT secure bucket aggregation method based on secret sharing and masking provided by the application, the multiple nodes include floating point numbers and fixed point numbers.

[0025] In a second aspect, the application further provides a multi-party longitudinal GBDT secure bucket aggregation system based on secret sharing and masking, comprising:

[0026] An input module is configured to obtain multiple gradient vector slice values of multiple nodes and an indication vector of a feature owner node.

[0027] An output module is configured to obtain sample gradient and slice values of any node by using the multiple gradient vector slice values and the indication vector.

[0028] The multiple nodes include multiple passive nodes, one active node and one auxiliary node, and any node in the multiple passive nodes is the feature owner node.

[0029] In a third aspect, the present application also provides an electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the multi-party longitudinal GBDT secure bucket aggregation method based on secret sharing and masking according to any of the above aspects when executing the program.

[0030] In a fourth aspect, the present application also provides a non-transitory computer readable storage medium having a computer program stored thereon, wherein the computer program is executable on a processor to implement the multi-party longitudinal GBDT secure bucket aggregation method based on secret sharing and masking according to any of the above aspects.

[0031] The multi-party longitudinal GBDT secure bucket aggregation method based on secret sharing and masking provided by the present application effectively avoids the expensive homomorphic encryption overhead by proposing a multi-party longitudinal GBDT secure bucket aggregation calculation method, and can easily extend to a multi-participant scenario by using a mask to protect the original data, thereby significantly improving the performance of privacy protection GBDT training on a distributed large-scale data set. BRIEF DESCRIPTION OF DRAWINGS

[0032] In order to more clearly illustrate the technical solutions in the present application or the prior art, the following will briefly introduce the drawings needed to be used in the embodiments or prior art description. Obviously, the drawings in the following description are some embodiments of the present application, and for those skilled in the art, other drawings can also be obtained without creative labor based on these drawings.

[0033] Figure 1 is a flowchart of the multi-party longitudinal GBDT secure bucket aggregation method based on secret sharing and masking provided by the present application;

[0034] Figure 2 is an embodiment diagram of the multi-party longitudinal GBDT secure bucket aggregation method based on secret sharing and masking provided by the present application;

[0035] Figure 3 is a structural schematic diagram of the multi-party longitudinal GBDT secure bucket aggregation system based on secret sharing and masking provided by the present application;

[0036] Figure 4 is a structural schematic diagram of the electronic device provided by the present application. DETAILED DESCRIPTION

[0037] In order to make the objects, technical solutions and advantages of the present application clearer, the technical solutions in the present application will be described clearly and completely below in combination with the drawings in the present application. Obviously, the described embodiments are part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative labor fall within the protection scope of the present application.

[0038] The prior art usually relies on a high-computing-cost homomorphic encryption (HE) scheme to improve communication efficiency when performing GBDT secure bucket aggregation, or is only applicable to a two-party environment. In view of the above two challenges, the present application proposes an innovative scheme which is not only applicable to multi-party vertical GBDT secure bucket aggregation calculation, but also effectively avoids expensive homomorphic encryption overhead. By using a mask to protect the original data, the scheme of the present application can be easily extended to a multi-participant scenario, thereby significantly improving the performance of privacy-preserving GBDT training on a distributed large-scale dataset.

[0039] Figure 1 is a flowchart of a multi-party vertical GBDT secure bucket aggregation method based on secret sharing and masking provided by an embodiment of the present application, as shown in Figure 1 , comprising:

[0040] Step 100: obtaining a plurality of gradient vector shard values of a plurality of nodes and an indicator vector of a feature owner node;

[0041] Step 200: obtaining a sample gradient and a shard value of any node by using the plurality of gradient vector shard values and the indicator vector;

[0042] The plurality of nodes include a plurality of passive party nodes, one active party node and one auxiliary party node, and any node in the plurality of passive party nodes is the feature owner node.

[0043] Specifically, since GBDT is a method widely used in the field of machine learning, inspired by the existing technology, the present application proposes a multi-party vertical GBDT secure bucket aggregation algorithm based on secret sharing and masking, assuming that all participants are honest and curious, which can effectively improve the calculation efficiency while ensuring data privacy in a multi-party participating environment.

[0044] The principle of the algorithm is described by taking a three-party scenario as an example, as shown in Figure 2 .

[0045] Suppose there are three participants participating in the training of the model, and the participants are denoted as , , , wherein , Is the passive party, Is the active party, the existing auxiliary party Responsible for assisting the training process. The number of samples is N. To grasp the tag value, you need Calculate the gradient vector based on the label value and the predicted value And share its secret with all participants, ,in The obtained gradient vector slice value is , The obtained gradient vector slice value is , The obtained gradient vector slice value , in particular, Will The value is sent to For one of the tree nodes, suppose is the feature owner, i.e. Known indicator vector .

[0046] enter: Gradient vector slice value , Gradient vector slice value and indicator vector , Gradient vector slice value , Gradient vector slice value .

[0047] Output: Get the indicator vectors With the gradient vector The inner product value of 's shard, that is, the shard of the sample gradient sum of the node.

[0048] Furthermore, based on the following settings:

[0049] (1) Feature owner Generate using a random number generator Random numbers , and calculate ;

[0050] (2) Will The value is sent to and ,Will The value of ;

[0051] (3) Participants and Compute the gradient sum of each slice respectively and ;

[0052] (4) Participating party Compute the gradient sum of each slice ;

[0053] (5) Auxiliary party Compute the gradient sum of each slice If the auxiliary party does not want to participate in the next operation, the secret share can be given to the participating party , , .

[0054] Finally, the correctness is proved:

[0055]

[0056] It can be understood that in actual development and application, conversion between floating point numbers and fixed point numbers needs to be performed according to known methods.

[0057] In particular, for more participating parties, the active participating party only needs to send the sum of gradient slices other than the gradient slice of the feature owner to the auxiliary party , and other steps remain unchanged, so that the algorithm has good scalability and avoids expensive homomorphic encryption.

[0058] Based on the above algorithm, the training and reasoning of GBDT can be performed according to known methods.

[0059] The multi-party vertical GBDT secure bucket aggregation system based on secret sharing and masking provided by the present application is described below. The multi-party vertical GBDT secure bucket aggregation system based on secret sharing and masking described below can be mutually corresponding to the multi-party vertical GBDT secure bucket aggregation method based on secret sharing and masking described above.

[0060] Figure 3 is a structural schematic diagram of the multi-party vertical GBDT secure bucket aggregation system based on secret sharing and masking provided by the present application, as Figure 3 shown, comprising an input module 31 and an output module 32, wherein:

[0061] The input module 31 is configured to obtain the plurality of gradient vector slice values of the plurality of nodes and the indication vector of the feature owner node; the output module 32 is configured to obtain the sample gradient and the slice value of any node by using the plurality of gradient vector slice values and the indication vector; wherein the plurality of nodes include a plurality of passive nodes, an active node and an auxiliary node, and any node in the plurality of passive nodes is the feature owner node.

[0062] Figure 4 An example of an entity structure diagram of an electronic device is shown in Figure 4 The electronic device can include a processor 410, a communications interface 420, a memory 430 and a communications bus 440, wherein the processor 410, the communications interface 420 and the memory 430 can communicate with each other through the communications bus 440. The processor 410 can invoke the logic instructions in the memory 430 to execute the multi-party longitudinal GBDT secure bucket aggregation method based on secret sharing and masking, which includes: obtaining a plurality of gradient vector slice values of a plurality of nodes and an indication vector of a feature owner node; obtaining the sample gradient and the slice value of any node by using the plurality of gradient vector slice values and the indication vector; wherein the plurality of nodes include a plurality of passive nodes, an active node and an auxiliary node, and any node in the plurality of passive nodes is the feature owner node.

[0063] In addition, the logic instructions in the memory 430 described above can be implemented in the form of a software functional unit and sold or used as an independent product, which can be stored in a computer readable storage medium. Based on such understanding, the technical solutions of the present application essentially or the part that contributes to the prior art or part of the technical solutions can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a plurality of instructions to make a computer device (which can be a personal computer, a server, or a network device, etc.) execute all or part of the steps of the methods described in various embodiments of the present application. The foregoing storage medium includes: a U disk, a mobile hard disk, a read-only memory (ROM, Read-Only Memory), a random access memory (RAM, Random Access Memory), a magnetic disk or an optical disk, and various media that can store program codes.

[0064] In another aspect, the present application also provides a non-transitory computer readable storage medium having stored thereon a computer program, which, when executed by a processor, implements a multi-party vertical GBDT secure bucket aggregation method based on secret sharing and masking provided by the above method, the method comprising: obtaining a plurality of gradient vector shard values of a plurality of nodes and an indicator vector of a feature owner node; obtaining a sample gradient and a shard value of any node by using the plurality of gradient vector shard values and the indicator vector; wherein the plurality of nodes comprise a plurality of passive party nodes, one active party node and one auxiliary party node, and any node in the plurality of passive party nodes is the feature owner node.

[0065] The device embodiments described above are merely illustrative, wherein the units described as separate components can or can not be physically separate, and the components displayed as units can or can not be physical units, i.e., can be located in one place, or can be distributed on multiple network units. Part or all of the modules can be selected to achieve the purpose of the embodiment scheme according to actual needs. Those skilled in the art can understand and implement without creative labor.

[0066] From the above description of the embodiments, those skilled in the art can clearly understand that the embodiments can be realized by means of software plus necessary universal hardware platforms, and of course can also be realized by hardware. Based on such understanding, the above technical solutions, essentially or in other words, the part that makes a contribution to the prior art, can be embodied in the form of a software product, which can be stored in a computer readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes a number of instructions to make a computer device (which can be a personal computer, a server, or a network device, etc.) execute the methods described in each embodiment or some parts of the embodiments.

[0067] Finally, it should be noted that: the above embodiments are only used to illustrate the technical solutions of the present application, and not to limit them; although the present application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that: it can still modify the technical solutions recorded in the foregoing embodiments, or make equivalent replacement to some technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. A multi-party longitudinal GBDT secure bucket aggregation method based on secret sharing and masking, characterized in that: include: Obtain multiple gradient vector slice values ​​of multiple nodes and indicator vectors of feature owner nodes; Using multiple gradient vector slice values ​​and indicator vectors, the sample gradient and slice value of any node are obtained; The plurality of nodes include a plurality of passive nodes, an active node and an auxiliary node, and any one of the plurality of passive nodes is a feature owner node; Before obtaining multiple gradient vector slice values ​​of multiple nodes and the indicator vector of the feature owner node, it also includes: Determine that multiple passive nodes and active nodes participate in model training, and auxiliary nodes assist in model training; Obtain the number of samples. The active node calculates the total gradient vector based on the label value, predicted value, and number of samples. The active node shards the total gradient vector and secretly shares it with multiple passive nodes. It also removes the gradient vector shard value corresponding to the feature owner node from the total gradient vector and sends it to the auxiliary node. According to the number of samples, determine the indicator vector of the feature owner node The feature owner node uses a random number generator to generate a random number vector, the number of which is equal to the number of samples, and calculates the sum of the indicator vector and the random number vector; The feature owner node sends the sum of the indication vector and the random number vector to the remaining passive nodes and the active node respectively, and sends the random number vector to the auxiliary node For multiple nodes excluding the feature owner node, the sum of the indicator vector and the random number vector is multiplied by the gradient vector slice value of each node to obtain the gradient and slice of each node; The feature owner node uses the indicator vector and the gradient vector slice value of the feature owner node to obtain the feature owner node gradient and slice; The auxiliary node multiplies the negative of the random number vector by the sum of the gradient vector slice values ​​of multiple nodes excluding the feature owner node to obtain the auxiliary node gradient and slice; Share worker node gradients and shard secrets to multiple passive and active nodes.

2. The multi-party longitudinal GBDT secure bucket aggregation method based on secret sharing and masking according to claim 1 is characterized in that Also includes: Add the auxiliary node gradient and shard, each node gradient and shard, and feature owner node gradient and shard to obtain the product of the indicator vector and the total gradient vector to complete the correctness proof.

3. The multi-party longitudinal GBDT secure bucket aggregation method based on secret sharing and masking according to claim 1 or 2, characterized in that: Several nodes include floating-point and fixed-point numbers.

4. A multi-party vertical GBDT secure bucket aggregation system based on secret sharing and masking, based on the multi-party vertical GBDT secure bucket aggregation method based on secret sharing and masking according to any one of claims 1 to 3, characterized in that: include: An input module, used to obtain multiple gradient vector slice values ​​of multiple nodes and an indicator vector of a feature owner node; An output module, configured to obtain a sample gradient and a slice value of any node using multiple gradient vector slice values ​​and indicator vectors; The multiple nodes include multiple passive nodes, one active node and one auxiliary node, and any one of the multiple passive nodes is a feature owner node.

5. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the program, the multi-party longitudinal GBDT secure bucket aggregation method based on secret sharing and masking is implemented as described in any one of claims 1 to 3.

6. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the multi-party longitudinal GBDT secure bucket aggregation method based on secret sharing and masking is implemented as described in any one of claims 1 to 3.

Images