Multi-party longitudinal GBDT security bucket aggregation method based on secret sharing and mask

Abstract

The invention provides a multi-party longitudinal GBDT security bucket aggregation method based on secret sharing and masks. The method comprises the following steps: acquiring a plurality of gradient vector fragment values of a plurality of nodes and an indication vector of a feature owner node; obtaining a sample gradient and a fragmentation value of any node by using the plurality of gradient vector fragmentation values and the indication vector; wherein the plurality of nodes comprise a plurality of passive party nodes, an active party node and an auxiliary party node, and any node in the plurality of passive party nodes is a feature owner node. According to the multi-party longitudinal GBDT security bucket aggregation calculation method, expensive homomorphic encryption overhead is effectively avoided, original data are protected by using masks, and the method can be easily expanded to a scene of multiple participants, so that the performance of privacy protection GBDT training on a distributed large-scale data set is remarkably improved.

Description

Technical Field

[0001] The present invention relates to the field of information security technology, and in particular to a multi-party vertical GBDT security bucket aggregation method based on secret sharing and masking. Background Art

[0002] Gradient Boosting Decision Trees (GBDT) is a method widely used in the field of machine learning. It is favored for its excellent performance in tasks such as fraud detection, online advertising recommendation, and risk management, as well as strong model interpretability. However, with the increasing demand for data privacy protection, traditional GBDT training methods face severe challenges, especially in scenarios of multi-party data collaboration. To solve this problem, multi-party computing protocols are introduced into the GBDT training process, allowing multiple data holders to collaborate on training GBDT models without having to disclose their private data sets. This joint modeling method not only effectively protects data privacy, but also improves the generalization ability and prediction accuracy of the model by integrating multi-party data.

[0003] In the scenario of federated learning or privacy protection, the secure bucket aggregation protocol is one of the key technologies for GBDT to achieve privacy-preserving joint training. The protocol solves the risk of data privacy leakage in traditional GBDT training by securely aggregating the gradient information of each participant. Specifically, GBDT needs to find the optimal feature splitting point to minimize the loss function in each round of iteration. The traditional method needs to calculate the loss function value of all possible splitting points of each feature based on the first-order gradient and the second-order gradient. This process has extremely high computational overhead when the amount of data is large. To solve this problem, the secure bucket aggregation protocol divides the feature values ​​into multiple intervals (called "buckets" or "bins") and counts the sum of the first-order gradient and the sum of the second-order gradient in each bucket, thereby significantly reducing the computational complexity. In addition, by combining encryption technology or differential privacy mechanism, the protocol realizes efficient feature splitting point search while protecting data privacy. This method provides important support for the application of GBDT in privacy scenarios.

[0004] The 23-year paper "Squirrel: A scalable secure two-party computation framework for training gradient boosting decision tree" obtains a secure and practical two-party vertical decision tree secure bucket aggregation scheme by utilizing hybrid encryption primitives, which generally improves the efficiency of privacy-preserving machine learning. However, it relies on a computationally expensive homomorphic encryption (HE) scheme in exchange for communication efficiency.

[0005] The 24-year paper "NodeGuard: A Highly Efficient Two-Party Computation Framework for Training Large-Scale Gradient Boosting Decision Tree" uses a new key bucket aggregation protocol to globally optimize communication and computational complexity during training, achieving efficiency advantages. This solution focuses on the vertical federated learning setting, in which the two participants and The same samples with different feature spaces, i.e. hold , hold , at the same time, let Holds label data To ensure privacy, node split information during training is only available to the party with the best split candidate. Assume that the sample size is , for any node , each participant holds Shard ,in Each participant wishes to calculate each feature Each bucket The gradient and. In addition, only the feature owner Knowing the indicator vector , the indicator vector consists only of 0 and 1, which is used to indicate the location of the sample. If a sample belongs to the feature Bucket , then the node's The corresponding element in is 1, otherwise it is 0. Then, the indicator vector is inner-producted with the gradient vector to obtain the feature Barrel The sum of sample gradients. Specifically, in the offline phase, a random number is generated by a trusted third party or a two-party multiplication protocol and ,calculate And split it into pieces, and Send to , and Send to . Online stage, calculate and send it to , calculate and send to .Then, and You can calculate features locally based on the data you already have Barrel Gradients and sharding . However, it only works with two participants, so its application scope is relatively limited. Summary of the invention

[0006] The present invention provides a multi-party vertical GBDT security bucket aggregation method based on secret sharing and masking, so as to solve the defect that the GBDT security bucket aggregation in the prior art is not suitable for multi-party scenarios.

[0007] In a first aspect, the present invention provides a multi-party longitudinal GBDT secure bucket aggregation method based on secret sharing and masking, comprising: Obtain multiple gradient vector slice values ​​of multiple nodes and an indication vector of a feature owner node; Using multiple gradient vector slice values ​​and indicator vectors, the sample gradient and slice value of any node are obtained; The multiple nodes include multiple passive nodes, an active node and an auxiliary node, and any node among the multiple passive nodes is a feature owner node.

[0008] According to a multi-party longitudinal GBDT secure bucket aggregation method based on secret sharing and masking provided by the present invention, before obtaining multiple gradient vector slice values ​​of multiple nodes and the indicator vector of the feature owner node, it also includes: Determine that multiple passive nodes and active nodes participate in model training, and auxiliary nodes assist in model training; Obtain the number of samples. The active node calculates the total gradient vector based on the label value, predicted value, and number of samples. The active node shards the total gradient vector and shares it secretly with multiple passive nodes. It also removes the gradient vector shard value corresponding to the feature owner node from the total gradient vector and sends it to the auxiliary node. According to the number of samples, the indicator vector of the feature owner node is determined.

[0009] According to a multi-party vertical GBDT secure bucket aggregation method based on secret sharing and masking provided by the present invention, the feature owner node uses a random number generator to generate a random number vector, the number of random number vectors is equal to the number of samples, and calculates the sum of the indicator vector and the random number vector; The feature owner node sends the sum of the indication vector and the random number vector to the remaining passive nodes and the active node respectively, and sends the random number vector to the auxiliary node.

[0010] According to a multi-party vertical GBDT secure bucket aggregation method based on secret sharing and masking provided by the present invention, multiple nodes excluding the feature owner node respectively multiply the gradient vector shard value of each node by the sum of the indicator vector and the random number vector to obtain the gradient and shard of each node; The feature owner node uses the indicator vector and the gradient vector slice value of the feature owner node to obtain the feature owner node gradient and slice.

[0011] According to a multi-party vertical GBDT secure bucket aggregation method based on secret sharing and masking provided by the present invention, the auxiliary node multiplies the negative of a random number vector by the sum of the gradient vector shard values ​​of multiple nodes excluding the feature owner node to obtain the auxiliary node gradient and shard; Share worker node gradients and shard secrets to multiple passive and active nodes.

[0012] A multi-party vertical GBDT secure bucket aggregation method based on secret sharing and masking provided by the present invention also includes: Add the auxiliary node gradients and shards, each node gradients and shards, and feature owner node gradients and shards to obtain the product of the indicator vector and the total gradient vector to complete the correctness proof.

[0013] According to a multi-party vertical GBDT secure bucket aggregation method based on secret sharing and masking provided by the present invention, multiple nodes include floating point numbers and fixed point numbers.

[0014] In a second aspect, the present invention also provides a multi-party vertical GBDT secure bucket aggregation system based on secret sharing and masking, comprising: An input module, used for obtaining multiple gradient vector slice values ​​of multiple nodes and an indication vector of a feature owner node; An output module, used to obtain a sample gradient and a slice value of any node using a plurality of gradient vector slice values ​​and an indicator vector; The multiple nodes include multiple passive nodes, an active node and an auxiliary node, and any node among the multiple passive nodes is a feature owner node.

[0015] In a third aspect, the present invention also provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, the multi-party vertical GBDT secure bucket aggregation method based on secret sharing and masking as described in any one of the above is implemented.

[0016] In a fourth aspect, the present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements a multi-party vertical GBDT secure bucket aggregation method based on secret sharing and masking as described in any one of the above.

[0017] The multi-party vertical GBDT secure bucket aggregation method based on secret sharing and mask provided by the present invention effectively avoids the expensive homomorphic encryption overhead by proposing a multi-party vertical GBDT secure bucket aggregation calculation method. By using masks to protect the original data, it can be easily expanded to scenarios with multiple participants, thereby significantly improving the performance of privacy-preserving GBDT training on distributed large-scale data sets. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] In order to more clearly illustrate the technical solutions in the present invention or the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0019] Figure 1 It is a flow chart of a multi-party vertical GBDT secure bucket aggregation method based on secret sharing and masking provided by the present invention; Figure 2 It is an embodiment diagram of the multi-party vertical GBDT secure bucket aggregation method based on secret sharing and masking provided by the present invention; Figure 3 It is a structural schematic diagram of a multi-party vertical GBDT secure bucket aggregation system based on secret sharing and masking provided by the present invention; Figure 4 It is a structural schematic diagram of the electronic device provided by the present invention. DETAILED DESCRIPTION

[0020] In order to make the purpose, technical solution and advantages of the present invention clearer, the technical solution of the present invention will be clearly and completely described below in conjunction with the drawings of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0021] When performing GBDT secure bucket aggregation, the existing technology usually relies on the computationally expensive homomorphic encryption (HE) scheme to improve communication efficiency, or is only applicable to a two-party environment. In response to the above two challenges, the present invention proposes an innovative solution that is not only applicable to multi-party vertical GBDT secure bucket aggregation calculations, but also effectively avoids the expensive homomorphic encryption overhead. By using masks to protect the original data, the solution of the present invention can be easily extended to scenarios with multiple participants, thereby significantly improving the performance of privacy-preserving GBDT training on distributed large-scale datasets.

[0022] Figure 1 : is a flow chart of a multi-party vertical GBDT secure bucket aggregation method based on secret sharing and masking provided in an embodiment of the present invention, such as Figure 1 As shown, including: Step 100: obtaining multiple gradient vector slice values ​​of multiple nodes and an indication vector of a feature owner node; Step 200: using multiple gradient vector slice values ​​and indicator vectors, obtaining a sample gradient and a slice value of any node; The multiple nodes include multiple passive nodes, an active node and an auxiliary node, and any node among the multiple passive nodes is a feature owner node.

[0023] Specifically, since GBDT is a method widely used in the field of machine learning, inspired by existing technologies, the present invention proposes a multi-party vertical GBDT secure bucket aggregation algorithm based on secret sharing and masking. Assuming that all participants are honest and curious, the algorithm can ensure data privacy while effectively improving computing efficiency in an environment with the participation of multiple parties.

[0024] Take the three-party scenario as an example to describe the principle of the algorithm. Figure 2 As shown: Assume that there are three participants in the training of the model, and the participants are denoted as , , ,in , Is the passive party, Is the active party, the existing auxiliary party Responsible for assisting the training process. The number of samples is N. To grasp the tag value, you need Calculate the gradient vector based on the label value and the predicted value And share its secret with all participants, ,in The obtained gradient vector slice value is , The obtained gradient vector slice value is , The obtained gradient vector slice value , in particular, Will The value is sent to For one of the tree nodes, suppose is the feature owner, i.e. Known indicator vector .

[0025] enter: The gradient vector slice value of , The gradient vector slice value of and indicator vector , The gradient vector slice value of , The gradient vector slice value of .

[0026] Output: The indicator vectors are With the gradient vector The inner product value of 's shard, that is, the shard of the sample gradient and 's of this node.

[0027] Furthermore, based on the following settings: (1) Feature Owner Generate using a random number generator Random numbers , and calculate ; (2) Will The value is sent to and ,Will The value is sent to the helper ; (3) Participants and Calculate the gradient and the slices separately and ; (4) Participants Computing gradients and sharding ; (5) Auxiliary party Computing gradients and sharding If you do not want the auxiliary party to participate in the subsequent calculations, you can Secret sharing with participants , , .

[0028] Finally, the correctness is proved:

[0029] It is understandable that in actual development and application, it is also necessary to convert between floating-point numbers and fixed-point numbers according to known methods.

[0030] In particular, for more participants, the active participant only needs to sum up the gradient slices except the gradient slice of the feature owner and send it to the auxiliary party. That is, the other steps remain unchanged, so the algorithm has good scalability and avoids expensive homomorphic encryption.

[0031] Based on the above algorithm, GBDT training and reasoning can be performed by referring to known methods.

[0032] The multi-party vertical GBDT security bucket aggregation system based on secret sharing and masking provided by the present invention is described below. The multi-party vertical GBDT security bucket aggregation system based on secret sharing and masking described below and the multi-party vertical GBDT security bucket aggregation method based on secret sharing and masking described above can be referenced to each other.

[0033] Figure 3 is a structural diagram of a multi-party vertical GBDT secure bucket aggregation system based on secret sharing and masking provided by an embodiment of the present invention, such as Figure 3 As shown, it includes: an input module 31 and an output module 32, wherein: The input module 31 is used to obtain multiple gradient vector slice values ​​of multiple nodes and the indicator vector of the feature owner node; the output module 32 is used to use the multiple gradient vector slice values ​​and the indicator vector to obtain the sample gradient and slice value of any node; wherein the multiple nodes include multiple passive nodes, an active node and an auxiliary node, and any node among the multiple passive nodes is a feature owner node.

[0034] Figure 4 An example of a physical structure diagram of an electronic device is shown in FIG. Figure 4As shown, the electronic device may include: a processor 410, a communication interface 420, a memory 430 and a communication bus 440, wherein the processor 410, the communication interface 420 and the memory 430 communicate with each other through the communication bus 440. The processor 410 may call the logic instructions in the memory 430 to execute a multi-party vertical GBDT secure bucket aggregation method based on secret sharing and masking, the method comprising: obtaining multiple gradient vector shard values ​​of multiple nodes and an indication vector of a feature owner node; using multiple gradient vector shard values ​​and indication vectors, obtaining a sample gradient and shard value of any node; wherein the multiple nodes include multiple passive nodes, an active node and an auxiliary node, and any of the multiple passive nodes is a feature owner node.

[0035] In addition, the logic instructions in the above-mentioned memory 430 can be implemented in the form of a software functional unit and can be stored in a computer-readable storage medium when it is sold or used as an independent product. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art or the part of the technical solution, can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk, etc. Various media that can store program codes.

[0036] On the other hand, the present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon. When the computer program is executed by a processor, it is implemented to execute the multi-party vertical GBDT secure bucket aggregation method based on secret sharing and masking provided by the above-mentioned methods. The method includes: obtaining multiple gradient vector slice values ​​of multiple nodes and an indicator vector of a feature owner node; using the multiple gradient vector slice values ​​and the indicator vector to obtain the sample gradient and slice value of any node; wherein the multiple nodes include multiple passive nodes, an active node and an auxiliary node, and any node among the multiple passive nodes is a feature owner node.

[0037] The device embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. Ordinary technicians in this field can understand and implement it without paying creative labor.

[0038] Through the description of the above implementation methods, those skilled in the art can clearly understand that each implementation method can be implemented by means of software plus a necessary general hardware platform, and of course, can also be implemented by hardware. Based on this understanding, the above technical solution is essentially or the part that contributes to the prior art can be embodied in the form of a software product, and the computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a disk, an optical disk, etc., including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or some parts of the embodiments.

[0039] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A multi-party vertical GBDT secure bucket aggregation method based on secret sharing and masking, characterized in that: include: Obtain multiple gradient vector slice values ​​of multiple nodes and an indication vector of a feature owner node; Using multiple gradient vector slice values ​​and indicator vectors, the sample gradient and slice value of any node are obtained; The multiple nodes include multiple passive nodes, an active node and an auxiliary node, and any node among the multiple passive nodes is a feature owner node.

2. The multi-party vertical GBDT secure bucket aggregation method based on secret sharing and masking according to claim 1 is characterized in that: Before obtaining multiple gradient vector slice values ​​of multiple nodes and the indicator vector of the feature owner node, it also includes: Determine that multiple passive nodes and active nodes participate in model training, and auxiliary nodes assist in model training; Obtain the number of samples. The active node calculates the total gradient vector based on the label value, predicted value, and number of samples. The active node shards the total gradient vector and shares it secretly with multiple passive nodes. It also removes the gradient vector shard value corresponding to the feature owner node from the total gradient vector and sends it to the auxiliary node. According to the number of samples, the indicator vector of the feature owner node is determined.

3. The multi-party vertical GBDT secure bucket aggregation method based on secret sharing and masking according to claim 2 is characterized in that: The feature owner node uses a random number generator to generate a random number vector, the number of which is equal to the number of samples, and calculates the sum of the indicator vector and the random number vector; The feature owner node sends the sum of the indication vector and the random number vector to the remaining passive nodes and the active node respectively, and sends the random number vector to the auxiliary node.

4. The multi-party vertical GBDT secure bucket aggregation method based on secret sharing and masking according to claim 3 is characterized in that: For multiple nodes except the feature owner node, the sum of the indicator vector and the random number vector is multiplied by the gradient vector slice value of each node to obtain the gradient and slice of each node; The feature owner node uses the indicator vector and the gradient vector slice value of the feature owner node to obtain the feature owner node gradient and slice.

5. The multi-party vertical GBDT secure bucket aggregation method based on secret sharing and masking according to claim 4 is characterized in that: The auxiliary node multiplies the negative of the random number vector by the sum of the gradient vector slice values ​​of multiple nodes excluding the feature owner node to obtain the auxiliary node gradient and slice; Share worker node gradients and shard secrets to multiple passive and active nodes.

6. The multi-party vertical GBDT secure bucket aggregation method based on secret sharing and masking according to claim 5 is characterized in that: Also includes: Add the auxiliary node gradients and shards, each node gradients and shards, and feature owner node gradients and shards to obtain the product of the indicator vector and the total gradient vector to complete the correctness proof.

7. The multi-party vertical GBDT secure bucket aggregation method based on secret sharing and masking according to any one of claims 1 to 6, characterized in that: Several nodes include floating point and fixed point numbers.

8. A multi-party vertical GBDT secure bucket aggregation system based on secret sharing and masking, characterized in that: include: An input module, used for obtaining multiple gradient vector slice values ​​of multiple nodes and an indication vector of a feature owner node; An output module, used to obtain a sample gradient and a slice value of any node using a plurality of gradient vector slice values ​​and an indicator vector; The multiple nodes include multiple passive nodes, an active node and an auxiliary node, and any node among the multiple passive nodes is a feature owner node.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the program, the multi-party vertical GBDT secure bucket aggregation method based on secret sharing and masking is implemented as described in any one of claims 1 to 7.

10. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the multi-party vertical GBDT secure bucket aggregation method based on secret sharing and masking is implemented as described in any one of claims 1 to 7.

Images