Gradient boosting decision tree vertical federal learning method, electronic equipment and storage medium

Abstract

The invention provides a gradient boosting decision tree vertical federal learning method, electronic equipment and a storage medium, and the method comprises the steps: providing two participants, and employing Paillier homomorphic encryption to carry out the encryption aggregation of gradient secret sharing shares; the weight of each leaf node is calculated, the weights of the leaf nodes are accumulated, and model output is updated; a function approximation method is adopted to process Sigmoid calculation in training, and Goldschmidt sequence expansion is adopted to process division calculation in the training process; and returning a complete model list of the gradient boosting decision tree model. According to the method, on the basis of completely protecting original data and feature distribution privacy of participants, the model training efficiency is remarkably improved, and the precision and generalization of the training model can be ensured.

Description

Technical Field

[0001] The present invention relates to the technical fields of multi-party secure computing and machine learning privacy protection. Specifically, it relates to a vertical federated learning method for gradient boosting decision trees, an electronic device, and a storage medium. Background Art

[0002] With the increasing demand for data privacy protection and the increasing complexity of machine learning models, privacy-preserving XGBoost (eXtreme Gradient Boosting) in the vertical federated scenario faces various challenges. Existing works (such as SecureBoost, SecureGBM) have proposed privacy-preserving XGBoost training methods for vertically partitioned data, but they can only protect the global gradient information, and there are problems such as leakage of intermediate statistics (such as split gain order, first-order gradient cumulative sum, and second-order gradient cumulative sum), low efficiency of non-linear calculations (such as division and Sigmoid functions), and excessive communication overhead for massive data, resulting in existing privacy protection solutions being difficult to balance privacy security and computing performance in large-scale practical scenarios. Summary of the Invention

[0003] Aiming at the defects in the prior art, the purpose of the present invention is to provide a vertical federated learning method for gradient boosting decision trees, an electronic device, and a storage medium. The method of the present invention significantly improves the efficiency of model training on the basis of completely protecting the privacy of the original data and feature distribution of the participating parties, and can ensure the accuracy and generalization of the trained model.

[0004] To solve the above problems, the technical solution of the present invention is as follows:

[0005] A vertical federated learning method for gradient boosting decision trees includes the following steps:

[0006] Provide two participating parties, and use Paillier homomorphic encryption to encrypt and aggregate the gradient secret sharing shares;

[0007] Calculate the weights for each leaf node, accumulate the weights of the leaf nodes, and update the model output;

[0008] Use the method of function approximation to process the Sigmoid calculation in training, and use the Goldschmidt sequence expansion to process the division calculation in the training process;

[0009] Return the model list of the complete gradient boosting decision tree model.

[0010] Preferably, the step of providing two participating parties and using Paillier homomorphic encryption to encrypt and aggregate the gradient secret sharing shares specifically includes: providing two participating parties P e and P 1-e, two participating parties P e and P 1-e respectively input the feature matrices and as well as the label vector y, call the secret sharing scheme, and initialize the state <g> e is the first-order gradient of the sample, <h> e is the second-order gradient of the sample, is the predicted value, where e ∈ {0, 1}, and the public parameters pp = {D > 0, B > 0, γ > 0, pk eA , pk eH}; pp includes the maximum depth D of the tree, the number of buckets B for each feature, the regularization parameter γ, and the public key of Paillier homomorphic encryption; each participating party P e and P 1-e locally buckets the feature matrices X e and X 1-e respectively, generates the bucket matrices M e and M 1-e , and records the bucket to which each sample is classified according to its features; each participating party P e initializes the sample indicator All samples are located at the root node; meanwhile, P e initializes the first-order gradient <g (1) > e = <g> e and the second-order gradient <h (1) > e = <h> e , call the secret sharing scheme in the ring to additively decompose the first-order and second-order gradient values of l bits into the sum of two secret shares and secretly share them between parties P e and P 1-e .

[0011] Preferably, the steps of calculating weights for each leaf node, accumulating the weights of leaf nodes, and updating the model output specifically include:

[0012] If k is a left node, the parties jointly execute the binary matrix-vector multiplication protocol to complete the calculation of bucketed gradient aggregation and obtain the secret sharing value of bucketed gradient aggregation;

[0013] Each party locally concatenates its share to obtain the updated gradient aggregation value;

[0014] If k is a right node, each party locally calculates the updated gradient aggregation value;

[0015] The parties use the Beaver triple multiplication protocol based on secret sharing to jointly calculate the partition gain of all features and buckets

[0016] The parties use the maximum index protocol based on garbled circuits to jointly calculate the feature of the best split and the bucket number

[0017] The parties publicly disclose one bit to each other <e> B , which is used to determine which participating party has the best segmentation feature;

[0018] The best segmentation identifier is disclosed to the participating party P that has this feature e and written into E e [k], while the participating party P 1-e writes NULL (empty) into E 1-e [k];

[0019] The participating party P 1-e keeps its sample indicator unchanged, and the participating party P e locally updates its sample indicator according to the segmentation result;

[0020] The participating party P e uses a correlated oblivious transfer protocol to jointly calculate the first-order gradient <g (2k) > and the second-order gradient <h (2k) > of the left child node, while each participating party P e locally calculates the first-order gradient <g (2k+1) > and the second-order gradient <h (2k +1) > of the right child node.

[0021] Preferably, the binary matrix and vector multiplication protocol is adopted to securely calculate the product M·g of the matrix M and the vector g, specifically including:

[0022] The participating party P 1-e holds the arithmetic secret sharing shard of g <g> 1-e , Participant P e holds the arithmetic secret sharing shard of g, g e and the binary matrix M; P e generates a public-private key pair for homomorphic encryption and sends the public key to Participant P 1-e , and the private key is private to P e ;

[0023] Participant P 1-e uses the public key to encrypt its local share <g> 1-e Generate ciphertext And send the ciphertext to participant P e , P e Uses its own public key to encrypt its local share <g> e Generate ciphertext And send the ciphertext to party P 1-e ;

[0024] Party P e Initializes a homomorphic encryption ciphertext array of length B·m, with all ciphertexts initially encrypted as zero. According to all index positions (j,i) where M[j,i]=1, party P e Extracts a homomorphic encryption ciphertext from and performs homomorphic addition to update the ciphertext qt , finally obtaining the dot product of the j-th row of matrix M and g; j

[0025] Party P e Holds the homomorphic encrypted aggregated gradient ciphertext data P e Selects a random value as a mask and homomorphically encrypts it using the public key to obtain the encrypted value <r> H , P e Perform the homomorphic subtraction operation <M·g - r> H , and send the encrypted difference to the participant P 1-e , P 1-e Use the private key to decrypt the ciphertext to obtain the new shard share <M·g - r>.

[0026] Preferably, the local update sample indicator specifically includes:

[0027] Maintain a sample indication vector b for each tree node k (k) , b (k) Record whether each sample belongs to the current node, and use the secret sharing scheme to share it between the participant P e and the participant P 1-e ;

[0028] Participant P e Construct an indication vector according to the best split point determined by the foregoing step protocol If the eigenvalue of the sample is greater than the threshold, the corresponding position of the indication vector is 1, otherwise it is 0; Let e = 0 to verify the correctness of the indicator update and ensure that the update is logical.

[0029] Preferably, the participant uses a joint computing protocol based on correlated oblivious transfer

[0030] to calculate the first-order gradient and second-order gradient of the left and right child nodes, specifically including: Define the updated first-order gradient vector g

[0031] and the second-order gradient vector h (2k) and perform element-wise multiplication to obtain a private selection vector; (2k) In the joint computing protocol

[0032] P as the sender, holds the secret sharing vector g e and the private selection vector (k) P as the receiver holds the selection bit 1-e and securely obtains the corresponding g [i]; (k)

[0033] The receiver P 1-e calculates the gradient update through the correlated oblivious transfer protocol COT.

[0034] Preferably, the steps of using the method of function approximation to process the Sigmoid calculation in training and using the Goldschmidt sequence expansion to process the division calculation in the training process specifically include:​ Split it into a three - segment form. The two linear segments on both sides adopt fixed values, and the middle segment is approximated by Fourier series and the error is dynamically adjusted.

[0035] Preferably, the step of using the Goldschmidt sequence expansion to process the division calculation in the training process specifically includes: gradually approaching the division result through a recurrence relation and controlling the error to reach a predetermined accuracy.

[0036] Furthermore, the present invention also provides an electronic device, including a processor and a memory for storing executable instructions of the processor. The processor is configured to execute the vertical federated learning method of the gradient - boosting decision tree as described above by executing the executable instructions.

[0037] Furthermore, the present invention also provides a computer - readable storage medium. The computer - readable storage medium is used to store program codes, and the program codes are used to execute the vertical federated learning method of the gradient - boosting decision tree as described above.

[0038] Compared with the prior art, the beneficial effects of the present invention are as follows:

[0039] 1. At the system architecture level, the present invention abstracts the XGBoost model training into a verifiable secure computing function, adopts a two - party collaborative computing model with vertical data partitioning, breaks through the bottleneck of traditional federated learning in privacy computing and efficiency. Through a hybrid computing protocol that combines optimized Paillier homomorphic encryption and secret sharing, it not only realizes the secure update and aggregation of gradient information, but also hides the sample distribution by designing an efficient oblivious transfer mechanism and a local indicator vector invariance update mechanism, further protecting the privacy of feature bucket statistics and significantly reducing the computational interaction overhead.

[0040] 2. At the computational optimization level, aiming at the computational bottleneck of homomorphic encryption, the Montgomery modular multiplication optimization algorithm and the ciphertext batch packaging technology are introduced, which improves the efficiency of modular exponentiation operation by 40% and reduces the communication overhead to 20% of the traditional method, effectively alleviating the performance pressure of encryption operations in a distributed environment; for non - linear operation units, a division optimization algorithm based on Goldschmidt iteration is introduced, which transforms the division operation into a non - interactive multiplication sequence, not only eliminating communication dependence, but also controlling the computational error within the order of magnitude of 10 -6 ; at the same time, a three - segment Fourier approximation Sigmoid algorithm is designed, combined with a secure comparison protocol based on secret sharing and the related Correlated Oblivious Transfer (COT) technology to achieve high - precision activation function calculation with an error lower than 2.2×10 -2 .

[0041] 3. At the privacy protection level, the present invention adopts a hierarchical encryption strategy: the Paillier homomorphic encryption is used at the bottom layer to protect the gradient information, the secret sharing is adopted at the middle layer to ensure the security of the calculation process, and the function approximation method is adopted at the top layer to process sensitive non-linear operations. This multi-layer protection mechanism ensures that all intermediate information (including the segmentation gain value and its order) remains encrypted except for the final model output. It effectively solves the bottleneck problems of the existing secure XGBoost training scheme in terms of practicability and scalability, and provides a new technical path for privacy-protected distributed machine learning. Brief Description of the Drawings

[0042] By reading the detailed description of the non-restrictive embodiments with reference to the following drawings, other features, objects, and advantages of the present invention will become more apparent:

[0043] Figure 1 It is a flowchart of the vertical federated learning method for gradient boosting decision trees of the present invention;

[0044] Figure 2 It is a schematic diagram of the invariance indication update of the present invention;

[0045] Figure 3 It is a structural diagram of the electronic device according to the embodiment of the present invention;

[0046] Figure 4 It is a structural diagram of the computer-readable storage medium according to the embodiment of the present invention. Detailed Embodiments

[0047] The present invention will be described in detail below with reference to specific embodiments. The following embodiments will help those skilled in the art to further understand the present invention, but do not limit the present invention in any form. It should be noted that those of ordinary skill in the art can make several changes and improvements without departing from the concept of the present invention. These all belong to the protection scope of the present invention.

[0048] Specifically, the present invention provides a vertical federated learning method for gradient boosting decision trees, as Figure 1 and Figure 2 shown. The method includes the following steps:

[0049] S1: Provide two participating parties, and encrypt and aggregate the gradient secret sharing shares by using Paillier homomorphic encryption;

[0050] Specifically, provide two participating parties P e and P 1-e , and the two participating parties P e and P 1-e respectively input the feature matrices and and the label vector y. Call the secret sharing scheme to initialize the state g> e is the first-order gradient of the sample, <h> e is the second-order gradient of the sample, is the predicted value, where e ∈ {0, 1}, and the public parameters pp = {D > 0, B > 0, γ > 0, pk eA , pk eH}, pp includes the maximum depth D of the tree, the number of buckets B for each feature, the regularization parameter γ, and the public key of Paillier homomorphic encryption.

[0051] Each participating party P e and P 1-e respectively perform local bucketing on the feature matrices X e and X 1-e to generate bucketing matrices M e and M 1-e , and record the bucket into which each sample is classified according to its features. Each participating party P e initializes the sample indicator All samples are located at the root node; meanwhile, P e initializes the first-order gradient <g (1) > e = <g> e and second-order gradient <h (1) > e = <h> e , call the secret sharing scheme in the ring to additively decompose the l-bit first-order and second-order gradient values into the sum of two secret shares, and secretly share them between parties P e and P 1-e .

[0052] S2: Calculate the weights for each leaf node, accumulate the weights of the leaf nodes, and update the model output;

[0053] Specifically, for each leaf node k, calculate the weight <ω k , accumulate the weights of the leaf nodes, and update the model output;

[0054] For each leaf node k of each tree, perform the following steps:

[0055] S21: If k is a left node, the parties jointly execute the binary matrix-vector multiplication protocol to complete the bucketing gradient aggregation calculation and obtain the secret sharing value of the bucketing gradient aggregation;

[0056] In step S21, the binary matrix-vector multiplication protocol is used to securely calculate the product M·g of the matrix M and the vector g, specifically:

[0057] S211. Party P 1-e holds the arithmetic secret sharing shard of g <g> 1-e , Participant P e holds an arithmetic secret sharing shard of g <g> e and binary matrix M; P e Generate a public-private key pair for homomorphic encryption and send the public key to participant P 1-e , and the private key is held by P e Private;

[0058] S212. Participant P 1-e Use the public key to encrypt its local share <g> 1-e Generate ciphertext And send the ciphertext to participant P e , P e Uses its own public key to encrypt its local share <g> e Generate ciphertext And send the ciphertext to participant P 1-e ;

[0059] S213. Participant P e Initialize a homomorphic encryption ciphertext array of length B·m, with all ciphertexts initially encrypted to zero. According to all index positions (j,i) where M[j,i]=1, participant P e Extract a homomorphic encryption ciphertext from and perform homomorphic addition to update the ciphertext qt to finally obtain the dot product of the j-th row of matrix M and g; j The dot product of the j-th row of matrix M and g is obtained;

[0060] S214. Participant P e Holds the homomorphically encrypted aggregated gradient ciphertext data P e Select a random value as a mask and homomorphically encrypt it using the public key to obtain the encrypted value <r> H , P e Perform the homomorphic subtraction operation <M·g - r> H , and send the encrypted difference to the participant P 1-e , P 1-e Use the private key to decrypt the ciphertext to obtain the new shard share <M·g - r>.

[0061] S22: Each participant locally splices its share to obtain the updated gradient aggregation value;

[0062] S23: If k is a right node, each participant locally calculates the updated gradient aggregation value;

[0063] S24: The participants use the Beaver triple multiplication protocol based on secret sharing to jointly calculate the division gains of all features and buckets

[0064] S25: The participants use the maximum index protocol based on garbled circuits to jointly calculate the optimal split feature and the bucketing serial number

[0065] S26: The participants publicly disclose a bit to each other <e> B , for determining which participating party has the best splitting feature;

[0066] S27: Make the best splitting identifier public to the participating party P that owns the feature, e and write it into E e [k], while the participating party P 1-e writes NULL (empty) into E 1-e [k];

[0067] S28: The participating party P 1-e keeps its sample indicator unchanged, and the participating party P e locally updates its sample indicator according to the splitting result;

[0068] In step S28, the local update of the sample indicator is specifically:

[0069] S281. Maintain a sample indication vector b (k) for each tree node k, (k) b e records whether each sample belongs to the current node, and uses a secret sharing scheme to share it between the participating party P 1-e and the participating party P;

[0070] S282. The participating party P e constructs an indication vector according to the best splitting point determined by the protocol in the foregoing steps If the feature value of the sample is greater than the threshold, the corresponding position of the indication vector is 1, otherwise it is 0;

[0071] S283. Let e = 0 to verify the correctness of the indicator update and ensure that the update is logical.

[0072] S29: The participating party P e uses a joint calculation protocol based on related oblivious transfer to jointly calculate the first-order gradient <g (2k) > and the second-order gradient <h (2k) > of the left child node, and at the same time each participating party p e locally calculates the first-order gradient <g (2k+1) > and the second-order gradient <h (2k+1) > of the right child node.

[0073] In step S29, the participating party uses a joint calculation protocol based on related oblivious transfer to calculate the first-order gradient and the second-order gradient of the left and right child nodes, specifically:

[0074] S291. Define the updated first-order gradient vector g (2k) ​and the second-order gradient vector h (2k) , perform element-wise multiplication to obtain the private selection vector;

[0075] S292. In the joint calculation protocol , P e acts as the sender and holds the secret sharing vector g (k) and the private selection vector P 1-e acts as the receiver and holds the selection bit to securely obtain the corresponding g (k) [i];

[0076] S293. The receiver P 1-e calculates the gradient update through the relevant oblivious transfer protocol COT.

[0077] S3: Use the method of function approximation to handle the Sigmoid calculation in training, and use the Goldschmidt sequence expansion to handle the division calculation in the training process;

[0078] Specifically, when using the logarithmic loss function for calculation, use an efficient and high-precision piecewise approximation to calculate the non-linear Sigmoid function, specifically:

[0079] Split into a three-segment form, that is where the linear segments on both sides directly take the fixed values of σ(-θ) or σ(θ) to avoid overly complex calculations, and the middle segment uses the Jth-order Fourier series for approximation, ω j ∈[-1,1] is the preset Fourier coefficient, and the approximation error can be adjusted to a smaller value by dynamically selecting θ and J.

[0080] For the Paillier homomorphic encryption used in all steps, Montgomery modular multiplication optimization is adopted, specifically: pre-compute the modular inverse element R of the constant R -1 modN, calculated by the extended Euclidean algorithm, perform the Montgomery modular multiplication operation and reduce the computational complexity.

[0081] For the Paillier homomorphic encryption algorithm, in the encryption stage, multiple plaintexts are packed into one for encryption, specifically: taking k plaintexts as a group, concatenate{m i} i＝1,2,…,k to obtain m = m k ||m k-1 ||…||m1. After packing optimization, the ciphertext size and the time consumption of the modular exponentiation in encryption and decryption are reduced to 1 / k of the original;

[0082] The division calculation in the training process is processed by expanding with the Goldschmidt sequence, specifically as follows:

[0083] Let w0 be the initial approximation of 1 / b, with a relative error of φ0 < 1, and let a0 = a, b0 = b. For i ≥ 1, calculate:

[0084] a i = a i-1 w i-1

[0085] b i = b i-1 w i-1

[0086] w i = 2 - b i

[0087] Let be expressed as r i , then there is:

[0088]

[0089] The relative error of the initial approximation is ∈0 = 1 - bw0. It can be proved by induction that and If ∈0 < 1, then b i converges to 1, so a i converges to the quotient a / b, and r i converges to the reciprocal 1 / b. Let be expressed as ψ i , and the recurrence relation of the approximate quotient can be written as:

[0090] a1 = aw0

[0091] a i+1 = a i (1 + ψ i-1 )

[0092]

[0093] After i iterations, we get So a i+1 ≈ a / b, with a relative error of A similar recurrence relation for 1 / b can also be obtained.

[0094] S4: Return the model list of the complete gradient boosting decision tree model.

[0095] Specifically, finally return a model list Model_list of the complete gradient boosting decision tree model.

[0096] Based on the method of the present invention, in Figure 2 Figure 2 shows a schematic diagram of a case using invariance update.

[0097] In summary, the present invention proposes a privacy protection method and device for gradient boosting decision tree vertical federated learning based on the optimization of multi-party secure computing processes. In the XGBoost training participated by two parties, by converting each step gradient represented under the secret sharing scheme into the representation under the homomorphic encryption scheme, and using homomorphic addition calculation, the communication complexity during the gradient bucket cumulative summation is reduced, and the privacy and computing efficiency of the two-party secure XGBoost training are improved. By using technologies such as Montgomery modular multiplication optimization and ciphertext packing encryption, the computing efficiency of homomorphic encryption is optimized, and thus the computing efficiency of training is indirectly improved. Fourier function approximation and Goldschmidt coefficient expansion are adopted to handle the computing overhead and adverse effects brought by the non-linear computing primitive Sigmoid and the confidential division calculation. This method has a certain generalization effect, can enhance the model accuracy and achieve efficient training.

[0098] According to another aspect of the embodiments of the present application, the present invention also provides an electronic device, as Figure 3 shown, the electronic device 200 may include a first processor 201, a first memory 202 and a bus, and may also include a computer program stored in the first memory 202 and executable on the first processor 201, such as a program 203 for a high-precision gradient boosting decision tree vertical federated learning method based on multi-party secure computing.

[0099] The electronic device 200 supports multi-party secure computing in a multi-node environment. Multiple independent nodes exchange data through a security protocol, and each node can only access local data and cannot directly obtain the data of other nodes. The method ensures that private information of each party is not leaked during data transmission and calculation through secret sharing, homomorphic encryption, oblivious transfer, and garbled circuit technologies. Each node trains a partial model according to local data and sends the calculation result to other nodes through a security protocol. All nodes jointly participate in the model training process without exchanging the original data. The computer program instructions are configured to implement cross-node data sharing and computing collaboration, ensuring effective and secure vertical federated learning of gradient boosting decision trees among multiple participating parties.

[0100] According to still another aspect of the embodiments of the present application, the present invention also provides a computer-readable storage medium, as Figure 4 shown, a storage medium 100, 103 is also provided between two participating parties. Programs are stored in memories 101, 104 and are executed by processors 102, 105 to implement a privacy protection method for gradient boosting decision tree vertical federated learning based on the optimization of multi-party secure computing processes.

[0101] The computer-readable storage medium can be any medium capable of storing computer programs, such as a hard disk drive, a solid-state drive, a memory module, or other devices suitable for storing programs and data. The computer programs in the storage medium include multiple modules for performing processes including but not limited to data preprocessing, model training, encrypted communication, computing collaboration, and result aggregation.

[0102] The execution device can be a CPU (Central Processing Unit), a general-purpose processor, a DSP (Digital Signal Processor), an ASIC (Application Specific Integrated Circuit), an FPGA (Field Programmable Gate Array), or other programmable logic devices, transistor logic devices, hardware components, or any combination thereof. It can implement or execute various exemplary logical blocks, modules, and circuits described in connection with the disclosure of this application. The execution device can also be a combination that implements computing functions, such as a combination including one or more microprocessors, a combination of a DSP and a microprocessor, etc.

[0103] Those skilled in the art should understand that the embodiments of the present invention can be provided as a method, a device, or a computer program product. Therefore, the present invention can take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware aspects. Moreover, the present invention can take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0104] The present invention is described with reference to the flowcharts and / or block diagrams of methods, terminal devices (systems), and computer program products according to the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, and the combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to the processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing terminal devices to generate a machine, such that the instructions executed by the processor of the computer or other programmable data processing terminal devices generate a device for implementing the functions specified in Figure 1 one process or multiple processes and / or blocks Figure 1 one block or multiple blocks.

[0105] These computer program instructions can also be stored in a computer-readable memory that can direct a computer or other programmable data processing terminal device to work in a specific manner, such that the instructions stored in the computer-readable memory produce a manufacture including an instruction device that implements the functions specified in one or more of the processes and / or blocks Figure 1 in one or more of the processes and / or blocks Figure 1 specified in the block or blocks.

[0106] These computer program instructions can also be loaded onto a computer or other programmable data processing terminal device, such that a series of operational steps are performed on the computer or other programmable terminal device to produce a computer-implemented process, whereby the instructions executed on the computer or other programmable terminal device provide steps for implementing the functions specified in one or more of the processes and / or blocks Figure 1 in one or more of the processes and / or blocks Figure 1 specified in the block or blocks.

[0107] The specific embodiments of the present invention have been described above. It should be understood that the present invention is not limited to the above specific embodiments, and those skilled in the art can make various changes or modifications within the scope of the claims, which do not affect the essence of the present invention. Without conflict, the embodiments of the present application and the features in the embodiments can be combined with each other arbitrarily.< / e> < / r> < / g> < / g> < / g> < / g> < / h> < / g> < / h> < / r> ​< / g> < / g> < / g> < / e> < / h> < / g> < / h> < / g>

Claims

1. A vertical federated learning method for gradient boosting decision trees, characterized in that, The method includes the following steps: Provide two participating parties, and use Paillier homomorphic encryption to encrypt and aggregate the gradient secret sharing shares; Calculate the weights for each leaf node, accumulate the weights of the leaf nodes, and update the model output; Use the method of function approximation to handle the Sigmoid calculation in training, and use the Goldschmidt sequence expansion to handle the division calculation in the training process; Return the model list of the complete gradient boosting decision tree model.

2. The gradient boosting decision tree vertical federated learning method according to claim 1, wherein The step of providing two participating parties and encrypting and aggregating the gradient secret sharing shares using Paillier homomorphic encryption specifically includes: providing two participating parties P e and P 1-e , two participating parties P e and P 1-e respectively input the feature matrices and and the label vector y, call the secret sharing scheme, and initialize the state state = <g> e is the first-order gradient of the sample, <h> e is the second-order gradient of the sample, is the predicted value, where e ∈ {0, 1}, and the public parameters pp = {D > 0, B > 0, γ > 0, pk eA , pk eH}; pp includes the maximum depth D of the tree, the number of buckets B for each feature, the regularization parameter γ, and the public key of Paillier homomorphic encryption; each participant P e and P 1-e respectively perform local bucketing on the feature matrices X e and X 1-e to generate bucketing matrices M e and M 1-e , and record the buckets into which each sample is classified according to its features; each participant p e initializes the sample indicator All samples are located at the root node; meanwhile, P e initializes the first-order gradient <g (1) > e = <g> e and second-order gradient <h (1) > e = <h> e , call the secret sharing scheme in the ring to additively decompose the l-bit first- and second-order gradient values into the sum of two secret shares, and secretly share them between the participating parties P e and P 1-e .< / h> < / g> < / h> < / g> 3. The gradient boosting decision tree vertical federated learning method according to claim 1, wherein The step of calculating the weights for each leaf node, accumulating the weights of the leaf nodes, and updating the model output specifically includes: If k is the left node, the participating parties jointly execute the binary matrix and vector multiplication protocol to complete the bucketed gradient aggregation calculation and obtain the secret sharing value of the bucketed gradient aggregation; Each participating party locally splices its share to obtain the updated gradient aggregation value; If k is the right node, each participating party locally calculates the updated gradient aggregation value; The participating parties use the Beaver triple multiplication protocol based on secret sharing Jointly calculate the partitioning gains of all features and buckets The participating parties use the maximum-index finding protocol based on garbled circuits Jointly calculate the features of the optimal segmentation and the bucket number The parties publicly disclose a bit to each other <e> B , for determining which participating party has the best segmentation feature;< / e> Disclose the optimal segmentation identifier to the participating party P that has this feature e and write it to E e [k], while the participating party P 1-e writes NULL (empty) to E 1-e [k]; Participant P 1-e keeps its sample indicator unchanged, and Participant P e locally updates its sample indicator according to the segmentation result; Participant P e Use the relevant oblivious transfer protocol Jointly calculate the first-order gradient <g (2k) > and the second-order gradient <h (2k) > of the left child node, while each participant P e Locally calculate the first-order gradient <g (2k+1) > and the second-order gradient <h (2k+1) > of the right child node.

4. The gradient boosting decision tree vertical federated learning method according to claim 3, characterized in that The use of the binary matrix and vector multiplication protocol to securely calculate the product M·g of the matrix M and the vector g specifically includes: Party P 1-e Holds an arithmetic secret sharing shard of g <g> 1-e , Participant P e holds an arithmetic secret sharing shard of g <g> e and binary matrix M; P e Generate a public-private key pair for homomorphic encryption and send the public key to participant P 1-e , and the private key is held by P e Private;< / g> < / g> Participant P 1-e Encrypt its local share using the public key <g> 1-e Generate ciphertext And send the ciphertext to the participant P e , P e Use its own public key to encrypt its local share <g> e Generate ciphertext and send the ciphertext to participant P 1-e ;< / g> < / g> Participant P e Initialize a homomorphic encryption ciphertext array of length B·m, with all ciphertexts initially encrypted as zero. According to all index positions (j, i) where M[j, i] = 1, Participant P e Extract a homomorphic encryption ciphertext from and perform homomorphic addition to update the ciphertext qt j , finally obtaining the dot product of the j-th row of matrix M and g; Participant P e Holds the homomorphic encrypted aggregated gradient ciphertext data P e Selects a random value As a mask and homomorphically encrypts it using the public key to obtain an encrypted value <r> H , P e Perform the homomorphic subtraction operation <M·g - r> H , and send the encrypted difference to the participant P 1-e , P 1-e Use the private key to decrypt the ciphertext to obtain the new shard share <M·g - r>.< / r> 5. The gradient boosting decision tree vertical federated learning method according to claim 3, wherein The local update of the sample indicator specifically includes: Maintain a sample indicator vector \(b\) for each tree node \(k\). (k) , \(b\) (k) records whether each sample belongs to the current node, and is shared between parties \(P\) e and party \(P\) 1-e using a secret sharing scheme; Participant P e Optimal segmentation point determined according to the foregoing step protocol Construct an indication vector If the eigenvalue of the sample is greater than the threshold, the corresponding position of the indication vector is 1; otherwise, it is 0. Let e = 0 to verify the correctness of the indicator update and ensure that the update is logical.

6. The gradient boosting decision tree vertical federated learning method according to claim 3, wherein The participating party uses a joint computing protocol based on relevant oblivious transfer Calculate the first-order gradient and second-order gradient of the left and right child nodes, specifically including: Define the updated first-order gradient vector g (2k) and the second-order gradient vector h (2k) , perform element-wise multiplication to obtain the private selection vector; In the joint computing protocol P e acts as the sender and holds the secret sharing vector g (k) and the private selection vector P 1-e acts as the receiver and holds the selection bit to securely obtain the corresponding g (k) [i]; Receiver P 1-e Calculate the gradient update through the relevant oblivious transfer protocol COT.

7. The gradient boosting decision tree vertical federated learning method according to claim 1, characterized in that The steps of using the method of function approximation to process the Sigmoid calculation in training and using the Goldschmidt sequence expansion to process the division calculation in the training process specifically include: splitting into a three-segment form, using fixed values for the two side linear segments, and approximating the middle segment by Fourier series and dynamically adjusting the error.

8. The gradient boosting decision tree vertical federated learning method according to claim 1, characterized in that The step of using the Goldschmidt sequence expansion to handle the division calculation in the training process specifically includes: gradually approaching the division result through the recurrence relation and controlling the error to reach the predetermined accuracy.

9. An electronic device, characterized in that, The electronic device includes a processor and a memory for storing the executable instructions of the processor, and the processor is configured to execute the gradient boosting decision tree vertical federated learning method according to any one of claims 1-8 by executing the executable instructions.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium is used to store program code, and the program code is used to execute the gradient boosting decision tree vertical federated learning method according to any one of claims 1-8.

Images