An XGBoost training method and system based on multi-party privacy protection

Abstract

The application provides an XGBoost training method and system based on multi-party privacy protection, and belongs to the technical field of machine learning; a key pair is generated for each user by encoding feature values in a one-hot encoding mode; data is encrypted by using a single instruction multiple data method and uploaded to the cloud; for a partition scene corresponding to a data type of the encrypted data, the encrypted data of each user is aggregated to construct an encrypted data set; based on the encrypted training set, a secure training algorithm is used to generate an XGBoost model in a ciphertext state for encrypted prediction services; wherein the secure training algorithm comprises secure gain calculation, secure division calculation, secure sigmoid calculation and secure tree building calculation. The private data of multiple participants can be encrypted respectively, privacy leakage caused by direct use of plaintext data is avoided, and the encrypted data can be used for offline training after being uploaded, so that practicability and deployment flexibility are considered.

Description

Technical Field

[0001] This invention belongs to the field of machine learning technology, and in particular relates to an XGBoost training method and system based on multi-party privacy protection. Background Technology

[0002] The statements in this section are merely background information related to the present invention and do not necessarily constitute prior art.

[0003] In scenarios such as healthcare, financial risk control, and government collaboration, multiple institutions or users often possess a portion of data, and they hope to "pool" this data to train a more accurate model. However, due to privacy regulations, trade secrets, and other reasons, raw data cannot be directly exchanged or centralized in one place.

[0004] Moreover, existing privacy-preserving training solutions typically require repeated online collaboration and frequent communication among all parties during the training process. Some solutions even require a party to have higher privileges to lead the training or undertake decryption work, which leads to complex deployment, high barriers to entry, and the intermediate information generated during the training process may still pose a risk of data leakage. Summary of the Invention

[0005] To overcome the shortcomings of the prior art, this invention provides an XGBoost training method and system based on multi-party privacy protection, which can encrypt the private data of multiple participants, avoid privacy leakage caused by directly using plaintext data, and allow offline training after the encrypted data is uploaded, thus balancing practicality and deployment flexibility.

[0006] To achieve the above objectives, one or more embodiments of the present invention provide the following technical solutions: The first aspect of this invention provides an XGBoost training method based on multi-party privacy protection.

[0007] An XGBoost training method based on multi-party privacy protection includes: One-hot coding is used as a unified representation of feature ciphertext, and each user is uniquely encoded. Data is encrypted using a single instruction multiple data method based on each user's individual key pair and then uploaded to the cloud. For partition scenarios corresponding to the data type of encrypted data, the encrypted data of each user is aggregated in ciphertext to construct an encrypted dataset; Based on the encrypted training set, a secure training algorithm is used to generate a secure XGBoost model for encrypted prediction services; wherein the secure training algorithm includes secure gain calculation, secure partitioning calculation, secure division calculation, secure sigmoid calculation, and secure tree construction calculation.

[0008] Furthermore, during the encoding process, a global encoding dimension is determined based on the number of features and the number of categories for each feature; for each feature, it is converted into a set of vectors with the same length as the global encoding dimension.

[0009] Furthermore, the data is encrypted using a single instruction multiple data method, including: dividing each encoding vector and tag vector into multiple sub-blocks; each user encrypts each sub-block based on their own public key to obtain a feature ciphertext set and a tag ciphertext set.

[0010] Furthermore, for the partition scenario corresponding to the data type of the encrypted data, the encrypted data of each user is aggregated with ciphertext data, including: in the vertical partition scenario, firstly, dimension negotiation is carried out, and the global encoding length is determined by disclosing the maximum number of categories of features held by each user; then, the feature data of each user is extended with One-Hot encoding.

[0011] Furthermore, the security gain calculation includes: first, obtaining the encryption gradient vector and constructing left and right subset mask vectors; then, performing encrypted state filtering on the gradient based on the mask, and obtaining the aggregate statistics of the left and right subsets through homomorphic summation; calculating the gain of each candidate condition in the ciphertext domain based on the XGBoost gain formula; and finally, determining the combination of gain values ​​and outputting them.

[0012] Furthermore, the secure division calculation includes: first, setting an initial approximate ciphertext for the reciprocal of the divisor; then, performing Newton's iteration via a cloud server.

[0013] Furthermore, the secure tree construction calculation includes: first, performing algorithm initialization and parameter setting, and calculating encryption gain through node traversal and security gain algorithm call; then, judging whether the split stopping condition is met based on the leaf nodes, and calculating the encryption weight of the leaf nodes after determining the optimal split condition and security data, thus completing the construction of the entire tree and updating the global prediction value.

[0014] A second aspect of the present invention provides an XGBoost training system based on multi-party privacy protection.

[0015] An XGBoost training system based on multi-party privacy protection includes: The one-hot coding module is configured to use one-hot coding as a unified representation of feature ciphertext, and to perform unique coding for each user. The encryption module is configured to encrypt data using a single instruction multiple data method based on each user's individual key pair, and then upload the data to the cloud. The cloud server is configured to: aggregate encrypted data from each user's encrypted data for partition scenarios corresponding to the data type of the encrypted data, in order to construct an encrypted dataset; and, based on the encrypted training set, generate a secure XGBoost model for encrypted prediction services using a secure training algorithm; wherein the secure training algorithm includes secure gain calculation, secure partitioning calculation, secure division calculation, secure sigmoid calculation, and secure tree construction calculation.

[0016] A third aspect of the present invention provides a computer-readable storage medium having a program stored thereon, which, when executed by a processor, implements the steps of an XGBoost training method based on multi-party privacy protection as described in the first aspect of the present invention.

[0017] A fourth aspect of the present invention provides an electronic device including a memory, a processor, and a program stored in the memory and executable on the processor, wherein the processor executes the program to implement the steps in an XGBoost training method based on multi-party privacy protection as described in the first aspect of the present invention.

[0018] The above one or more technical solutions have the following beneficial effects: This invention introduces multi-key homomorphic encryption (MK) The CKKS scheme allows each participant to upload ciphertext to the cloud server by using their own public key to perform one-hot encoding and SIMD block encryption on their local data. The cloud server, without ever accessing the plaintext data or relying on any single high-privilege participant, utilizes a complete set of operations—including secure gain calculation, secure partitioning, secure division, secure sigmoid, and secure tree construction—to fully train the XGBoost model. Throughout the process, the original data remains in ciphertext. Even if the cloud server or an external attacker obtains the transmitted and stored ciphertext, they cannot decrypt it to obtain any useful information. This fundamentally solves the privacy protection problem of multi-party data that "cannot be shared but needs to be jointly modeled," enabling secure cross-institutional collaboration in various scenarios under legal and compliant conditions.

[0019] This invention implements a non-interactive encrypted training process. Each participant can go offline after a one-time encrypted data upload. Subsequent model building and computation are independently completed by the cloud server within the encrypted domain, without any online participation or secondary communication from any participant. Furthermore, this invention does not establish a centralized trusted third party or a high-authority controlling party; all intermediate statistics are calculated and transmitted in encrypted form. Even if intercepted during training, no plaintext information regarding data distribution or model structure will be leaked. Compared to existing solutions, this invention significantly reduces the communication overhead and deployment complexity of multi-party collaboration, eliminates the security risks of online dependence on specific participants and centralized permissions, and balances practicality, security, and deployment flexibility.

[0020] Advantages of additional aspects of the invention will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of the invention. Attached Figure Description

[0021] The accompanying drawings, which form part of this invention, are used to provide a further understanding of the invention. The illustrative embodiments of the invention and their descriptions are used to explain the invention and do not constitute an improper limitation of the invention.

[0022] Figure 1 This is a flowchart of an XGBoost training method based on multi-party privacy protection according to Embodiment 1 of the present invention. Detailed Implementation

[0023] It should be noted that the following detailed descriptions are exemplary and intended to provide further illustration of the invention. Unless otherwise specified, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains.

[0024] It should be noted that the terminology used herein is for the purpose of describing particular implementations only and is not intended to limit the exemplary implementations of the present invention.

[0025] Where there is no conflict, the embodiments and features in the embodiments of the present invention can be combined with each other.

[0026] The overall approach of this invention is as follows: This invention provides an XGBoost training method based on multi-party privacy protection. In this method, each participant only needs to encrypt their data before uploading it to a cloud server (CS). The cloud server utilizes its computing power to complete model training without the data content being visible. Each party can go offline after uploading, without needing continuous participation or multiple rounds of interaction during the training process. Furthermore, this invention can adapt to situations where multiple parties have inconsistent data distributions (e.g., each party holds different samples or different features), and it does not rely on trusted third parties or centralized high-privilege participants, thus improving the usability and security of the training process while ensuring privacy. To facilitate understanding of the technical solution of this invention, the following preliminary common knowledge is explained: 1) MK-CKKS multi-key homomorphic encryption scheme.

[0027] MK-CKKS is an MKHE scheme that supports homomorphic operations on both real and floating-point numbers, including addition, multiplication, and rotation. MK-CKKS also supports batch processing, allowing multiple encrypted data sets to be processed simultaneously. The MK-CKKS algorithm achieves high computational and communication efficiency while performing encrypted computations. It utilizes polynomial rings and specific operational techniques to maintain high efficiency in both computation and communication. Most importantly, MK-CKKS supports homomorphic operations on both real and floating-point numbers, making it suitable for machine learning applications that do not require precise results.

[0028] The following is a brief introduction to the MK-CKKS encryption scheme (based on RLWE): Enter a security parameter Run Setup( It also returns the parameters related to the generated key. Specifically, this includes the RLWE dimension n and the ciphertext model. Key distribution Error distribution on R Random vectors can be generated based on this. .in, This indicates the initialization of a multi-key homomorphic encryption scheme. Represents a polynomial ring It is uniformly distributed in the d-dimensional direction.

[0029] Each participant passes through Generate your own private and public keys. (From the key distribution...) Mid-sampling An error vector This allows us to obtain the public key of each participant. .

[0030] pass Generate their respective computation keys. Sampling. , , , ,make , Obtain the computation key. .in, This indicates the evaluation of the key generation algorithm. This represents a small polynomial sampled from the key distribution; Indicates from the polynomial ring Random vectors sampled from a uniform distribution. The first column represents the evaluation key; , This represents the error vector sampled from the error distribution.

[0031] Encryption process. Each participant uses their own public key to encrypt information. Input plaintext. , such that a = a[0] and Input plaintext Take the first component of the global common reference string. and the The first component of the public key of each participant, pk .sampling , . , Received the ciphertext = .

[0032] Decryption process. Let... for The encrypted text and the private keys of U participants (1, Related to, return .

[0033] : Ciphertext rescaling process. Given ciphertext ,when ,calculate Return ciphertext .

[0034] : Ciphertext rotation process. Assume The vector was encoded and encrypted. ,when At that time, a ciphertext is returned. Its encoded and encrypted vector is When k is negative, it means the vector has been rotated to the right, which will not be described further here.

[0035] This invention explains the homomorphic addition and homomorphic multiplication of MKHE-CKKS as follows. Assume there exist two... ciphertext and Their corresponding plaintexts are respectively and The corresponding private keys are respectively and To enable homomorphic operations on two ciphertexts under different keys, they are transformed into extended ciphertexts. and , They share the same private key. , .

[0036] Homomorphic addition Enter two ciphertexts and calculate Output .in, express and The encryption result of the corresponding plaintext value.

[0037] Homomorphic multiplication , Enter two ciphertexts and calculate Return ciphertext After homomorphic multiplication, the ciphertext dimension expands, requiring rescaling to reduce the ciphertext's modulus to ensure successful decryption later. .

[0038] 2) Approxsign(x) is a polynomial-based approximation comparison function.

[0039] The `ApproxSign(x)` function approximates the sign of each numerical component in ciphertext without decryption. It takes an encrypted ciphertext `x` containing M slot values ​​`x[0], x[1], ..., x[M-1]` and approximates the behavior of the standard sign function `sign(·)` by applying a predefined polynomial function `Poly(·)` to the ciphertext. After execution, it returns a new ciphertext where each slot value is close to 1 or -1, corresponding to whether the original input slot value was positive or negative: if the original value was greater than 0, the output is close to 1; if it was less than 0, the output is close to -1. The entire process is completed within the ciphertext space, without any plaintext interaction or decryption, achieving a secure approximate determination of the sign of encrypted numerical values.

[0040] 3) The ciphertext-based splitting condition selection algorithm FindSplitCond.

[0041] FindSplitCond determines the "position" of the optimal splitting condition from the vectors corresponding to all candidate splitting conditions of the current node without decryption by CS, and outputs this position as the encrypted splitting feature identifier and encryption threshold of the node.

[0042] Specifically, CS first encapsulates the values ​​of all candidate conditions in different slots of the same ciphertext (or several batches of ciphertext) in a pre-agreed order, making each slot uniquely correspond to a candidate condition. Then, it calls FindMaxPos to perform a search for the "maximum value and its position" within the ciphertext domain. This search process aligns the slots to be compared by rotating the ciphertext, and then calls ApproxSign() based on polynomial approximation on the difference between the two slots to obtain an approximate sign result, thereby constructing a closed-state selection factor (equivalent to implementing "if..." in the ciphertext domain). Then select Otherwise choose The system uses a selection logic to write the larger selected value back to the specified slot. Through multiple rounds of iteration (pairwise comparisons, group reduction), the global maximum gain value is finally obtained in the ciphertext, along with a "position indicator ciphertext" (which can be represented as a one-hot encoding or an equivalent slot marker) indicating the slot index where the maximum value is located. Finally, the server uses the pre-agreed slot index to... The mapping relationship is used to determine the "optimal slot" as the optimal splitting feature of the current node. With split threshold (All data are kept in ciphertext) for subsequent secure partitioning algorithms to construct masks and partition the ciphertext training dataset based on the splitting conditions.

[0043] Example 1 This embodiment discloses an XGBoost training method based on multi-party privacy protection.

[0044] like Figure 1 As shown, an XGBoost training method based on multi-party privacy protection includes: Step S1: Use one-hot coding as a unified representation of the feature ciphertext to uniquely encode each user; Step S2: Based on each user's individual key pair, encrypt the data using the single instruction multiple data method and upload it to the cloud; Step S3: For the partition scenario corresponding to the data type of the encrypted data, aggregate the encrypted data of each user to construct an encrypted dataset; Step S4: Based on the encrypted training set, a secure training algorithm is used to generate a secure XGBoost model for encrypted prediction services; wherein, the secure training algorithm includes secure gain calculation, secure partitioning calculation, secure division calculation, secure sigmoid calculation, and secure tree construction calculation.

[0045] Based on the above process, this invention can encrypt the private data of multiple participants individually, avoiding privacy leaks caused by directly using plaintext data. Furthermore, the encrypted data can be uploaded and used offline for training, balancing practicality and deployment flexibility. To facilitate understanding of the technical solution of this invention, the specific implementation methods of this invention will be further explained and described below.

[0046] In step S1, one-hot encoding is used as a unified representation of the feature ciphertext to uniquely encode each user.

[0047] Existing XGBoost exact greedy algorithms require sorting eigenvalues ​​to calculate the optimal split gain, which introduces a large number of interactions in homomorphic encryption environments. To avoid this problem, this invention employs one-hot encoding as a unified representation of the eigenvalue ciphertext. The specific implementation of eigenvalue encoding is as follows: Assuming there is a total Features Each feature For an ordered categorical variable, the number of categories is: Based on this, the system determines a global encoding dimension. .

[0048] For each feature Convert it into a set of lengths vector .in, It is a length equal to the total number of samples A column vector whose elements represent the corresponding sample in the feature The Is the category 1 (yes) or 0 (no)? If the actual number of categories for a feature is less than... Then fill the length with a zero vector. .

[0049] Original data rows for each sample Converted into a having A vector of n elements, where the first n elements are... Each element represents the encoded feature, and the last element is the label. Label vector. Leave it unchanged; do not encode it.

[0050] In step S2, data is encrypted using a single instruction multiple data method based on each user's individual key pair and uploaded to the cloud.

[0051] To improve the efficiency of ciphertext computation, Single Instruction Multiple Data (SIMD) technology is adopted, which divides each encoded vector... and the corresponding label vector Divided into Sub-blocks ( (Number of slots for encrypted text). Each user Use your own public key Each sub-block is encrypted to obtain a set of characteristic ciphertexts. and Tag Encryption Collection .

[0052] In step S3, for the partition scenario corresponding to the data type of the encrypted data, the encrypted data of each user is aggregated to construct an encrypted dataset.

[0053] To address the different data partitioning types of datasets, this invention proposes various encrypted data aggregation schemes, which can be implemented through the following methods: 1) Vertical partitioning scenario.

[0054] In a vertical partitioning scenario, each user holds different features for the same samples. First, dimension negotiation occurs, with each user u publicly disclosing the maximum number of categories for the features they hold. Determine the global encoding length Subsequently, the user obtains their feature data using extended One-Hot encoding. The encrypted data is then uploaded. The cloud server aggregates all the encrypted data to form a global training set. .in, Indicates the first The user's holding of the first The number of categories for each feature Indicates the first The largest number of categories among all features held by a user; Indicates the first Among users, characteristics After One-Hot extended encoding, the... The column vector corresponding to the last (i.e., the last) coded bit; User characteristics become after One-Hot encoding OK, The space of real matrices in the column; Indicates the first The user's holding of the first The complete plaintext matrix after the features are fully extended One-Hot encoded.

[0055] 2) Horizontal partitioning scenario.

[0056] In a horizontal partitioning scenario, users holding the same feature set but different sample subsets first negotiate to determine the global encoding length L and the total sample size. Assume the user The set of sample indexes held is For features For the j-th encoded bit, construct an encoding vector with precise mapping. Each element is defined as follows: ; in, Indicates user In the global first At each sample location, the feature The The binary indicator value corresponding to each coded bit (takes a value of 0 or 1); This represents the sample index in the global training dataset. Indicates the first The set of indices of a user's local samples in the global sample space; Indicates the global number Each sample in features The original category value.

[0057] Each user encodes and encrypts their own data, resulting in a local ciphertext dataset with the same structure as the encrypted dataset held by user u. The server then aligns the positions of all user-uploaded ciphertexts and sums them element-wise to complete the aggregation. , The final training set is represented as: ; in, This indicates that the cloud server (CS) has completed the aggregation of encrypted data uploaded by all users into an encrypted training set, and the corresponding decryption private key set is the private key set of all users. This refers to all user-uploaded content related to features. The The coded bit, the ... The aggregated result of summing the element-by-element features of each data block's ciphertext; This represents the encrypted tag information value after aggregation. Indicates user The uploaded ciphertext tags correspond only to user u's decryption private key set.

[0058] In step S4, based on the encrypted training set, a secure training algorithm is used to generate a secure XGBoost model for encrypted prediction services; wherein, the secure training algorithm includes secure gain calculation, secure partitioning calculation, secure division calculation, secure sigmoid calculation, and secure tree construction calculation.

[0059] 1) Safety gain calculation.

[0060] 1.1) Functional description of the algorithm.

[0061] In each round of XGBoost training, the first and second gradients of each sample need to be obtained based on the current model output. These gradients are used for subsequent gain calculations for node partitioning and leaf node weight updates. This algorithm's cloud server, without decryption, first obtains the encrypted gradient / Hessian value (dense gradient vector), and then uses a mask vector to perform "dense filtering and aggregation" on the left and right subsets corresponding to different candidate partitioning conditions. Finally, it obtains the encrypted gain value for each candidate condition and encapsulates and outputs it.

[0062] Input: CS retrieves the encrypted input for this training round at the current node, and the aggregated and encrypted label information value uploaded by the user. Initialize the set prediction probability value These encrypted messages are divided into m data blocks according to the SIMD slot capacity.

[0063] Output: Gain values ​​under different partitioning conditions in encrypted state. .

[0064] 1.2) Algorithm execution.

[0065] 1.2.1) Obtaining the Encrypted Gradient Vector: Based on the gradient definition of the "binary classification task," the cloud server performs homomorphic operations on each data block within the ciphertext domain according to the gradient calculation formula to obtain the first-order gradient ciphertext. ; Subtract the probability prediction value from the vector where each slot is 1. .

[0066] 1.2.2) Construction of left and right subset mask vectors: For each candidate partitioning condition at the current node To determine whether a feature meets a threshold, CS constructs two vectors sequentially using homomorphic addition, and then homomorphically adds the left subset mask vector. and right subset mask vector The mask vector means that the sample rows that meet the splitting condition are marked as 1, and the rest are marked as 0.

[0067] 1.2.3) Perform "dense-state filtering" (pointwise multiplication) on the gradient using a mask, and obtain the aggregation statistics of the left and right subsets by homomorphic summation: On the cloud server, for each candidate split condition, perform pointwise multiplication of the first-order gradient ciphertext vector with the left mask vector to obtain the left gradient ciphertext that "only retains the positions of the left subset samples and sets the other positions to zero". Then, perform homomorphic summation on the cloud server (slot summation can be achieved through rotation and accumulation) to obtain the aggregation value of the first-order gradient of the left subset under the candidate split condition. The same method was used to obtain the Hessian ciphertext on the left. Similarly, this process can be performed once on the right mask to obtain the corresponding gradient / Hessen ciphertext on the right side. and .

[0068] 1.2.4) Calculate the gain for each candidate condition in the ciphertext domain according to the XGBoost gain formula: Perform the following homomorphic operation based on the gain formula: First, calculate the squared term: using homomorphic multiplication and addition. , , ;by The corresponding plaintext value represents If the ciphertext is given, then all subsequent values ​​will be ciphertext values ​​used when calculating the gain; and These are the ciphertexts of the first-order gradient sums of the left and right subsets, respectively. The space in which they belong is a ciphertext space, packed using SIMD, with each slot corresponding to a candidate splitting condition.

[0069] Then calculate the denominator: using homomorphic multiplication and addition. , , .

[0070] Finally, Newton's iteration method is used to approximate the calculation of each fraction: , , .

[0071] 1.2.5) Gain Value Combination and Output: The encryption gain value is obtained by combining the components using homomorphic addition and subtraction. To improve efficiency, the cloud server organizes the "aggregated gradients of the left and right subsets of all candidate splitting conditions" in a SIMD parallel manner, constructing a set of ciphertext vectors, so that different slots of each ciphertext vector correspond to the aggregation results of different candidate splitting conditions; the gain calculation of multiple candidate conditions is performed in parallel within the ciphertext; finally, all gain values ​​are uniformly encapsulated into a single gain ciphertext vector. The ciphertext contains plaintext information about the gain value under each different partitioning condition. Indicates the first Features In the The gain value of the candidate splitting condition at each coding bit (i.e., the threshold position).

[0072] 2) Safety partitioning calculation.

[0073] 2.1) Functional description of the algorithm.

[0074] In the encrypted state, the algorithm first calculates the encryption gain value obtained in the previous round. The optimal splitting condition for the current node is determined, and the splitting feature and threshold corresponding to the maximum gain are extracted. Then, by constructing a mask vector and performing ciphertext rotation operations, the encrypted training data of the current node is divided into two subsets, left and right, without revealing any splitting condition information.

[0075] Input: Encrypted training data for the current node Encryption gain vector .

[0076] Output: Encrypted training data for the left child node Encrypted training data for right child nodes .

[0077] 2.2) Algorithm execution.

[0078] 2.2.1) Split Information Extraction: The cloud server calls the FindSplitCond() function from the node structure to extract information from the encryption gain vector. Extract the encryption split feature identifier corresponding to the maximum gain value. (feature (index) and encrypted split value (feature The corresponding threshold j).

[0079] 2.2.2) Feature Selection Mask Generation: CS generates a mask for each candidate ciphertext feature. Rotate to the left in sequence Calculation of positions: ; in, This indicates that only the first slot is 1. This indicates feature selection mask ciphertext. Indicates the current node The optimal split feature identifier ciphertext, Indicates the number of rotation steps in the cycle. Only when If the result is true, the first slot of the result is 1; otherwise, it is 0. Then, a slot copy operation is performed, copying the value of the first slot of the ciphertext to all slots of the ciphertext to form a complete feature selection mask. .

[0080] 2.2.3) Feature Data Extraction: Using the above mask, extract and split features. All related encrypted data. Specifically, feature selection masks. Ciphertext with all features of the current node (r,p,q) By performing homomorphic multiplication digit by digit and then homomorphically summing the products across all feature dimensions, a new set of ciphertext is obtained. Semantically, these are equivalent to retaining only the one-hot column of the i-th feature in the training set, while clearing the remaining feature columns to zero, in order to extract the split features. Related encrypted data.

[0081] 2.2.4) Lock threshold j in the ciphertext domain and generate... Required mask: cloud server for each threshold location Construct "whether it belongs to the previous" The threshold selection factor for each category will be used to perform a homomorphic rotation operation. Rotate to the left At each position, the 0 / 1 of the first slot is copied to all slots to construct a threshold mask. ; ; in, This represents the threshold mask ciphertext (representing...) (indicator vector), corresponding to the ciphertext space. Each slot is either 0 or 1, used to filter samples whose feature value is less than the threshold j; This represents the unit mask ciphertext with the first slot set to 1, i.e., the plaintext vector. The encryption result is used to extract the first slot value of the rotated ciphertext.

[0082] The cloud server extracts the data from step 2.2.3). Multiply each threshold mask obtained in step 2.2.4) element-wise, and then... arrive The parts are homomorphically summed to obtain a ciphertext. This is also the complete left subset mask. Its semantics are equivalent to the plaintext. That is, the sample satisfies The value is 1 if it is in the slot, otherwise it is 0.

[0083] 2.2.5) Generate the left subset data corresponding to the left child node: Cloud server for Perform the same type of operation on all feature column ciphertexts and all label column ciphertexts, and then ciphertext each feature column ciphertext... With left mask Perform element-wise multiplication to clear the positions of samples not belonging to the left subset to zero; similarly, ciphertext each label. With left mask Perform element-wise multiplication to preserve the labels corresponding to the left subset and clear the rest of the labels to zero, thus completing the encrypted training data for the left child nodes. .

[0084] 2.2.6) Right Subset Data Generation: The cloud server constructs the right subset data in a complementary manner, obtained by subtracting the left subset data from the original data. This is achieved through homomorphic subtraction, subtracting the left subset data from the original training data to obtain the encrypted training data for the right child nodes. .

[0085] 2.2.7) Data Output: Output the data of the left child node. and right child node data For use in subsequent recursive training.

[0086] 3) Calculation of safe division.

[0087] 3.1) Functional description of the algorithm.

[0088] In multi-key fully homomorphic encryption, the encryption circuit does not support division operations. Therefore, this paper uses Newton's iteration method to approximate the division process. The core idea is to transform division into an operation that relies only on addition and multiplication, thus adapting it to the characteristics of homomorphic encryption operations.

[0089] Input: CS is the ciphertext of the dividend. The ciphertext of the divisor The number of iterations is K.

[0090] Output: Encrypted division result .

[0091] 3.2) Algorithm execution.

[0092] 3.2.1) Initialization: Set the initial approximate ciphertext of the reciprocal of the divisor. (Right now The initial estimate is encoded into the ciphertext slot using either a constant or prior estimate. 3.2.2) Newton iteration in CS: For arrive implement: First, calculate the "product error" of the current approximation. ; Reconstruct the correction term for Newton's iteration and calculate CS. ; Finally, update the reciprocal approximation to make As the encrypted domain gradually approaches , .

[0093] 3.2.3) Calculate the quotient: .

[0094] 3.2.4) Batch Optimization (Optional): If multiple divisions need to be calculated simultaneously in the same round, CS can perform multiple batch optimizations. The above steps are iterated in parallel for different slots encoded into the ciphertext, thereby reducing the number of homomorphic operations and interaction overhead.

[0095] 4) Safe sigmoid calculation.

[0096] 4.1) Functional description of the algorithm.

[0097] In the XGBoost model, the Sigmoid function is typically used to activate the model's predictions, which is defined as follows: ; in, This represents the predicted value of sample i in the T-th tree. Since the MKHE-CKKS homomorphic encryption scheme cannot directly perform exponentiation, this invention uses the Taylor expansion formula to expand the exponent part of the sigmoid function. The Taylor expansion formula is as follows: ; Input: CS input the encrypted prediction value of the t-th tree The maximum order K of the Taylor expansion; Output: Probability prediction of encryption .

[0098] 4.2) Algorithm execution.

[0099] 4.2.1) Initialize the Taylor accumulation variable: initialize the Taylor expansion result. Current item initial factorial value All are set to ciphertext vectors with each slot containing 1. .

[0100] 4.2.2) Successive generation of exponential approximation terms (Taylor expansion): For k=1 to K, CS is executed sequentially. First update the factorial reciprocal: change the constant. Encoded into ciphertext and then... Homomorphic multiplication, to gradually obtain Approximate cumulative value .

[0101] Re-accumulated exponentiation: Update the current term .

[0102] Based on this, the k-th term is formed: This yields an approximation. , .

[0103] Finally, sum them up in the result: .

[0104] 4.3) Calculate the Sigmoid value.

[0105] 4.3.1) Constructing the ciphertext of the sigmoid: cloud server computing Obtain the dense state denominator .

[0106] 4.3.2) Calling safe division to obtain the Sigmoid output: Cloud server with For the dividend, To use the divisor, call SecureDivision: .

[0107] 5) Calculation of secure tree construction.

[0108] 5.1) Functional description of the algorithm.

[0109] The secure tree-building algorithm is used in a multi-key homomorphic encryption environment (MK-CKKS) to recursively construct the first tree by a cloud server without decrypting the training data or revealing the splitting conditions and intermediate statistics. A cryptographic decision tree.

[0110] Input: Encrypted training dataset Total number of nodes K, and the encrypted prediction value of the previous tree. Cryptographic probability prediction of the previous tree The aggregated ciphertext of tags .

[0111] Output: Tree model to be added to the ensemble model Encrypted predictions of all samples required for the next round of training. .

[0112] 5.2) Algorithm execution.

[0113] 5.2.1) Algorithm Initialization and Parameter Setting: Initialize the cloud server with an empty tree structure. Set the root node Corresponding to the complete encrypted training set Configure training hyperparameters: maximum depth Regularization parameters Leaf Punishment Item Learning rate At the same time, the queue of nodes to be processed is initialized to contain only the root node.

[0114] 5.2.2) Node Traversal and Calculation of Encryption Gain Using the Security Gain Algorithm: The cloud server retrieves a node from the current queue of nodes to be processed. Read the encrypted training data corresponding to this node. The security gain algorithm in section 2.3 is called to first calculate the gradient value, and then the encryption gain vector of the node is calculated, encapsulated, and output in the ciphertext domain. .

[0115] 5.2.3) Determine if the splitting stop condition is met (leaf node determination): The cloud server determines whether a node has reached its maximum depth according to a preset stopping strategy. Should the splitting continue? If it is determined to stop, proceed to the leaf weight calculation step; if the splitting continues, proceed to step [step number missing].

[0116] 5.2.4) Optimal Splitting Condition Selection and Secure Data Partitioning: When a node needs to continue splitting, the cloud server executes the complete process of the secure partitioning algorithm. First, based on the encryption gain vector... The FindSplitCond algorithm is used to extract the split feature identifier corresponding to the maximum gain. and threshold Then, perform a data partitioning operation to obtain the data of the left child node. The right child node data is obtained through complementary subtraction. After the partitioning is completed, the cloud server records the nodes. The splitting information is obtained, and the child node data pairs are added to the pending queue. Then, return to step ② to continue processing.

[0117] 5.2.5) Leaf node encryption weight calculation: When node When a node is identified as a leaf node, the cloud server calculates the node's output weight in encrypted form. This can be achieved through the following method: Leaf node mask construction: For leaf nodes Construct a mask that covers all samples of this node: ,in For nodes Chinese characteristics No. Each category has a cryptographic indicator vector.

[0118] Gradient aggregation: Calculates the sum of gradients for samples within a node using homomorphic dot product. , .

[0119] Weight Calculation: Based on the XGBoost leaf node weight formula, the 2.5 safe division algorithm is used to calculate the weights. Calculated The encrypted model parameters of this leaf node are stored in the tree structure.

[0120] 5.2.6) Complete the entire tree construction and global prediction update: When the queue of nodes to be processed is empty (all nodes have been split or stopped as leaf nodes), the cloud server completes the first step. The tree is constructed, and then the global predictions are updated. The cloud server globally integrates the weights of all leaf nodes and updates the model's encrypted predictions. This is done by assigning the weight contribution of each leaf node to the learning rate. After weighting, the values ​​are added to the previous round of predictions. .

[0121] 5.2.7) Updating the encryption probability prediction in preparation for the next round of training: The cloud server calls the secure Sigmoid algorithm to perform an activation transformation on the updated encryption prediction value, and calculates the new encryption probability prediction value: .

[0122] 5.2.8) Output the construction result of a single tree: The cloud server outputs the first... Encrypted decision tree model This includes information related to the training process of the next tree.

[0123] Example 2 This embodiment discloses an XGBoost training system based on multi-party privacy protection.

[0124] An XGBoost training system based on multi-party privacy protection includes: The one-hot coding module is configured to use one-hot coding as a unified representation of feature ciphertext, and to perform unique coding for each user. The encryption module is configured to encrypt data using a single instruction multiple data method based on each user's individual key pair, and then upload the data to the cloud. The cloud server is configured to: aggregate encrypted data from each user's encrypted data for partition scenarios corresponding to the data type of the encrypted data, in order to construct an encrypted dataset; and, based on the encrypted training set, generate a secure XGBoost model for encrypted prediction services using a secure training algorithm; wherein the secure training algorithm includes secure gain calculation, secure partitioning calculation, secure division calculation, secure sigmoid calculation, and secure tree construction calculation.

[0125] Example 3 The purpose of this embodiment is to provide a computer-readable storage medium.

[0126] A computer-readable storage medium having a computer program stored thereon that, when executed by a processor, implements the steps of an XGBoost training method based on multi-party privacy protection as described in Embodiment 1 of this disclosure.

[0127] Example 4 The purpose of this embodiment is to provide an electronic device.

[0128] An electronic device includes a memory, a processor, and a program stored in the memory and executable on the processor, wherein the processor executes the program to implement the steps in an XGBoost training method based on multi-party privacy protection as described in Embodiment 1 of this disclosure.

[0129] The steps and methods involved in the apparatuses of Embodiments 2, 3, and 4 above correspond to those in Embodiment 1. For specific implementation details, please refer to the relevant description section of Embodiment 1. The term "computer-readable storage medium" should be understood as a single medium or multiple media including one or more instruction sets; it should also be understood as including any medium capable of storing, encoding, or carrying an instruction set for execution by a processor and enabling the processor to perform any of the methods in this invention.

[0130] Those skilled in the art will understand that the modules or steps of the present invention described above can be implemented using general-purpose computer devices. Optionally, they can be implemented using computer-executable program code, thereby allowing them to be stored in a storage device for execution by a computer device, or they can be fabricated as separate integrated circuit modules, or multiple modules or steps can be fabricated as a single integrated circuit module. The present invention is not limited to any particular combination of hardware and software.

[0131] While the specific embodiments of the present invention have been described above in conjunction with the accompanying drawings, this is not intended to limit the scope of protection of the present invention. Those skilled in the art should understand that various modifications or variations that can be made by those skilled in the art without creative effort based on the technical solutions of the present invention are still within the scope of protection of the present invention.

Claims

1. An XGBoost training method based on multi-party privacy protection, characterized in that, include: One-hot encoding is used as a unified representation of feature ciphertext, and each user is uniquely encoded. Specifically, by introducing multi-key homomorphic encryption, each participant uses only their own public key to perform one-hot encoding on local data. Data is encrypted using a single instruction multiple data method based on each user's individual key pair and then uploaded to the cloud. For partition scenarios corresponding to the data type of encrypted data, the encrypted data of each user is aggregated in ciphertext to construct an encrypted dataset; Based on the encrypted training set, a secure training algorithm is used to generate a secure XGBoost model for encrypted prediction services; wherein, the secure training algorithm includes secure gain calculation, secure partitioning calculation, secure division calculation, secure sigmoid calculation, and secure tree construction calculation. The security gain calculation includes: first, obtaining the encryption gradient vector and constructing left and right subset mask vectors; then, performing encrypted state filtering on the gradient based on the mask, and obtaining the aggregation statistics of the left and right subsets through homomorphic summation; calculating the gain of each candidate condition in the ciphertext domain based on the XGBoost gain formula; and finally, determining and outputting the combination of gain values. The secure division calculation includes: first, setting an initial approximate ciphertext for the reciprocal of the divisor; then, performing Newton's iteration via a cloud server; The secure tree construction calculation includes: first, performing algorithm initialization and parameter setting, and calculating encryption gain through node traversal and security gain algorithm call; then, judging whether the split stopping condition is met based on the leaf nodes, and calculating the encryption weight of the leaf nodes after determining the optimal split condition and security data, thus completing the construction of the entire tree and updating the global prediction value.

2. The XGBoost training method based on multi-party privacy protection as described in claim 1, characterized in that, During the encoding process, a global encoding dimension is determined based on the number of features and the number of categories for each feature; for each feature, it is converted into a set of vectors with the same length as the global encoding dimension.

3. The XGBoost training method based on multi-party privacy protection as described in claim 1, characterized in that, The data is encrypted using a single instruction multiple data method, which includes: dividing each encoding vector and tag vector into multiple sub-blocks; each user encrypts each sub-block based on their own public key to obtain a feature ciphertext set and a tag ciphertext set.

4. The XGBoost training method based on multi-party privacy protection as described in claim 1, characterized in that, For partitioning scenarios corresponding to the data type of encrypted data, encrypted data aggregation is performed on the encrypted data of each user. This includes: in vertical partitioning scenarios, firstly, dimension negotiation is carried out, and the global encoding length is determined by publicly disclosing the maximum number of categories of features held by each user; then, extended One-Hot encoding is performed on the feature data of each user.

5. An XGBoost training system based on multi-party privacy protection, employing the XGBoost training method as described in any one of claims 1-4, characterized in that, include: The one-hot coding module is configured to use one-hot coding as a unified representation of feature ciphertext, and to perform unique coding for each user. The encryption module is configured to encrypt data using a single instruction multiple data method based on each user's individual key pair, and then upload the data to the cloud. The cloud server is configured to: aggregate encrypted data from each user's encrypted data for partition scenarios corresponding to the data type of the encrypted data, in order to construct an encrypted dataset; and, based on the encrypted training set, generate a secure XGBoost model for encrypted prediction services using a secure training algorithm; wherein the secure training algorithm includes secure gain calculation, secure partitioning calculation, secure division calculation, secure sigmoid calculation, and secure tree construction calculation.

6. A computer-readable storage medium having a program stored thereon, characterized in that, When executed by the processor, the program implements the steps in the XGBoost training method based on multi-party privacy protection as described in any one of claims 1-4.

7. An electronic device, comprising a memory, a processor, and a program stored in the memory and executable on the processor, characterized in that, When the processor executes the program, it implements the steps in the XGBoost training method based on multi-party privacy protection as described in any one of claims 1-4.

Images