Longitudinal federated learning method based on fully homomorphic ScureBoost model

Abstract

The invention provides a longitudinal federated learning method based on a fully homomorphic ScureBoost model, which comprises the following steps that: an active node calculates a first derivative g and a second derivative h for each sample, encrypts and packs the first derivative g and the second derivative h by using a fully homomorphic encryption algorithm CKKS and sends the first derivative g and the second derivative h to a passive node; the passive node uses a security weighted quantile algorithm to find candidate segmentation points, and gradient aggregation is carried out on the encrypted derivative according to a result; the active node determines an optimal split point by utilizing a bucket subtraction method between tree nodes based on the aggregation gradient sent by the passive node and the own aggregation gradient; the passive node divides sample data according to the optimal splitting point; the active node updates the node information and the sample distribution condition of the split tree model; repeating the steps to the maximum depth of the tree to obtain a complete tree model; and after the model training is finished, carrying out PSI-based model federated reasoning. According to the method, the model training efficiency can be improved, the side channel attack resistance and security are improved, the communication traffic is reduced, and the practical application value of the fully homomorphic encryption algorithm is proved.

Description

Technical Field

[0001] The present invention relates to the technical field of privacy computing, and in particular to a vertical federated learning method based on a fully homomorphic SecureBoost model. Background Art

[0002] With the rapid development of Internet technology, human society has entered the big data era. As a new type of production factor, data faces risks such as leakage and tampering while creating huge value. Therefore, the research on privacy computing has attracted more and more attention. Privacy-preserving computing refers to a series of information technologies that perform analysis and calculation on data on the premise of ensuring that the data provider does not disclose the original data, including federated learning, secure multi-party computing, confidential computing, differential privacy, and homomorphic encryption. Among them, federated learning and homomorphic encryption are two key technologies of privacy-preserving computing.

[0003] Federated learning technology is a method for joint training of multi-party data. During the training process, federated learning transfers models and gradients instead of sample data to protect the privacy of sample data. Although all parties participating in the federated training can complete the collaborative training of the model without sharing data, a large number of training parameters need to be exchanged during the training process. Although federated learning has avoided the direct exposure of data, many methods for inferring users' sensitive data from the exchanged intermediate parameters have been proposed in recent years. The emergence of homomorphic encryption provides a solution to the above problems. It allows calculations to be performed on encrypted data, and the calculation results can be decrypted to be consistent with the calculation results in the plaintext state. Therefore, homomorphic encryption technology can effectively protect data privacy and model security, and the use of homomorphic encryption enables the secure training of federated learning models using encrypted data, thus effectively protecting privacy data.

[0004] In current mainstream federated learning platforms, for privacy protection solutions, mostly semi-homomorphic algorithms are used. However, semi-homomorphic algorithms have many problems and limitations in practical applications. For example, they cannot meet the calculation requirements of some complex functions (such as requiring both addition and multiplication operations at the same time, or a large number of homomorphic calculation executions). While fully homomorphic algorithms can achieve a more accurate approximation of the model, but the fully homomorphic algorithms have a large computational overhead and a much lower computational efficiency compared with semi-homomorphic algorithms. Therefore, it is of great significance to design a federated learning model training method based on fully homomorphic encryption and comprehensively improve the efficiency, security, and practicality. Summary of the Invention

[0005] The purpose of the present invention is to overcome the deficiencies of the prior art and provide a vertical federated learning method based on a fully homomorphic SecureBoost model to improve efficiency, security, and practicality.

[0006] To achieve the above object, the present invention adopts the following technical solutions.

[0007] In a first aspect, the present invention provides a vertical federated learning method based on a fully homomorphic SecureBoost model. The method is applied to a federated learning system, which includes an active node and a passive node. The active node has class labels. The method includes:

[0008] The active node calculates the first derivative g and the second derivative h for each sample, and encrypts and packages them using the fully homomorphic encryption algorithm CKKS and sends them to the passive node;

[0009] The passive node uses a secure weighted quantile algorithm to find candidate split points, aggregates the encrypted derivatives according to the results, and sends the aggregated gradients to the active node;

[0010] The active node determines the best split point using the bucket subtraction between tree nodes based on the aggregated gradients sent by the passive node and its own aggregated gradients, and sends the best split point to the corresponding passive node;

[0011] The passive node divides the sample data according to the best split point and sends the divided sample data to the active node;

[0012] The active node updates the node information and sample distribution of the tree model for this split according to the divided sample data;

[0013] Repeat the above steps until the maximum depth of the tree to obtain a complete tree model;

[0014] After the model training is completed, model federated inference based on PSI is performed.

[0015] Further, the active node calculates the first derivative g and the second derivative h for each sample, and encrypts and packages them using the fully homomorphic encryption algorithm CKKS and sends them to the passive node, including:

[0016] The active node uses the fully homomorphic encryption algorithm CKKS to package the first derivative g and the second derivative h of the sample into a ciphertext vector and sends it to the passive node.

[0017] Further, the candidate split point is obtained through the following secure weighted quantile algorithm:

[0018] S1, the active node calculates the total weight according to the aggregated gradients, and calculates the quantile threshold according to the total weight;

[0019] S2, the passive node sorts the feature values, divides the sorted feature values into multiple equal-width buckets, calculates the encrypted weights for the equal-width buckets and sends them to the active node;

[0020] S3. The active node decrypts the encrypted weights sum of the equal-width buckets and locates the candidate buckets from the equal-width buckets according to the quantile threshold;

[0021] S4. The active node repeats steps S2 - S3 to further bucket the candidate buckets and update the candidate buckets until the number of samples in the latest candidate bucket is less than or equal to the minimum number of samples or reaches the threshold, so as to obtain the candidate bucket closest to the threshold;

[0022] S5. The passive node calculates and encrypts the cumulative sum of the candidate bucket closest to the threshold, and sends the encrypted cumulative sum sequence to the active node;

[0023] S6. The active node decrypts the encrypted cumulative sum sequence and finds the index that meets the quantile threshold, and sends the index to the passive node;

[0024] S7. The passive node calculates and obtains the candidate points according to the index.

[0025] Further, based on the aggregated gradient sent by the passive node and its own aggregated gradient, the active node determines the best split point by using bucket subtraction between tree nodes, and sends the best split point to the corresponding passive node, including:

[0026] Using bucket subtraction between tree nodes, for any specific parent node of the tree, when the specific parent node is split, the samples related to the specific parent node are assigned to either the left node or the right node; based on the sum of the buckets of the left child node and the right child node must be equal to the bucket of the parent node after node splitting, the bucket of the parent node, the bucket of the left child node can be constructed, and the bucket of the right child node can be directly obtained by bucket subtraction.

[0027] Further, after the model training is completed, model federated inference based on PSI is performed, including:

[0028] The active node and the passive node number the tree nodes and form a node list;

[0029] The active node and the passive node sequentially remove the distinguishable nodes from the node list according to the features of their own predicted samples, and finally perform a private set intersection operation on the remaining nodes, and the active node obtains the private intersection result and then obtains the final sample division result.

[0030] Further, before the active node calculates the first derivative g and the second derivative h for each sample and encrypts and packs them using the fully homomorphic encryption algorithm CKKS and sends them to the passive node, the method further includes:

[0031] In the initialization phase, the active node and the passive node agree on security parameters, a hash function, and a random function.

[0032] In the pre-computation phase, the active node samples a random string The passive node initializes matrix D and samples a random PRF key Perform pre-computation;

[0033] In the oblivious transfer phase, the active node and the passive node perform w times of oblivious transfer in total.

[0034] In the private set intersection phase, the active node calculates the OPRF value for each sample and sends the resulting set Ψ to the passive node. The passive node also calculates its own OPRF value ψ. If and only if ψ ∈ Ψ, it is the intersection of the two data sets, and the result is stored in a file.

[0035] Second invention, the present invention also provides a vertical federated learning device based on a fully homomorphic SecureBoost model. The device is applied to a federated learning system, which includes an active node and a passive node. The active node has class labels. The device includes:

[0036] A derivative encryption module, which is used for the active node to calculate the first-order derivative g and the second-order derivative h for each sample, and encrypt and package them using the fully homomorphic encryption algorithm CKKS and send them to the passive node;

[0037] A gradient aggregation module, which is used for the passive node to use the secure weighted quantile algorithm to find candidate split points, aggregate the encrypted derivatives according to the results, and send the aggregated gradients to the active node;

[0038] An optimal split point determination module, which is used for the active node to determine the optimal split point based on the aggregated gradients sent by the passive node and its own aggregated gradients, and send the optimal split point to the corresponding passive node;

[0039] A sample partitioning module, which is used for the passive node to partition the sample data according to the optimal split point and send the partitioned sample data to the active node;

[0040] A model update module, which is used for the active node to update the node information and sample distribution of the tree model for this split according to the partitioned sample data;

[0041] A repetition module, which is used to repeat the above modules until the maximum depth of the tree to obtain a complete tree model;

[0042] An inference module, which is used for performing model federated inference based on PSI after the model training is completed.

[0043] In a third aspect, the present invention further provides an electronic device, including a memory, a processor, and a computer program stored on the memory and executable on the processor. When the processor executes the program, the above-mentioned method is implemented.

[0044] In a fourth aspect, the present invention further provides a computer-readable storage medium storing a computer program, and when the computer program is executed by a processor, the above-mentioned method is implemented.

[0045] In a fifth aspect, the present invention further provides a computer program product including a computer program, and when the computer program is executed by a processor, the above-mentioned method is implemented.

[0046] Advantages of the present invention: The vertical federated learning method based on the fully homomorphic SecureBoost model provided by the present invention combines the fully homomorphic encryption algorithm with federated learning. The fully homomorphic encryption algorithm CKKS is used to replace the semi-homomorphic encryption algorithm used in the SecureBoost model. At the same time, three improvements are proposed during the model training process, namely, the secure weighted quantile algorithm for finding candidate split points, ciphertext packing, and bucket subtraction between tree nodes to improve the model training efficiency. In addition, during the model inference process, the private set intersection technology is used to solve the problem of sample division path leakage, and at the same time, it has lower communication volume. The model training and inference methods proposed by the present invention have higher training efficiency and practicability compared with the traditional semi-homomorphic-based model training and inference methods. At the same time, it can better resist side-channel attacks, has higher security and lower communication volume. The present invention solves the limitations of the fully homomorphic encryption algorithm in terms of efficiency and proves the practical application value of the fully homomorphic encryption algorithm.

[0047] Additional aspects and advantages of the present invention will be given in part in the following description, which will become apparent from the following description, or can be understood through the practice of the present invention. Description of the Drawings

[0048] To more clearly illustrate the technical solutions of the embodiments of the present invention, the following will briefly introduce the drawings required for the description of the embodiments. Obviously, the following drawings are only some embodiments of the present invention. For those of ordinary skill in the art, without creative efforts, other drawings can be obtained based on these drawings.

[0049] Figure 1 It is a schematic flowchart of a vertical federated learning method based on a fully homomorphic SecureBoost model provided by an embodiment of the present invention;

[0050] Figure 2 It is a schematic diagram of finding candidate split points by the secure weighted quantile algorithm provided by an embodiment of the present invention;

[0051] Figure 3 Schematic diagram of bucket subtraction between tree nodes provided by an embodiment of the present invention;

[0052] Figure 4 Schematic diagram of the sample partitioning paths of three participating parties provided by an embodiment of the present invention;

[0053] Figure 5 Schematic diagram of federated inference based on private set intersection provided by an embodiment of the present invention;

[0054] Figure 6 Performance comparison chart of the SecureBoost model provided by an embodiment of the present invention. Detailed implementation manners

[0055] The following details the implementation manners of the present invention. The examples of the implementation manners are shown in the accompanying drawings, where the same or similar reference numerals represent the same or similar elements or elements with the same or similar functions from beginning to end. The implementation manners described below with reference to the accompanying drawings are exemplary and are only used to explain the present invention, and cannot be construed as a limitation to the present invention.

[0056] Those skilled in the art of the present technology can understand that unless specifically stated otherwise, the singular forms "a", "an" and "the" used herein may also include the plural forms. It should be further understood that the term "including" used in the specification of the present invention means the presence of features, integers, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components and / or their groups. It should be understood that when we say an element is "connected" or "coupled" to another element, it can be directly connected or coupled to other elements, or there may also be intermediate elements. In addition, the "connection" or "coupling" used herein may include wireless connection or coupling. The phrase "and / or" used herein includes any unit and all combinations of one or more of the associated listed items.

[0057] Those skilled in the art of the present technology can understand that unless otherwise defined, all terms (including technical terms and scientific terms) used herein have the same meaning as the general understanding of those of ordinary skill in the art to which the present invention belongs. It should also be understood that terms such as those defined in a general dictionary should be understood to have a meaning consistent with the meaning in the context of the prior art, and will not be interpreted with an idealized or overly formal meaning unless defined as here.

[0058] For ease of understanding of the embodiments of the present invention, the following will further explain with several specific embodiments as examples in conjunction with the accompanying drawings, and each embodiment does not constitute a limitation to the embodiments of the present invention.

[0059] Embodiment 1

[0060] See Figure 1 , a vertical federated learning method based on a fully homomorphic SecureBoost model, which is applied to a federated learning system. The federated learning system includes an active node P1 and a passive node P2. The active node has class labels, and the method includes the following steps:

[0061] S101, the active node calculates the first-order derivative g and the second-order derivative h for each sample, encrypts and packs them using the fully homomorphic encryption algorithm CKKS, and sends them to the passive node.

[0062] That is, in the model training process, the ciphertext packing technology is adopted to pack g i and h i into a ciphertext vector <g i ,h i > for homomorphic addition operation. This operation will reduce the costs of encryption, decryption, and homomorphic addition operations and greatly improve the model training efficiency.

[0063] S102, the passive node uses the secure weighted quantile algorithm to find candidate split points, aggregates the encrypted derivatives according to the results, and sends the aggregated gradients to the active node.

[0064] S103, the active node determines the best split point using the bucket subtraction between tree nodes based on the aggregated gradients sent by the passive node and its own aggregated gradients, and sends the best split point to the corresponding passive node.

[0065] S104, the passive node divides the sample data according to the best split point and sends the divided sample data to the active node.

[0066] S105, the active node updates the node information and sample distribution of the tree model for this split according to the divided sample data;

[0067] S106, repeat the above steps S101 - S105 until the maximum depth of the tree to obtain a complete tree model;

[0068] S107, after the model training is completed, perform model federated inference based on PSI.

[0069] Illustratively, the dataset used in this embodiment is the breast cancer dataset, which contains 569 samples and 30 features. The data samples come from the measurement of breast mass images and whether they are cancerous masses. The goal is to use these measurements to predict whether the mass is cancerous. In this dataset, the number of positive and negative samples is balanced, so the accuracy can be used to evaluate the model quality.

[0070] The vertical federated learning method based on the fully homomorphic SecureBoost model provided by the embodiments of the present invention can reduce the model training time, solve the problem of low efficiency of fully homomorphic encryption calculation, and improve the training efficiency of the SecureBoost model.

[0071] In some embodiments of the present invention, as Figure 2 shown, the candidate split points are obtained through the secure weighted quantile algorithm. Through hierarchical bucketing and interactive protocols, combined with the homomorphic properties of CKKS, quantile estimation in the encrypted state is realized, and the samples under the current node are divided into parts, so as to obtain candidate split points, where ∈ is the sampling rate. This method of finding candidate split points not only does not reduce the accuracy of the model, but also makes the model have better efficiency. The secure weighted quantile algorithm is as follows:

[0072] S1, the active node calculates the total weight according to the aggregated gradient The quantile threshold T = ε·S is calculated according to the total weight total .

[0073] S2, the passive node sorts the feature values and divides the sorted feature values into multiple equal-width buckets, calculates the encrypted weight sum for the equal-width buckets and sends it to the active node.

[0074] That is, the passive node divides the sorted feature values into K equal-width buckets {b1, b2,..., b K}, and for each bucket b j , calculates the encrypted weight sum Sends to the active node.

[0075] S3, the active node decrypts the encrypted weight sum of the equal-width buckets and locates the candidate buckets from the equal-width buckets according to the quantile threshold.

[0076] That is, the active node decrypts the bucket weight sum Calculates the prefix sum For the threshold T, find the smallest j * Satisfying Returns j * , so as to locate the candidate buckets.

[0077] S4, the active node repeats steps S2 - S3 to further bucket the candidate buckets and update the candidate buckets until the number of samples in the latest candidate bucket is less than or equal to the minimum number of samples or reaches the threshold, so as to obtain the candidate bucket closest to the threshold.

[0078] That is, the active node performs operations on the candidate bucket j *, recursively divide the sub-buckets, repeat steps S2 - S3, and stop when the number of samples in the bucket is less than or equal to the minimum number of samples or reaches d max and retain the first K - 1 candidate buckets b that are closest to the threshold candidate .

[0079] S5. The passive node calculates and encrypts the cumulative sum of the candidate buckets closest to the threshold, and sends the encrypted cumulative sum sequence to the active node.

[0080] That is, on the passive node, for the first K - 1 candidate buckets b candidate = [x r , x l , calculate and encrypt the cumulative sum for each, and generate an encrypted cumulative sum sequence The passive node sends {Enc(S i )} to the active node.

[0081] S6. The active node decrypts the encrypted cumulative sum sequence and finds the index that satisfies the quantile threshold, and sends the index to the passive node.

[0082] That is, after the active node decrypts, it finds the index {i1, i2,..., i i} that satisfies S K-1 ≥T.

[0083] S7. The passive node calculates the candidate points based on the index. That is, on the passive node, calculate the candidate points

[0084] Schematically, the passive node performs the gradient aggregation algorithm on the encrypted derivative as follows:

[0085]

[0086]

[0087] In addition, the active node decrypts the aggregated gradient G, H and merges it with its own aggregated gradient of features, and uses to calculate the best split point and send it to the passive node; the passive node records and divides the samples and sends the sample division result to the active node; the active node records and divides the samples, and thus completes a node split.

[0088] In some embodiments of the present invention, as Figure 3 shown, the active node determines the best split point using the bucket subtraction between tree nodes based on the aggregated gradient sent by the passive node and its own aggregated gradient, and sends the best split point to the corresponding passive node, including:

[0089] The bucket subtraction method is adopted between tree nodes. For any specific parent node of the tree, when the specific parent node is split, the samples associated with the specific parent node are assigned to either the left node or the right node; based on the fact that the sum of the buckets of the left child node and the right child node must be equal to the bucket of the parent node after node splitting, the bucket of the parent node, the bucket of the left child node, and the bucket of the right child node are constructed. The bucket of the right child node can be directly obtained by bucket subtraction, that is, the bucket of the right child node = the bucket of the parent node - the bucket of the left child node.

[0090] The specific node splitting algorithm is as follows:

[0091]

[0092]

[0093] In some embodiments of the present invention, after the model training is completed, model federated inference based on PSI is performed, including:

[0094] The active node and the passive node number the tree nodes and form a node list.

[0095] The active node and the passive node sequentially remove the distinguishable nodes from the node list according to the characteristics of their own predicted samples, and finally perform a private set intersection operation on the remaining nodes. The active node obtains the private intersection result and thus obtains the final sample partitioning result.

[0096] In this embodiment, since the partitioning paths from the root node to the leaf node are different for each sample, the number of nodes assigned to the passive node along this path is different. Therefore, in some cases, the passive node can use the number of interactions and the communication volume with the active node to determine the sample partitioning path and its associated leaf node. The leaf node to which the sample belongs is sensitive information, and security problems will occur if it is leaked. Considering Figure 4In a tree model involving three or more participating parties, a passive node can even distinguish the final result without directly participating in the partitioning process, just by eavesdropping on the communication between an active node and other parties. For example, if passive node 2 monitors the communication between an active node and passive node 1 without participating in the sample partitioning process, then if there is one communication between the active node and passive node 1, the partitioning path of the sample is: root node → node 1 → node 3; if there are two communications between the active node and passive node 1, the partitioning path of the sample is: root node → node 1 → node 4. That is, except for the active node, passive nodes have the ability to infer the final partitioning result by analyzing the partitioning path of the sample. To solve this problem, a method that can complete federated inference with only one communication between the active node and the passive node is needed, that is, using PSI in the federated inference process. This method ensures that except for the active node, passive nodes cannot speculate on the final partitioning path of the sample.

[0097] As Figure 5 shown in the tree model, for the sake of illustration, the tree nodes are numbered, and the thresholds of the active node and the passive node are directly shown (but the thresholds are only visible to themselves). Assume that the passive node has features f0, f1,..., f9, and the active node has features f 10 , f 11 ,..., f 19 . In the tree model, the selected features are f1, f 15 , f8, f 12 , f2, f5, f 13 . There are existing samples x p distributed on both sides. Passive node: f1 = 13, f2 = 10, f5 = 30, f8 = 10, active node: f 12 = 5, f 13 = 50, f 15 = 10.

[0098] The active node and the passive node sequentially remove the distinguishable nodes from the node list, including the following sub-steps:

[0099] S201. For the passive node, it initially has the node list [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14]. For sample x p , f1 = 13 is less than the threshold 15 of node 0, so the sample will be partitioned to node 1. Therefore, the passive node will remove 2, 5, 6, 11, 12, 13, and 14 from the node list; for sample x p , f2 = 10 is greater than the threshold 7 of node 4, so the passive node will remove 9 from the node list. Finally, the node list of the passive node is retained as [0, 1, 3, 4, 7, 8, 10].

[0100] S202. For the active node, there is also a node list [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14] at the beginning, and the sample x p 's f 15 = 10 is greater than the threshold 6 of node 1. Therefore, the active node removes 3, 7, and 8; the sample x p 's f 13 = 50 is greater than the threshold 40 of node 6. Therefore, the active node removes 13 from the node list. Finally, the node list of the active node remains as [0, 1, 2, 4, 5, 6, 9, 10, 11, 12, 14].

[0101] S203. The active node and the passive node perform PSI operations on the final node list. Finally, the active node obtains the PSI result [0, 1, 4, 10], and thus knows that the sample x p is finally partitioned into the leaf node w4.

[0102] The performance comparison between the fully homomorphic SecureBoost model and the traditional semi - homomorphic - encryption - based SecureBoost model during the model training stage is as Figure 6 shown. We conducted 20 experiments and obtained the model training time and the model accuracy respectively. By analyzing the experimental data, we found that the average training time of the fully homomorphic - encryption - based SecureBoost model is 75.04 seconds, and the average accuracy rate is 93.40%; while on the same data set, the average training time of the classical SecureBoost model is 150.07 seconds, and the average accuracy rate is 92.55%. Therefore, the training efficiency of our scheme is twice that of the classical SecureBoost model on this data set, and the accuracy rate is also slightly increased by 0.85%.

[0103] The comparison of the communication volume between the federated inference based on private set intersection and the traditional federated inference scheme during the model inference stage is shown in Table 1. We quantified the number of communication bytes exchanged by the participating parties during the federated inference process at different tree depths. Experimental analysis shows that the federated inference based on PSI significantly reduces the communication volume at different depths: at depth 1, compared with the classical federated inference, the communication volume is reduced by about 24% to 34%; at depth 2, the reduction range is between 55% and 58%; similarly, at depth 3, the reduction is about 47% to 51%; and at depth 4, the reduction range is about 27% to 30%.

[0104] Table 1: Comparison of communication overhead

[0105]

[0106]

[0107] The vertical federated learning method of the SecureBoost model based on fully homomorphic encryption provided by the embodiments of the present invention uses the results of private set intersection for federated inference. During the inference process, regardless of the feature values of the samples, a sample only needs to perform PSI once to complete the federated inference process, avoiding the problem of inferring sample partitioning based on the number of communications and having less communication volume, effectively resisting side-channel attacks, solving the problem that malicious participants can infer the partitioning path of samples by using the number of interactions and communication volume between parties, improving the security of the model, and reducing the communication volume during the federated inference process.

[0108] In some embodiments of the present invention, before the active node calculates the first-order derivative g and the second-order derivative h for each sample and encrypts and packs them using the fully homomorphic encryption algorithm CKKS and sends them to the passive node, the method further includes:

[0109] In the initialization stage, the active node and the passive node agree on security parameters, a hash function, and a pseudorandom function.

[0110] In the precomputation stage, the active node samples a random string The passive node initializes the matrix D and samples a random PRF key Perform precomputation.

[0111] In the oblivious transfer stage, the active node and the passive node perform w oblivious transfers in total.

[0112] In the private set intersection stage, the active node calculates the OPRF value for each sample and sends the resulting set Ψ to the passive node. The passive node also calculates its own OPRF value ψ. If and only if ψ ∈ Ψ, it is the intersection of the two data sets, and the result is stored in a file.

[0113] Specifically, in the initialization stage, the active node P1 and the passive node P2 agree on the security parameters λ, κ, and calculate the protocol parameters m, w, l1, l2 according to the security parameters and the sizes n1, n2 of the two-party data sets. In addition, two hash functions are agreed and Pseudorandom function

[0114] In the precomputation stage, P1 samples a random string P2 initializes a matrix D of m×w with all elements being 1. Use D1,...,D w Define its column vectors; sample a random PRF key For each y ∈ Y, calculate v = F k (H1(y)). Suppose for each i ∈ [w] there is D i[v i = 0。

[0115] In the oblivious transfer phase, P1 randomly selects an m×w binary matrix and calculates the matrix with inputs {A i , B i}. i∈[w] P2 inputs S[1], …, S[w]. Then, the two parties perform w oblivious transfers, and P2 obtains w m - byte - long strings as the column vectors of matrix C, where the dimension of matrix C is m×w.

[0116] In the PSI phase, P2 sends the PRF key k to P1. For each x ∈ X, P1 calculates v = F k (H1(x)) and its OPRF value ψ = H2(C1[v1]||…||C w [v w ), and sends ψ to P2. P2 uses Ψ to represent the set C n1 of the OPRF values received from P1. For each y ∈ Y, P2 calculates v = F k (H1(y)) and its OPRF value ψ = H2(A1[v1]||…||A w [v w ). If and only if ψ ∈ Ψ, y is output. All these output y values are the intersection X ∩ Y of the two data sets, which are stored in a file, and finally the obtained intersection is passed to P1.

[0117] The vertical federated learning method for the SecureBoost model based on fully homomorphic encryption provided by the embodiments of the present invention combines the fully homomorphic encryption algorithm with federated learning. It uses the fully homomorphic encryption algorithm CKKS to replace the semi - homomorphic encryption algorithm used in the SecureBoost model. At the same time, three improvements are proposed during the model training process, namely, the secure weighted quantile algorithm for finding candidate split points, ciphertext packing, and bucket subtraction between tree nodes to improve the model training efficiency. In addition, during the model inference process, the private set intersection technology is used to solve the problem of sample partitioning path leakage, and it also has lower communication volume. The model training and inference methods proposed by the present invention have higher training efficiency and practicality compared with the traditional semi - homomorphic - based model training and inference methods. At the same time, it can better resist side - channel attacks, has higher security and lower communication volume. The present invention solves the limitations of the fully homomorphic encryption algorithm in terms of efficiency and proves the practical application value of the fully homomorphic encryption algorithm.

[0118] Example 2

[0119] Based on Embodiment 1, Embodiment 2 provides a vertical federated learning device for a SecureBoost model based on fully homomorphic encryption. This vertical federated learning device for a SecureBoost model based on fully homomorphic encryption corresponds to the above-mentioned vertical federated learning method for a SecureBoost model based on fully homomorphic encryption, and specifically includes:

[0120] The device is applied to a federated learning system, which includes an active node and a passive node. The active node has class labels. The device includes:

[0121] A derivative encryption module, configured to calculate the first-order derivative g and the second-order derivative h for each sample by the active node, and encrypt and package them using the fully homomorphic encryption algorithm CKKS and send them to the passive node;

[0122] A gradient aggregation module, configured to use a secure weighted quantile algorithm by the passive node to find candidate split points, perform gradient aggregation on the encrypted derivatives according to the results, and send the aggregated gradients to the active node;

[0123] An optimal split point determination module, configured to determine the optimal split point based on the aggregated gradients sent by the passive node and its own aggregated gradients by the active node using the bucket subtraction method between tree nodes, and send the optimal split point to the corresponding passive node;

[0124] A sample partitioning module, configured to partition the sample data by the passive node according to the optimal split point, and send the partitioned sample data to the active node;

[0125] A model update module, configured to update the node information and sample distribution of the tree model for this split by the active node according to the partitioned sample data;

[0126] A repetition module, configured to repeat the above modules until the maximum depth of the tree to obtain a complete tree model;

[0127] An inference module, configured to perform model federated inference based on PSI after the model training is completed.

[0128] For specific details, refer to the description in the part of the vertical federated learning method for a SecureBoost model based on fully homomorphic encryption, which will not be elaborated here.

[0129] Embodiment 3

[0130] Embodiment 3 of the present invention provides an electronic device, including a memory and a processor, the processor and the memory communicate with each other, the memory stores program instructions executable by the processor, and the processor calls the program instructions to execute a vertical federated learning method based on a fully homomorphic SecureBoost model. The method is applied to a federated learning system, and the federated learning system includes an active node and a passive node. The active node has class labels. The method includes the following process steps:

[0131] The active node calculates the first-order derivative g and the second-order derivative h for each sample, and encrypts and packages them using the fully homomorphic encryption algorithm CKKS and sends them to the passive node;

[0132] The passive node uses the secure weighted quantile algorithm to find candidate split points, aggregates the encrypted derivatives according to the results, and sends the aggregated gradients to the active node;

[0133] The active node determines the best split point using the bucket subtraction between tree nodes based on the aggregated gradients sent by the passive node and its own aggregated gradients, and sends the best split point to the corresponding passive node;

[0134] The passive node divides the sample data according to the best split point and sends the divided sample data to the active node;

[0135] The active node updates the node information and sample distribution of the tree model for this split according to the divided sample data;

[0136] Repeat the above steps until the maximum depth of the tree to obtain a complete tree model;

[0137] After the model training is completed, perform model federated inference based on PSI.

[0138] Embodiment 4

[0139] Embodiment 4 of the present invention provides a computer-readable storage medium, which stores a computer program. When the computer program is executed by a processor, it implements a vertical federated learning method based on a fully homomorphic SecureBoost model. The method is applied to a federated learning system, and the federated learning system includes an active node and a passive node. The active node has class labels. The method includes the following process steps:

[0140] The active node calculates the first-order derivative g and the second-order derivative h for each sample, and encrypts and packages them using the fully homomorphic encryption algorithm CKKS and sends them to the passive node;

[0141] The passive node uses the secure weighted quantile algorithm to find candidate split points, aggregates the encrypted derivatives according to the results, and sends the aggregated gradients to the active node;

[0142] The active node determines the best splitting point using bucket subtraction between tree nodes based on the aggregated gradients sent by the passive node and its own aggregated gradients, and sends the best splitting point to the corresponding passive node;

[0143] The passive node divides the sample data according to the best splitting point and sends the divided sample data to the active node;

[0144] The active node updates the node information and sample distribution of the tree model for this split according to the divided sample data;

[0145] Repeat the above steps until the maximum depth of the tree to obtain a complete tree model;

[0146] After the model training is completed, perform model federated inference based on PSI.

[0147] Embodiment 5

[0148] Embodiment 5 of the present invention provides a computer program product, which includes a computer program. When the computer program is executed by a processor, it implements a vertical federated learning method for a SecureBoost model based on fully homomorphic encryption. The method is applied to a federated learning system, and the federated learning system includes an active node and a passive node. The active node has class labels. The method includes the following process steps:

[0149] The active node calculates the first-order derivative g and the second-order derivative h for each sample, and encrypts and packs them using the fully homomorphic encryption algorithm CKKS and sends them to the passive node;

[0150] The passive node uses a secure weighted quantile algorithm to find candidate splitting points, aggregates the encrypted derivatives according to the results, and sends the aggregated gradients to the active node;

[0151] The active node determines the best splitting point using bucket subtraction between tree nodes based on the aggregated gradients sent by the passive node and its own aggregated gradients, and sends the best splitting point to the corresponding passive node;

[0152] The passive node divides the sample data according to the best splitting point and sends the divided sample data to the active node;

[0153] The active node updates the node information and sample distribution of the tree model for this split according to the divided sample data;

[0154] Repeat the above steps until the maximum depth of the tree to obtain a complete tree model;

[0155] After the model training is completed, perform model federated inference based on PSI.

[0156] Those of ordinary skill in the art can understand that the accompanying drawings are only schematic diagrams of one embodiment, and the modules or processes in the drawings are not necessarily essential for implementing the present invention.

[0157] Each embodiment in this specification is described in a progressive manner. For the same or similar parts among the embodiments, reference can be made to each other. Each embodiment focuses on the differences from other embodiments. In particular, for method or system embodiments, since they are basically similar to method embodiments, they are described relatively simply, and the relevant parts can be referred to the descriptions of method embodiments. The method and system embodiments described above are only illustrative. The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed to multiple network units. Some or all of the modules can be selected according to actual needs to achieve the purpose of the solution of this embodiment. Those of ordinary skill in the art can understand and implement it without creative efforts.

[0158] The above is only a preferred specific implementation manner of the present invention, but the protection scope of the present invention is not limited thereto. Any changes or substitutions that can be easily thought of by those skilled in the art within the technical scope disclosed by the present invention should be covered by the protection scope of the present invention. Therefore, the protection scope of the present invention should be subject to the protection scope of the claims.

Claims

1. A vertical federated learning method based on a fully homomorphic SecureBoost model, characterized in that The method is applied to a federated learning system, which includes an active node and a passive node. The active node has class labels. The method includes: The active node calculates the first-order derivative g and the second-order derivative h for each sample, encrypts and packs them using the CKKS fully homomorphic encryption algorithm, and sends them to the passive node; The passive node uses the secure weighted quantile algorithm to find candidate split points, aggregates the encrypted derivatives according to the results, and sends the aggregated gradients to the active node; Based on the aggregated gradients sent by the passive node and its own aggregated gradients, the active node determines the best split point using the bucket subtraction between tree nodes, and sends the best split point to the corresponding passive node; The passive node divides the sample data according to the best split point and sends the divided sample data to the active node; The active node updates the node information and sample distribution of the tree model for this split according to the divided sample data; Repeat the above steps until the maximum depth of the tree to obtain a complete tree model; After the model training is completed, perform model federated inference based on PSI.

2. The method according to claim 1, wherein The candidate split point is obtained through the following secure weighted quantile algorithm: S1. The active node calculates the total weight according to the aggregated gradients and calculates the quantile threshold according to the total weight; S2. The passive node sorts the feature values, divides the sorted feature values into multiple equal-width buckets, calculates the encrypted weight sum for the equal-width buckets, and sends it to the active node; S3. The active node decrypts the encrypted weight sum of the equal-width buckets and locates the candidate buckets from the equal-width buckets according to the quantile threshold; S4. The active node repeats steps S2 - S3 to further bucket the candidate buckets and update the candidate buckets until the number of samples in the latest candidate bucket is less than or equal to the minimum number of samples or reaches the threshold, so as to obtain the candidate bucket closest to the threshold; S5. The passive node calculates and encrypts the cumulative sum of the candidate bucket closest to the threshold, and sends the encrypted cumulative sum sequence to the active node; S6. The active node decrypts the encrypted cumulative sum sequence and finds the index that satisfies the quantile threshold, and sends the index to the passive node; S7. The passive node calculates the candidate point according to the index.

3. The method according to claim 1, wherein Based on the aggregated gradients sent by the passive node and its own aggregated gradients, the active node determines the best split point using the bucket subtraction between tree nodes, and sends the best split point to the corresponding passive node, including: Using the bucket subtraction between tree nodes, for any specific parent node of the tree, when the specific parent node is split, the samples related to the specific parent node are assigned to either the left node or the right node; based on the sum of the buckets of the left child node and the right child node must be equal to the bucket of the parent node after node splitting, the bucket of the parent node and the bucket of the left child node are constructed, and the bucket of the right child node can be directly obtained by bucket subtraction.

4. The method according to claim 1, characterized in that, The active node calculates the first-order derivative g and the second-order derivative h for each sample, encrypts and packs them using the CKKS fully homomorphic encryption algorithm, and sends them to the passive node, including: The active node uses the fully homomorphic encryption algorithm CKKS to pack the first-order derivative g and the second-order derivative h of the sample into a ciphertext vector and send it to the passive node.

5. The method according to claim 1, wherein After the model training is completed, model federated inference based on PSI is performed, including: The active node and the passive node number the tree nodes and form a node list; The active node and the passive node sequentially remove the distinguishable nodes from the node list according to the features of their own predicted samples, and finally perform a private set intersection operation on the remaining nodes. The active node obtains the private intersection result and thus obtains the final sample division result.

6. The method according to claim 1, wherein Before the active node calculates the first-order derivative g and the second-order derivative h for each sample and encrypts and packs them using the fully homomorphic encryption algorithm CKKS and sends them to the passive node, the method further includes: In the initialization stage, the active node and the passive node agree on security parameters, a hash function, and a random function; In the pre-computation phase, the active node samples a random string The passive node initializes matrix D and samples a random PRF key Perform pre-computation; In the oblivious transfer stage, the active node and the passive node perform oblivious transfer w times in total; In the private set intersection stage, the active node calculates the OPRF value for each sample and sends the obtained set Ψ to the passive node. The passive node also calculates its own OPRF value ψ. If and only if ψ ∈ Ψ, it is the intersection of the two data sets, and the result is stored in a file.

7. A vertical federated learning device for a SecureBoost model based on fully homomorphic encryption, characterized in that The device is applied to a federated learning system. The federated learning system includes an active node and a passive node. The active node has class labels. The device includes: A derivative encryption module, configured to calculate the first-order derivative g and the second-order derivative h for each sample by the active node, and encrypt and pack them using the fully homomorphic encryption algorithm CKKS and send them to the passive node; A gradient aggregation module, configured to use the secure weighted quantile algorithm to find candidate split points by the passive node, aggregate the encrypted derivatives according to the results, and send the aggregated gradients to the active node; An optimal split point determination module, configured to determine the optimal split point using the bucket subtraction method based on the aggregated gradients sent by the passive node and its own aggregated gradients by the active node, and send the optimal split point to the corresponding passive node; A sample division module, configured to divide the sample data by the passive node according to the optimal split point, and send the divided sample data to the active node; A model update module, configured to update the node information and sample distribution of the tree model for this split by the active node according to the divided sample data; A repetition module, configured to repeat the above modules until the maximum depth of the tree to obtain a complete tree model; An inference module, configured to perform model federated inference based on PSI after the model training is completed.

8. An electronic device, comprising a memory, a processor, and a computer program stored on the memory and executable on the processor, characterized in that, When the processor executes the program, it implements the method according to any one of claims 1-6.

9. A computer-readable storage medium, characterized in that, It stores a computer program, and when the computer program is executed by a processor, it implements the method according to any one of claims 1-6.

10. A computer program product comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the method according to any one of claims 1-6.

Images