Privacy protection method based on decision tree model training

Abstract

The invention provides a privacy protection method based on decision tree model training, which comprises the following steps that: an active participant synchronizes decision tree model training related hyper-parameters and security parameters to a passive participant, generates a plurality of indication vectors for indicating different types of samples, and secretly shares all the indication vectors with the passive participant; pre-calculation is carried out by each participant, samples are scattered into different buckets according to the characteristics of the participant, and corresponding indication vectors are recorded; each participant selects a random numerical value thereof and aggregates the random numerical value with the random vectors encrypted by other participants, and each participant calculates statistical information of the random vector of the ciphertext according to the channel indication vector and converts the ciphertext into a secret sharing form; and each participant randomizes the indication vectors of different labels and performs secure multi-party calculation. The communication overhead can be reduced, and the overall calculation efficiency of the scheme can be improved.

Description

Technical Field

[0001] The present invention belongs to the technical field of federated learning privacy protection, and specifically relates to a privacy protection method based on decision tree model training. Background Art

[0002] The statements in this section merely provide background information related to the present invention and do not necessarily constitute prior art.

[0003] The training of machine learning models generally requires a large amount of sample data support, but when a single participant does not have enough sample data or has limited computing power, a distributed training method is required, that is, multiple participants collaborate to complete the model training task. In a distributed model, sample data generally contains sensitive content involving personal privacy and institutional secrets. How to ensure the security and privacy of data without affecting the training task is often a concern for researchers, corporate institutions and governments.

[0004] Federated learning, as a distributed learning paradigm, is proposed to solve the privacy problem in the model training process. It allows all participants to collaborate to complete the training task of the global model without uploading local data. Federated learning can be classified into horizontal federated learning and vertical federated learning according to the way each participant divides the sample data. In the former, each participant holds a different sample space but the samples of each participant share the same feature space, while in the latter, each participant holds the same sample space but the samples of different participants share different and often non-overlapping feature spaces. Whether it is horizontal federated learning or vertical federated learning, it follows the following basic steps: 1) Synchronize the global model, that is, complete the initialization or update of the model; 2) Each participant uses its local sample data to train the model and obtain a local update of the model; 3) Aggregate the local updates of each participant and obtain a global update. 4) Repeat 1) to 3) until the model converges. In the horizontal federated learning scenario, there are generally multiple clients holding sample data and one or more servers. The server does not directly participate in the model training task except for aggregating results, verification and data forwarding, and the model training task is directly performed by the client using its sample data set locally. In vertical federated learning scenarios, there are generally two roles: the master client and the slave client. The master client holds some features of the sample data and the label information of the samples and coordinates the process of building the tree model, while the slave client only holds some feature information of the sample data and completes the model training task under the coordination of the master client.

[0005] In the machine models of horizontal federated learning, although models such as deep neural networks, convolutional neural networks, and long short-term memory have excellent performance in computer vision and natural language processing, and can complete difficult and complex tasks such as image classification, numerical regression, and language pattern recognition, their models generally do not have good interpretability. In vertical federated learning, tree models such as decision trees, gradient boosted decision trees, and decision forests have good interpretability compared to non-tree models due to the characteristics of their model structure, and therefore have attracted widespread attention from industries such as finance and medicine.

[0006] The decision tree model is composed of nodes, and its overall structure is an inverted comb. Nodes can be divided into internal nodes and leaf nodes: internal nodes mainly store information related to splitting, such as features, threshold information, etc., which determine the direction of the decision path in the tree model; while leaf nodes mainly store classification and regression related information, such as categories, weights, etc., which determine the final output of the model. The construction process of the tree model is top-down and recursive. Generally, starting from the root node, the current node is continuously split according to certain rules and new nodes are generated. If the new node can continue to split, the node is an internal node; otherwise, the node is a leaf node. Tree models such as gradient boosting decision trees and decision forests are integrated from multiple decision tree models according to a rule. The rule affects the relationship between different decision trees, but in essence still follows the training paradigm of the decision tree model.

[0007] In horizontal federated learning, there are many attack methods, such as member reasoning attack and model reversal attack, which can be successfully implemented and destroy the security of the protocol, thus leading to the leakage of private information. Although the tree model structure is relatively discrete compared to the horizontal federated learning model, more and more studies have shown its potential privacy and security threats, leading to the leakage of the distribution characteristics of the training sample labels and sample feature space. The construction of the secure training protocol of the tree model mainly relies on secure multi-party computing technology based on secret sharing and partially homomorphic or fully homomorphic encryption technology. The current protocol construction scenarios are mostly aimed at secure two-party computing, while multi-party scenarios are less considered, and many protocols rely on strong assumptions, such as some intermediate calculation results can be directly used in plaintext to participate in the calculation, and some participants are honest and non-curious. In order to achieve the expansion from the two-party training scenario to the multi-party training scenario, in addition to solving the problems in the calculation process, it is necessary to consider how to ensure the privacy and security of the training sample data and the model when multiple participants collude.

[0008] The current security protocol construction for multi-party scenarios is based on the secure multi-party computing technology of secret sharing. The existing technology can well support nonlinear operations in the decision tree training process, such as division and comparison, but it usually faces the problem of high communication complexity. This problem comes from the need to calculate the inner product of the vector indicating the sample split and the label or gradient vector when each participant needs to calculate its statistical information for the possible split values ​​of its different features. However, in order to avoid leaking the distribution information of its corresponding features, this operation is generally implemented using multiplication and addition based on secret sharing technology, and multiplication requires the exchange of a large amount of data information. The essence of the above problem is that the split-related indicator vectors generated by each participant locally by sorting the features cannot be directly synchronized with other participants. In order to reduce the communication complexity, some construction schemes introduce homomorphic encryption technology to enable each participant to use its feature sorting information locally to sort the encrypted labels or gradient vectors accordingly to solve this problem. However, the problem directly brought about by homomorphic encryption technology is high computational complexity. Existing technical solutions adopt a technical route that combines secure multi-party computing technology and homomorphic encryption technology to compromise communication overhead and computing overhead, but the solutions still have many shortcomings, such as failing to fully utilize some of the calculation results, which leads to increased computational complexity. Summary of the invention

[0009] In order to solve the above problems, the present invention proposes a privacy protection method based on decision tree model training, which can reduce communication overhead and improve the overall computing efficiency of the solution.

[0010] According to some embodiments, the present invention adopts the following technical solutions:

[0011] A privacy protection method based on decision tree model training includes the following steps:

[0012] The active participant synchronizes the hyperparameters and security parameters related to the training of the decision tree model with the passive participant, and generates multiple indicator vectors indicating different categories of samples, and secretly shares all the above indicator vectors with the passive participant;

[0013] Each participant performs pre-calculation, scatters samples into different buckets according to their characteristics and records the corresponding indicator vectors;

[0014] Each participant selects its random value and aggregates it with the random vectors encrypted by other participants. Each participant calculates the statistical information of the ciphertext random vector based on the channel indicator vector and converts the ciphertext into a secret sharing form.

[0015] Each participant randomizes the indicator vectors of different labels and performs secure multi-party computation. Each participant compares all split gains, determines the corresponding indicator vectors, and calculates the Gini impurity and split gain Gain of all possible split situations.

[0016] Determine whether the pre-pruning condition is met. If yes, each participant calculates the current node as an internal node and updates the share vectors of different labels. If no, each participant calculates the current node as a leaf node and determines the classification information of the leaf node.

[0017] Determine whether there are still splittable internal nodes. If so, return to iterative execution of the steps of randomizing the indication vectors of different labels of each participant and performing secure multi-party computation. Otherwise, terminate the above process.

[0018] As an optional implementation, the decision tree model training related hyperparameters include maximum depth, maximum number of leaf nodes in the tree, minimum number of node split samples, minimum split threshold and maximum allowed number of buckets;

[0019] The security parameters include the Paillier key size, the calculation domain and the corresponding modulus and statistical parameters.

[0020] As an optional implementation, the process of generating multiple indicator vectors indicating different categories of samples and secretly sharing all the above indicator vectors with the passive participants includes:

[0021] The active participant initializes the vector indicating the current node that can be used to split the sample and generates multiple indicator vectors indicating its different categories based on the sample label, and shares all the above indicator vectors secretly with the passive participant; the passive participant needs to receive and store the initialization parameter, and all participants need to sort the sample data according to each feature they hold, and then scatter it evenly into each bucket according to the maximum number of buckets of the parameter and record the corresponding indicator vector.

[0022] As an optional implementation method, each participant performs pre-calculation, scatters samples into different buckets according to their characteristics and records the corresponding indicator vectors. The process includes: each participant generates a local random vector and encrypts it; each participant aggregates all local random vectors into a global random vector; each participant selects components in the global random vector according to the recorded bucket indicator vector to aggregate to obtain its corresponding aggregated statistical result for global randomness and converts it into a secret sharing form; each participant stores the generated local random mask vector and the secret share of the global randomness aggregated statistical result for subsequent calculations.

[0023] As an optional implementation, each participant performs pre-calculation, scatters samples into different buckets according to their characteristics and records the corresponding indicator vectors, and all random values ​​generated in the pre-calculation stage are not reused and are removed after use and their positions are covered by subsequent values;

[0024] The process is executed in a loop and linear parallel processing technology is used to accelerate the calculation.

[0025] As an optional implementation method, each participant selects its random value and aggregates it with the random vectors encrypted by other participants. Each participant calculates the statistical information of the ciphertext random vector based on the channel indicator vector. The process of converting the ciphertext into a secret sharing form includes: each participant uses the generated local randomness mask to add it to the share vector indicating different category labels held by it; each participant aggregates its calculation results, synchronizes and obtains the masked indicator vector; each participant aggregates the elements of the corresponding positions of the masked indicator vector according to the generated indicator vectors for different buckets of different features and accumulates the statistical data of each bucket one by one to obtain the statistical data related to different splitting points under the corresponding feature; each participant again uses the secret share of the calculated global statistical data to eliminate the mask in the calculated statistical data and obtain the form of its secret share.

[0026] As an optional implementation method, each participant compares all split gains, determines the corresponding indicator vector, and calculates the Gini impurity and split gain Gain of all possible split situations, including: each participant uses a security comparison algorithm and a dichotomy method to calculate the local optimal split gain of different features one by one in a certain order; each participant continues to obtain the local optimal split gain and the global optimal split gain in the same way;

[0027] During the comparison process, all possible splits will be indexed according to the participants and their characteristics, and each participant needs to record the comparison results during the calculation of the local best split gain;

[0028] Each participant uses the corresponding comparison results to construct an indication vector indicating the best splitting point from top to bottom. Each participant encrypts the share of the indication vector. The participant holding the splitting point aggregates the ciphertext and uses the privacy information retrieval technology locally to obtain the corresponding ciphertext of the indication vector of how to split the current node. Each participant needs to jointly convert the ciphertext into a secret share form.

[0029] As an optional implementation, it is determined whether the pre-pruning condition is satisfied. If yes, each participant calculates the current node as an internal node and updates the share vectors of different labels. If no, each participant calculates the current node as a leaf node and determines the classification information of the leaf node. The process includes: each participant checks whether the pre-pruning condition is satisfied. If yes, the type of the current tree node is an internal node, otherwise it is a leaf node;

[0030] If the current node is an internal node, each participant uses the vector indicating how to split the current node to update the vector indicating different categories, and further determines the types of the left and right child nodes of the current node through training; if the current node is a leaf node, each participant needs to update the corresponding information of the leaf node according to the training task.

[0031] A computer-readable storage medium is used to store computer instructions, and when the computer instructions are executed by a processor, the steps in the above method are completed.

[0032] An electronic device comprises a memory and a processor, and computer instructions stored in the memory and executed on the processor. When the computer instructions are executed by the processor, the steps in the above method are completed.

[0033] Compared with the prior art, the present invention has the following beneficial effects:

[0034] The present invention does not use homomorphic encryption technology when constructing vectors for indicating different categories of samples in the current node, but adopts the form of linear secret sharing, which has the following advantages: 1) compared with homomorphic encryption technology, linear sharing has higher computational efficiency; 2) when updating the indication vector, there is no need to use linear secret sharing and the semi-homomorphic properties of the encryption scheme to simulate the multiplication between ciphertexts, and there is no need to convert the ciphertext into a secret sharing form and then restore it to its ciphertext form. Only direct multiplication between secret shares is required, which has higher communication efficiency.

[0035] The present invention makes full use of the comparison results between different gain information, thereby avoiding the costly equality test based on secret sharing when constructing the vector required by the privacy information retrieval (PIR) technology, making the construction of the scheme simpler and more efficient.

[0036] The present invention makes full use of the invariants and consistency of operations in the decision tree training process, and packs multiple ciphertext information into a single ciphertext, thereby reducing the waste of plaintext space and improving the parallel capability of training data processing.

[0037] Based on the above, compared with existing solutions, the present invention can further improve the speed of model training and communication overhead without sacrificing system security.

[0038] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, preferred embodiments are given below and described in detail with reference to the accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] The accompanying drawings in the specification, which constitute a part of the present invention, are used to provide a further understanding of the present invention. The exemplary embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute improper limitations on the present invention.

[0040] Figure 1 This is a system architecture diagram of the present invention;

[0041] Figure 2 This is a decision tree training flow chart of the present invention;

[0042] Figure 3 A communication flow diagram between the parties involved in the present invention;

[0043] Figure 4 This is a schematic diagram of ciphertext packaging of the present invention;

[0044] Figure 5 The figure is a schematic diagram showing the principle of constructing an indicator vector using intermediate calculation results in the present invention. DETAILED DESCRIPTION

[0045] The present invention will be further described below in conjunction with the accompanying drawings and embodiments.

[0046] It should be noted that the following detailed descriptions are all illustrative and intended to provide further explanation of the present invention. Unless otherwise specified, all technical and scientific terms used herein have the same meanings as those commonly understood by those skilled in the art to which the present invention belongs.

[0047] It should be noted that the terms used herein are only for describing specific embodiments and are not intended to limit exemplary embodiments according to the present invention. As used herein, unless the context clearly indicates otherwise, the singular form is also intended to include the plural form. In addition, it should be understood that when the terms "comprising" and / or "including" are used in this specification, it indicates the presence of features, steps, operations, devices, components and / or combinations thereof.

[0048] In the absence of conflict, the embodiments in this application and the features in the embodiments may be combined with each other.

[0049] A privacy-preserving method based on decision tree model training, or a privacy-preserving decision tree model training method, such as Figure 1 As shown in the figure, the (t,m) thresholded partially homomorphic Paillier encryption and secure multi-party computation based on secret sharing are used as technical support, which is characterized by 1) including two types of participants: active participant P0 and passive participant P i∈{1...m-1} , where P0 holds some features of the sample and the corresponding label information, P i Only partial feature information is held, and the feature information held by each participant is different; multiple ciphertexts with the same operation are packaged into a single ciphertext, such as Figure 4 As shown in Figure 1, the calculation of the required statistical data is divided into a pre-calculation phase and a training phase. i∈{1...m-1} Under the coordination of P0, necessary data information is exchanged to complete the training of the tree model.

[0050] like Figure 2 As shown, the method comprises the following steps:

[0051] Step 1: Initialization phase. This phase is only executed once during the training of the decision tree model. The active participants synchronize model training related hyperparameters with the passive participants, such as the maximum depth, the maximum number of leaf nodes, the minimum number of node split samples, the minimum split gain, and the number of buckets (which determines the maximum number of feature split points).

[0052] The active participant initializes the vector indicating the current node that can be used to split the sample and generates multiple indication vectors indicating different categories according to the sample label, and secretly shares all the above indication vectors with the passive participant.

[0053] like Figure 3 As shown, the passive participants need to receive and store the initialization parameters, and all participants need to sort the sample data according to each feature they hold, and then scatter them evenly into each bucket according to the maximum number of buckets of the parameter and record the corresponding indicator vector.

[0054] Step 2: Pre-computation phase: This phase is different from general pre-computation techniques in that it is not independent of data input.

[0055] The pre-computation concept used in the present invention is only based on the invariance of data input during the training process, that is, it indicates that the sample vector in each bucket does not change during the training process, so as to improve the parallel processing capability of the solution.

[0056] In this phase, each participant collaborates to generate random data for the training phase. The random data consists of two parts: one part is used as a random mask for the indicator category vector in the training phase; the other part is used to convert the ciphertext into a secret sharing form. First, each participant generates a local random vector and encrypts it; then, each participant aggregates all local random vectors into a global random vector; finally, each participant selects the components in the global random vector according to the indicator vector of the bucket recorded in the initialization phase, aggregates them to obtain the corresponding aggregated statistical results for global randomness and converts them into a secret sharing form. Each participant stores the generated local random mask vector and the secret share of the global random aggregate statistical results for subsequent calculations.

[0057] Step 3, local calculation stage. In this stage, each participant uses the local randomness mask generated in the preprocessing stage to add to the share vector indicating different category labels held by it; then, each participant aggregates its calculation results, synchronizes and obtains the masked indication vector. Each participant aggregates the elements of the corresponding positions of the masked indication vector according to the indication vectors of different buckets for different features generated in the initialization stage and accumulates the statistics of each bucket one by one to obtain the statistics related to different splitting points under the corresponding feature. Each participant again uses the secret share of the global statistics calculated in the pre-calculation stage to eliminate the mask in the calculated statistics and obtain the form of its secret share.

[0058] Step 4: Secure multi-party computing stage. In this stage, each participant uses its existing secure multi-party computing technology to interactively implement the calculation of relevant mathematical formulas, such as the Gini impurity used by CART in classification tasks and the sample variance in regression tasks. In addition to calculating the Gini impurity / sample variance of the current node, each participant still needs to calculate the corresponding data on the left and right sides formed by different splitting points under different features and calculate the splitting gain generated by the splitting point. Each participant uses a secure comparison algorithm and a binary search method to calculate the local optimal splitting gain of different features one by one in a certain order; then, each participant continues to obtain the local optimal splitting gain and the global optimal splitting gain in the same way. During the comparison process, all possible splits will be indexed according to the participants and their characteristics, and each participant needs to record the comparison results in the process of calculating the local optimal splitting gain. Each participant uses the corresponding comparison results from top to bottom as follows Figure 5 As shown in the figure, the indicator vector indicating the best splitting point is 0 except for the best splitting point. Each participant encrypts the share of the indicator vector, and the participant holding the splitting point aggregates the ciphertext and uses the privacy information retrieval (PIR) technology locally to obtain the corresponding ciphertext of the indicator vector of how to split the current node. Each participant needs to jointly convert the ciphertext into the form of a secret share.

[0059] Step 5: Model update phase. In this phase, each participant checks whether the pre-pruning conditions are met according to the initialization phase, such as whether the tree reaches the maximum tree depth and the number of leaf nodes, the minimum number of splits and the minimum split threshold, etc., to determine the type of the current tree node, that is, whether it is an internal node or a leaf node. If the current node is an internal node, each participant needs to use the vector indicating how to split the current node to update the vector indicating different categories, that is, it is necessary to further determine the types of the left and right child nodes of the current node through training; if the current node is a leaf node, each participant needs to update the corresponding information of the leaf node according to the training task, such as the maximum number of label information in the classification task and the mean information of the sample label in the regression task.

[0060] The above-mentioned local computing phase, secure multi-party computing phase, and model updating phase need to be executed in sequence according to the training needs of the tree model.

[0061] The following is a detailed description of each step:

[0062] The step 1 specifically includes:

[0063] 1.1P0, as an active participant, selects appropriate hyperparameters according to the training task for pre-pruning to avoid overfitting of the model, such as the maximum tree depth MAX_DEPTH, the maximum number of tree nodes MAX_NODES_NUM, the minimum number of node split samples MIN_SPLIT_NUM, the minimum split threshold MIN_SPLIT_THESH, the maximum number of allowed buckets MAX_BINS_NUM, and security-related parameters, such as the Paillier key size KEY_SIZF, the computational domain F and the corresponding modulus p and statistical parameter κ. P0 provides other passive participants P i∈{1...m-1} Synchronize the above hyperparameters and security parameters.

[0064] 1.2P i According to the set of characteristics it possesses Each feature f in the sample X is sorted by value, and the unique value val in the feature is counted unique If |val unique |≤MAX_BINS_NUM, then according to val unique Each value in determines the set of indicator vectors belonging to the samples in each bucket Otherwise, certain buckets identified by unique values ​​will be merged according to the rules agreed upon by all parties.

[0065] 1.3P0 derives vectors β indicating different categories based on the label vector y of the samples it holds k :=

[0066] (β k [1],...,β k [N]) and where β k [j] = 1 if and only if X[j] belongs to category k; otherwise, β k [j] = 0. P0 vs. β k Secret sharing is <β k >∶=(<β k [1]>,...,<β k [N]>), each participant P i Holding share <β k > i ∶=(<β k [1]> i ,...,<β k[N]> i ),in And <β k [j]> i ∈ R {0...2 k}.

[0067] 1.4P i According to KEY_SIZE, the (m,m) thresholded Paillier key generation protocol is executed in a distributed manner. In the present invention, the threshold value is set to m, which means that all participants are required to correctly decrypt. When the protocol ends, each participant obtains the public key and its partial private key pair (pk, sk u ).

[0068] The step 2 is specifically as follows:

[0069] 2.1P u From {0...2 k+k+v -1} uniformly randomly generates l sets of mask vectors of size N, and the set composed of the mask vectors is denoted as {mask i,l} l ; then compress it into a single mask vector mask i ,in j∈{0...N-1}, offset∶=k+2(k+v)+logN; finally, encrypt it with the public key pk to obtain [[mask i ]].

[0070] 2.2P i From {0...2 k+2(k+v)+logN -1{Uniformly randomly generate B groups of random secret share vectors of size l, and record the set composed of the random share vectors as {share_r i,B} B ,in This is then compressed into a single share vector share_r i ,in j∈{0...B-1}, offset∶=k+2(κ+v)+logN; finally, encrypt it with the public key pk to obtain

[0071] 2.3P i∈{1...m-1} Will and Send to P0; P0 uses Paillier's homomorphic properties to calculate and P0 and Synchronize to P i∈{1...m-1} .

[0072] 2.4P i According to INV i The indicator vector of each bucket of different features in is used to calculate the statistics in each bucket in a "select"-"sum" manner. in b∈{0...MAX_BINS_NUM-1}.

[0073] 2.5P i use Calculate the prefix and statistics belonging to different features at different split points in

[0074] 2.6P i From the reshare share vector Select The element composition has TEMPi matrix with MAX_BINS_NUM rows and columns and calculate P i First Send to P j≠i ; by P j≠i Use its private key sk j right Perform partial decryption and return the partial decryption result to P i ;P i Aggregate all from P j≠i The partial decryption results are obtained <SUM i > i And unpack it into { <SUM i,l > i} l , and P j≠i Just replace { <SUM j,l > j} l The conversion from ciphertext to share can be completed by setting it to the corresponding random value in the set of re-secret sharing vectors generated in 2.2.

[0075] The above steps 2.1 to 2.6 are executed in a loop according to the training needs of the data model, and the process can be further accelerated by using linear parallel processing technology. In addition, all random values ​​generated in the pre-calculation stage are not reused and are removed after use and replaced by subsequent values, that is, each selection is made from the beginning. For the convenience of description, redundant subscripts are no longer given without affecting understanding.

[0076] The step three is as follows:

[0077] 3.1Pi From {mask i,l} l Select k groups and calculate <λ k > i ∶=<β k > i +mask i,k , and <λ k > i Send to P0. P0 aggregates all shares to obtain λ k And synchronize it to all other participants P i∈{1...m-1} .

[0078] 3.2P i According to INV i The indicator vector of each bucket of different features in is used to calculate the statistics AGG in each bucket in a "select"-"sum" manner i,left ,in b∈{0...MAX_BINS_NUM-1}.

[0079] 3.3P i Using AGG i,left,k Calculate the frequency statistics FREQ of samples divided into each category in the left child node under different features and different splitting points i,left ,in P i from{ <SUM i,l > i} l Select k and combine it into the corresponding matrix form <SUM i > i And calculate <FREQ i,left > i : = FREQ i,left + <SUM i > i , the corresponding frequency statistics of each category in the right child node can be indirectly calculated as That is, the frequency of each type of label minus the frequency of the corresponding category scattered in the left child node.

[0080] 3.4P i Will <num>i i Set to the number of samples that can be used for splitting in the left and right child nodes calculated in the parent node of the current node. The value is initially N. i The frequency information of samples with different features and different split points can be used to calculate the number of samples that can be used in the left child node. It can be used for samples in the right node and can be calculated as <NUM i,right > i := <num> i - <NUM i,left > i .

[0081] 3.5P i according to where p l is the sampling probability of sample label l, that is, Pr(y=l), and the node before splitting is calculated respectively. <gini>And after the split left <Gini i,left >, right child node <Gini i,right > the impurity information and the corresponding p l Should also be replaced by p i,left,l 、p i,right,l .P i Statistics calculated using 3.3 and 3.4 <num> i , <NUM i,left > i , <NUM i,right > i and <FREQ i,l,eft>i , <FREQ i,right > i According to the relationship between probability and frequency Calculate the required probability values ​​above.

[0082] 3.6P i According to Gain∶=w left ·Gini left +w right ·Gini right -Gini, among

[0083] And use the impurity information calculated in 3.5 <gini>, <Gini i,left > and <Gini i,right > Calculate the gain information generated by splitting the current node with different features and different splitting points <Gain i > i .

[0084] 3.7 Participants P i First, the gain information held by each feature is indexed in natural order to identify the user and feature to which it belongs; then, the gain information is compared using the binary search method to obtain the local optimal split gain relative to each feature. <Gain i,best > i ; Then, continue to use the binary method to compare the local optimal split gain to obtain the global optimal split gain <Gain best > i Specifically, in the binary comparison, each participant P u Compare two adjacent gain values ​​and use max( <gain1> i , <gain2> i )∶＝ <gain1> i + <flag> i ·( <gain1> i - <gain1> i ) Filter and obtain the maximum value for the next round of gain information comparison and record and update the corresponding comparison information <flag> i , <P * > i , <f * > i ,in <flag> i ∶=1 <gain1> i≥ <gain2> i < / gain2> < / gain1> and <P * > i , <f * > i Used to record the users and features described by the maximum value. Each participant P i The above is repeated until there is no more gain information to compare, that is, the maximum gain information has been obtained. ∈{1...m-1} The share held by <P * > i and <f * > i Send to P0; P0 reconstructs P * and f * And give it to other participants P ∈{1...m-1} .

[0085] The step 4 is as follows:

[0086] 4.1P i According to P * and f * Retrieve the records of the comparison process at different comparison levels <flag> i , starting from the root node, update the current node layer by layer from top to bottom <flag> i ∶ <flag> i · <flag parent > i ; At this time, among all the leaf nodes, there is only one leaf node. <flag> i ∶=1 and the other leaf nodes are all 0, each participant P i Constructed by leaf nodes <flag> i The vector composed of <inv * > i The only record is the position of the best split point index in the vector. i Use the public key pk <inv * > i Encrypted as and send it to P * ;P * Will Aggregate

[0087] 4.2P * Construct the matrix TEMP locally, whose row vectors are determined by the characteristics f * The indicator vectors derived from different splitting points in are formed; then, P * Calculation using PIR technology The multiplication between the vector and the TEMP matrix finally obtains the required indicator vector P * First The elements are evenly divided into Then, each group of ciphertexts is shifted by its plaintext value (k+κ+v)*(lz-1), and each group of ciphertexts is multiplied to obtain a vector of size N′ That is to say Where z represents the position of the ciphertext in its group.

[0088] 4.3P i From {0,...,2 (k+κ+v) }Uniformly randomly generate a random share vector of size N <r> i , and pack the vector into a random share vector of size N′ in the same way as in 4.2<r′> i And encrypt it using the public key pk as Then send to P * .P * calculate and broadcast it to P i ;P i Use its private key sk i right Perform partial decryption and send the result to P * ;P * Aggregate from other participants P i The partial decryption result of P is obtained u′, and it is deconstructed into a vector u of size N. * Will * Set to u- <r> * , while other participants P i∈{0,...,m-1}\* Will i Set to <r> i Complete the conversion from ciphertext to secret sharing shares.

[0089] The step five is as follows:

[0090] 5.1P0 Check whether the current training depth DEPTH and the number of nodes NODES_NUM are both less than their preset values ​​MAX_DEPTH and MAX_NODES_NUM; i Check the current number of nodes <num> i and maximum split gain <Gain best > i Are they both less than their preset values ​​MIN_SPLIT_NUM and MIN_SPLIT_THESH? The check result is reconstructed at P0 and sent to other participants P0. i∈{1,...,m-1} Broadcast the result.

[0091] 5.2 If all condition checks in 5.1 are passed, mark the current node as an internal node and execute this step. i Update the indicator vectors of different labels in the left child node to <β left,k > i ∶=<β k > i ⊙ i ; accordingly, P i The indicator vectors indicating different labels in the right child node can be updated to <β right,k > i ∶=<β k > i -<β left,k > i .

[0092] 5.3 If the condition check in 5.1 fails, mark the current node as a leaf node and execute this step. i Calculate the frequency of different categories in the current node as Select the label with the maximum frequency by comparison <k> i As the classification information of the current leaf node.

[0093] Those skilled in the art will appreciate that embodiments of the present invention may be provided as methods, systems, or computer program products. Therefore, the present invention may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0094] The present invention is described with reference to flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0095] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0096] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0097] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. For those skilled in the art, the present invention may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made by those skilled in the art within the spirit and principle of the present invention without creative labor shall be included in the protection scope of the present invention.< / k> < / num> < / r> < / r> < / r> < / flag> < / flag> < / flag> < / flag> < / flag> < / flag> < / flag> < / gain1> < / flag> < / gain1> < / gain2> < / gain1> < / gini> < / num> < / gini> < / num> < / num>

Claims

1. A privacy protection method based on decision tree model training, characterized in that: The following steps are involved: The active participant synchronizes the hyperparameters and security parameters related to the training of the decision tree model with the passive participant, and generates multiple indicator vectors indicating different categories of samples, and secretly shares all the above indicator vectors with the passive participant; Each participant performs pre-calculation, scatters samples into different buckets according to their characteristics and records the corresponding indicator vectors; Each participant selects its random value and aggregates it with the random vectors encrypted by other participants. Each participant calculates the statistical information of the ciphertext random vector based on the channel indicator vector and converts the ciphertext into a secret sharing form. Each participant randomizes the indicator vectors of different labels and performs secure multi-party computation. Each participant compares all split gains, determines the corresponding indicator vectors, and calculates the Gini impurity and split gain Gain of all possible split situations. Determine whether the pre-pruning condition is met. If yes, each participant calculates the current node as an internal node and updates the share vectors of different labels. If no, each participant calculates the current node as a leaf node and determines the classification information of the leaf node. Determine whether there are still splittable internal nodes. If so, return to iterative execution of the steps of randomizing the indication vectors of different labels of each participant and performing secure multi-party computation. Otherwise, terminate the above process.

2. A privacy protection method based on decision tree model training as claimed in claim 1, characterized in that: The decision tree model training related hyperparameters include maximum depth, maximum number of leaf nodes in the tree, minimum number of node split samples, minimum split threshold and maximum number of buckets allowed; The security parameters include the Paillier key size, the calculation domain and the corresponding modulus and statistical parameters.

3. The privacy protection method based on decision tree model training as claimed in claim 1, characterized in that: The process of generating multiple indicator vectors of different categories of indicator samples and secretly sharing all of the above indicator vectors with passive participants includes: The active participant initializes the vector indicating the current node that can be used to split the sample and generates multiple indicator vectors indicating its different categories based on the sample label, and shares all the above indicator vectors secretly with the passive participant; the passive participant needs to receive and store the initialization parameter, and all participants need to sort the sample data according to each feature they hold, and then scatter it evenly into each bucket according to the maximum number of buckets of the parameter and record the corresponding indicator vector.

4. The privacy protection method based on decision tree model training as claimed in claim 1, characterized in that: Each participant performs pre-calculation, scatters samples into different buckets according to their characteristics, and records the corresponding indicator vectors. The process includes: each participant generates a local random vector and encrypts it; each participant aggregates all local random vectors into a global random vector; each participant selects components in the global random vector according to the recorded bucket indicator vector to aggregate to obtain the corresponding aggregated statistical result of global randomness and converts it into a secret sharing form; each participant stores the generated local random mask vector and the secret share of the global randomness aggregated statistical result for subsequent calculations.

5. The privacy protection method based on decision tree model training as claimed in claim 1, characterized in that: Each participant performs pre-calculation, scatters samples into different buckets according to their characteristics and records the corresponding indicator vectors. All random values ​​generated in the pre-calculation stage are not reused and are removed after use and replaced by subsequent values. The process is executed in a loop and linear parallel processing technology is used to accelerate the calculation.

6. The privacy protection method based on decision tree model training as claimed in claim 1, characterized in that: Each participant selects its random value and aggregates it with the random vectors encrypted by other participants. Each participant calculates the statistical information of the ciphertext random vector based on the channel indicator vector. The process of converting the ciphertext into a secret sharing form includes: each participant uses the generated local randomness mask to add it to the share vector indicating different category labels held by it; each participant aggregates its calculation results, synchronizes and obtains the masked indicator vector; each participant aggregates the elements of the corresponding positions of the masked indicator vector according to the indicator vectors of different buckets generated for different features and accumulates the statistical data of each bucket one by one to obtain the statistical data related to different splitting points under the corresponding feature; each participant again uses the secret share of the calculated global statistical data to eliminate the mask in the calculated statistical data and obtain the form of its secret share.

7. The privacy protection method based on decision tree model training as claimed in claim 1, characterized in that: The process in which each participant compares all split gains, determines the corresponding indicator vector, and calculates the Gini impurity and split gain Gain of all possible split situations includes: each participant uses the security comparison algorithm and the dichotomy method to calculate the local optimal split gain of different features one by one in a certain order; each participant continues to obtain the local optimal split gain and the global optimal split gain in the same way; During the comparison process, all possible splits will be indexed according to the participants and their characteristics, and each participant needs to record the comparison results during the calculation of the local best split gain; Each participant uses the corresponding comparison results to construct an indication vector indicating the best splitting point from top to bottom. Each participant encrypts the share of the indication vector. The participant holding the splitting point aggregates the ciphertext and uses the privacy information retrieval technology locally to obtain the corresponding ciphertext of the indication vector of how to split the current node. Each participant needs to jointly convert the ciphertext into a secret share form.

8. The privacy protection method based on decision tree model training as claimed in claim 1, characterized in that: Determine whether the pre-pruning condition is met. If yes, each participant calculates the current node as an internal node and updates the share vectors of different labels. If no, each participant calculates the current node as a leaf node and determines the classification information of the leaf node. The process includes: each participant checks whether the pre-pruning condition is met. If yes, the type of the current tree node is an internal node, otherwise it is a leaf node; If the current node is an internal node, each participant uses the vector indicating how to split the current node to update the vector indicating different categories, and further determines the types of the left and right child nodes of the current node through training; if the current node is a leaf node, each participant needs to update the corresponding information of the leaf node according to the training task.

9. A computer-readable storage medium, characterized in that: Used to store computer instructions, which, when executed by a processor, complete the steps of the method according to any one of claims 1 to 8.

10. An electronic device, characterized in that: The method comprises a memory and a processor, and computer instructions stored in the memory and executed on the processor, wherein when the computer instructions are executed by the processor, the steps in the method according to any one of claims 1 to 8 are completed.

Images