A privacy protection method based on decision tree model training

Abstract

The application provides a privacy protection method based on decision tree model training, active participation direction passive participation party synchronizes decision tree model training related hyperparameters and security parameters, and generates a plurality of indication vectors indicating different categories of samples, and all the above indication vectors are secret shared with the passive participation party; each participation party precomputes, scatters the samples into different barrels according to the characteristics and records the corresponding indication vectors; each participation party selects a random value and aggregates the encrypted random vectors of other participation parties, each participation party calculates the statistical information of the ciphertext random vectors according to the channel indication vector, and converts the ciphertext into a secret sharing form; each participation party randomizes the indication vectors of different labels and performs secure multi-party computation. The application can reduce communication overhead and improve the overall calculation efficiency of the scheme.

Description

TECHNICAL FIELD

[0001] The present application belongs to the technical field of federated learning privacy protection, and particularly relates to a privacy protection method based on decision tree model training. BACKGROUND

[0002] The statements in this section merely provide background information related to the present application and do not necessarily constitute the prior art.

[0003] The training of a machine learning model generally requires a large amount of sample data support, but when the sample data of a single participant is insufficient or the computing power is limited, a distributed training method is needed, that is, multiple participants cooperate to complete the training task of the model. In a distributed model, sample data generally contains sensitive content related to personal privacy and institutional secrets. How to ensure the security and privacy of data without affecting the training task is often a concern for researchers, enterprises and governments.

[0004] Federated learning, as a distributed learning paradigm, is proposed to solve the privacy problem in the model training process, allowing participants to collaborate to complete the training task of the global model without uploading local data. According to the way each participant divides the sample data, federated learning can be classified into horizontal federated learning and vertical federated learning. In the former, different participants hold different sample spaces but share the same feature space among their samples, while in the latter, different participants hold the same sample space but different samples between different participants share different and often disjoint feature spaces. Whether it is horizontal federated learning or vertical federated learning, it follows the following basic steps: 1) synchronize the global model, that is, complete the initialization or update of the model; 2) each participant trains the model using its local sample data to obtain a local update of the model; 3) aggregate the local updates from each participant and obtain a global update. 4) Repeat 1) to 3) until the model converges. In the horizontal federated learning scenario, there are usually multiple clients holding sample data and one or more servers. In addition to aggregating results, verifying and forwarding data, the server does not directly participate in the model training task, while the model training task is directly performed by the client using its sample data set locally. In the vertical federated learning scenario, it generally includes two roles of master client and slave client. The master client holds part of the sample data features and sample label information and coordinates the establishment process of the tree model, while the slave client only holds part of the sample data feature information and completes the training task of the model under the coordination of the master client.

[0005] In the horizontal federated learning machine model, although models such as deep neural networks, convolutional neural networks and long short-term memory have excellent performance in computer vision and natural language processing, and can complete difficult and complex tasks such as image classification, numerical regression and language pattern recognition, the model generally does not have good interpretability. In vertical federated learning, tree models such as decision trees, gradient boosting decision trees and decision forests have good interpretability due to the characteristics of their model structure, and are therefore widely concerned by the financial and medical industries.

[0006] A decision tree model is composed of nodes, and its overall structure is an inverted comb. Nodes can be divided into internal nodes and leaf nodes: internal nodes mainly store information related to splitting, such as feature and threshold information, which determines the direction of the decision path in the tree model; and leaf nodes mainly store classification and regression related information, such as categories and weights, which determine the final output result of the model. The construction process of a tree model is top-down and recursive, generally starting from the root node and continuously splitting the current node according to certain rules to generate new nodes. If the new node can continue to split, the node is an internal node; otherwise, the node is a leaf node. Tree models such as gradient boosting decision trees and decision forests are integrated according to a rule, which affects the relationship between different decision trees, but essentially still follow the training paradigm of decision tree models.

[0007] In horizontal federated learning, there are many attack methods, such as member inference attacks and model inversion attacks, which can successfully implement and compromise the security of the protocol, resulting in the leakage of private information. Although tree models are relatively discrete compared to horizontal federated learning models, more and more research indicates their potential privacy security threats, leading to the leakage of training sample labels and sample feature space distribution characteristics. The construction of secure training protocols for tree models mainly relies on secure multi-party computation technology based on secret sharing and partial homomorphic or fully homomorphic encryption technology. Current protocol construction scenarios are mostly for secure two-party computation, with less consideration for multi-party scenarios, and many protocols rely on strong assumption conditions, such as some intermediate calculation results can be directly calculated in plaintext form, and some participants are honest non-curious. To achieve the expansion from two-party training scenarios to multi-party training scenarios, in addition to solving problems in the calculation process, we need to consider how to ensure the privacy and security of training sample data and models when multiple participants collude.

[0008] The current security protocol construction of multi-party scene is based on secret sharing secure multi-party computation technology. The existing technology can well support the nonlinear operation in the decision tree training process, such as division and comparison, but generally faces the problem of high communication complexity. The problem is caused by the need to calculate the inner product of the sample split indicating vector and the label or gradient vector when each participant needs to calculate the statistical information of its different feature possible split value. However, in order to avoid leaking the distribution information of its corresponding feature, the operation generally uses multiplication and addition based on secret sharing technology, and multiplication requires exchanging a large amount of data information. The essence of the above problem is that the split related indicating vector generated by each participant locally by sorting the features cannot be directly synchronized with other participants. In order to reduce the communication complexity, some construction schemes introduce homomorphic encryption technology to enable each participant to locally sort the encrypted label or gradient vector using its sorting information of the feature, thereby solving the problem. However, the problem directly caused by homomorphic encryption technology is high computational complexity. The existing technical solutions adopt the technical route of combining secure multi-party computation technology and homomorphic encryption technology to compromise the communication overhead and computational overhead, but there are still many deficiencies in the scheme, such as not fully utilizing part of the calculation results, thereby increasing the computational complexity. SUMMARY

[0009] The present application proposes a privacy protection method based on decision tree model training to solve the above problems. The present application can reduce the communication overhead and improve the overall computational efficiency of the scheme.

[0010] According to some embodiments, the present application adopts the following technical solutions:

[0011] A privacy protection method based on decision tree model training, comprising the following steps:

[0012] The active participant synchronizes the decision tree model training related hyperparameters and security parameters to the passive participant, and generates a plurality of indicating vectors indicating different categories of samples, and performs secret sharing of all the indicating vectors with the passive participant;

[0013] Each participant performs pre-computation, scatters the samples into different buckets according to its feature and records the corresponding indicating vector;

[0014] Each participant selects its random value and aggregates with the encrypted random vector of other participants, and each participant calculates the statistical information of the ciphertext random vector according to the channel indicating vector, and converts the ciphertext into a secret sharing form;

[0015] Each participant randomizes the indicating vector of different labels and performs secure multi-party computation, each participant compares all the split gains, determines the corresponding indicating vector, and calculates the Gini impurity and split gain Gain of all possible split cases.

[0016] determining whether the pre-pruning condition is satisfied, if yes, each participant calculates the current node as an internal node and updates the share vector of different labels, if no, each participant calculates the current node as a leaf node and determines the classification information of the leaf node;

[0017] determining whether there still exists a splittable internal node, if yes, returning to the step of iteratively performing the randomization of the indication vector of different labels by each participant and the secure multi-party computation, if no, ending the above process.

[0018] As an optional implementation, the decision tree model training related hyperparameters include maximum depth, maximum number of leaf nodes of the tree, minimum number of node split samples, minimum split threshold and maximum number of allowed buckets;

[0019] The security parameters include Paillier key size, calculation domain and corresponding modulus and statistical parameters.

[0020] As an optional implementation, the process of generating multiple indication vectors indicating different categories of samples and secret sharing all the indication vectors with the passive participant includes:

[0021] The active participant initializes the vector indicating the samples available for splitting in the current node and generates multiple indication vectors indicating different categories of samples according to the sample labels, and secret shares all the indication vectors with the passive participant; the passive participant needs to receive and store the initialization parameters, and all participants need to sort the sample data according to each feature they hold, and then scatter them into each bucket according to the parameter maximum number of buckets and record the corresponding indication vector.

[0022] As an optional implementation, each participant performs pre-computation, scatters the samples into different buckets according to their features and records the corresponding indication vector, which includes: each participant generates a local random vector and encrypts it; each participant aggregates all local random vectors into a global random vector; each participant selects components in the global random vector according to the recorded indication vector of the bucket to aggregate to obtain the corresponding aggregation statistical result of global randomness and convert it into a secret sharing form; each participant stores the generated local random mask vector and the secret share of the global randomness aggregation statistical result for subsequent calculation.

[0023] As an optional implementation, in the process of pre-computation, scattering samples into different buckets according to their features and recording the corresponding indication vector, all random values generated in the pre-computation stage are not reused and are discarded after use and the subsequent values cover their positions.

[0024] And the process is executed in turn and uses linear parallel processing technology to accelerate the calculation.

[0025] As an alternative embodiment, each participant selects its random value and aggregates with other participants' encrypted random vectors, each participant calculates the statistical information of the encrypted random vector according to the channel indication vector, and the process of converting the ciphertext into a secret share form includes: each participant adds the generated local randomness mask to the share vector indicating different categories of labels held by each participant; each participant aggregates its calculation results, synchronizes and obtains the masked indication vector; each participant aggregates the elements of the corresponding positions of the masked indication vector according to the generated indication vector of different buckets for different features, and accumulates the statistical data of each bucket one by one to obtain the statistical data related to different split points under the corresponding feature; each participant again eliminates the mask in the calculated statistical data using the secret share of the global statistical data and obtains it in the form of a secret share.

[0026] As an alternative embodiment, each participant compares all the split gains to determine the corresponding indication vector, and the process of calculating the Gini impurity and split gain Gain of all possible splits includes: each participant uses a secure comparison algorithm and binary search to calculate the local best split gain of different features one by one in a certain order; each participant continues to obtain the local best split gain and the global best split gain in the same way;

[0027] During the comparison process, all possible splits will be indexed according to the participants and their features, and each participant needs to record the comparison results during the calculation of the local best split gain;

[0028] Each participant constructs an indication vector indicating the best split point from top to bottom using the corresponding comparison results, each participant encrypts the share of the indication vector, and the participant holding the split point aggregates the ciphertext and obtains the ciphertext of the indication vector indicating how to split the current node using the local privacy information retrieval technology, and each participant needs to jointly convert the ciphertext into a secret share form.

[0029] As an alternative embodiment, if the pre-pruning condition is met, each participant calculates that the current node is an internal node and updates the share vector of different labels, and if not, each participant calculates that the current node is a leaf node and determines the classification information of the leaf node. The process includes: each participant checks whether the pre-pruning condition is met, if it is met, the type of the current tree node is an internal node, otherwise it is a leaf node;

[0030] If the current node is an internal node, each participant updates the vector indicating different categories of the current node using a vector indicating how to split the current node, and further determines the types of the left and right child nodes of the current node through training; if the current node is a leaf node, each participant needs to update the corresponding information of the leaf node according to the training task.

[0031] A computer readable storage medium for storing computer instructions, which, when executed by a processor, complete the steps in the above method.

[0032] An electronic device comprising a memory and a processor, and computer instructions stored on the memory and running on the processor, which, when executed by the processor, complete the steps in the above method.

[0033] Compared with the prior art, the beneficial effects of the present application are:

[0034] The present application does not use homomorphic encryption technology when constructing the vector indicating different categories of samples in the current node, but adopts the form of linear secret sharing, which has the advantages of 1) higher computational efficiency than homomorphic encryption technology; 2) when updating the indicating vector, there is no need to simulate the multiplication between ciphertexts using the semi-homomorphic property of linear secret sharing and encryption scheme, there is no need to convert the ciphertext into a secret sharing form and then restore its ciphertext form, only the multiplication between secret shares is needed, which has higher communication efficiency.

[0035] The present application makes full use of the comparison results between different gain information, thereby avoiding the costly secret sharing-based equality test when constructing the vector required by the privacy information retrieval (PIR) technology, making the construction of the scheme more simple and efficient.

[0036] The present application makes full use of the invariants and consistency of operations in the decision tree training process, and packs multiple ciphertext information into a single ciphertext, thereby reducing the waste of plaintext space and improving the parallel ability of data processing in training.

[0037] Based on the above, the present application can further improve the speed of model training and communication overhead without compromising system security compared with existing schemes.

[0038] In order to make the above-mentioned purposes, features and advantages of the present application more obvious and easy to understand, the following preferred embodiments are described in detail below, and the accompanying drawings are used for explanation. BRIEF DESCRIPTION OF DRAWINGS

[0039] The drawings accompanying the specification of the present application form part of the present application and are used to provide a further understanding of the present application, the illustrative embodiments of the present application and the description thereof serve to explain the present application, and do not constitute an improper limitation of the present application.

[0040] Figure 1 is a system architecture diagram of the present application;

[0041] Figure 2 is a decision tree training flowchart of the present application;

[0042] Figure 3 is a communication flowchart between each participant of the present application;

[0043] Figure 4 is a ciphertext packing schematic diagram of the present application;

[0044] Figure 5 is a schematic diagram of how to use intermediate calculation results to construct an indication vector in the present application. DETAILED DESCRIPTION

[0045] The present application is further described below in conjunction with the accompanying drawings and embodiments.

[0046] It should be noted that the following detailed description is exemplary in nature and is intended to provide further description of the present application. Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which the present application belongs.

[0047] It should be noted that the terms used herein are only for the purpose of describing specific embodiments and are not intended to limit exemplary embodiments according to the present application. As used herein, the singular form is intended to include the plural form unless the context clearly indicates otherwise, and it should also be understood that when the terms "comprise" and / or "include" are used in the specification, there is a presence of a feature, step, operation, device, component, and / or combinations thereof.

[0048] The embodiments in the present application and the features in the embodiments can be combined with each other without conflict.

[0049] A privacy protection method based on decision tree model training, or a privacy protection decision tree model training method, as shown in Figure 1 , uses (t, m) threshold partial homomorphism Paillier encryption and secret sharing based secure multi-party computation as technical support, characterized in that 1) two types of participants are included: active participants P0 and passive participants P i∈{1...m-1} , wherein P0 holds part of the characteristics of the sample and the corresponding label information, P i only holds part of the characteristic information, and the characteristic information held by each participant is different from each other; multiple ciphertexts with the same operation are packed into a single ciphertext, as shown in Figure 4 ; the calculation of the required statistical data is divided into a pre-computation phase and a training phase. i∈{1...m-1} Under the coordination of P0, the necessary data information is exchanged to complete the training of the tree model.

[0050] As Figure 2 shown, the method comprises the following steps:

[0051] Step one, initialization phase. This phase is only performed once in the training process of the decision tree model. The active participants are involved in the passive participants to synchronize the model training related hyperparameters, such as the maximum depth, the maximum number of leaf nodes, the minimum number of node split samples, the minimum split gain, and the number of buckets (determining the maximum number of feature split points).

[0052] The active participants initialize the vector indicating the samples available for splitting in the current node and generate a plurality of indication vectors indicating different categories according to the sample labels, and secret share all the above indication vectors with the passive participants.

[0053] As Figure 3 shown, the passive participants need to receive and store the initialization parameters, and all participants need to sort the sample data according to each feature they hold, and then scatter them evenly into each bucket according to the maximum number of buckets and record the corresponding indication vector.

[0054] Step two, pre-computation phase. This phase is different from the general pre-computation technology, which has no relevance to data input.

[0055] The pre-computation idea used in the present application is only based on the invariance of data input in the training process, that is, the sample vector indicating each bucket does not change during the training process, in order to improve the parallel processing capability of the scheme.

[0056] In this phase, the participants cooperatively generate random data for the training phase, which includes two parts: one part is used as a random mask for the indication category vector in the training phase; the other part is used to convert the ciphertext into a secret sharing form. First, each participant generates a local random vector and encrypts it; then, each participant aggregates all local random vectors into a global random vector; finally, each participant selects the components in the global random vector according to the indication vector of the bucket recorded in the initialization phase to obtain the corresponding aggregation statistics of the global randomness and convert it into a secret sharing form. Each participant stores the generated local random mask vector and the secret share of the global randomness aggregation statistics for subsequent calculation.

[0057] Step 3: Local calculation phase. In this phase, each participant uses the local randomness mask generated in the preprocessing phase to add to the share vectors it holds that indicate different category labels; then, each participant aggregates its calculation results, synchronizes, and obtains the masked indicator vector. Each participant aggregates the elements of the corresponding positions of the masked indicator vector based on the indicator vectors for different buckets of different features generated in the initialization phase and accumulates the statistics of each bucket one by one to obtain the statistics related to different splitting points under the corresponding feature. Each participant again uses the secret share of the global statistics calculated in the precalculation phase to eliminate the mask in the calculated statistics and obtain the form of its secret share.

[0058] Step 4, secure multi-party computing stage. In this stage, each participant uses its existing secure multi-party computing technology to interactively implement the calculation of relevant mathematical formulas, such as the Gini impurity used by CART in classification tasks and the sample variance in regression tasks. In addition to calculating the Gini impurity / sample variance of the current node, each participant still needs to calculate the corresponding data on the left and right using different splitting points under different features and calculate the splitting gain generated by the splitting point. Each participant uses a secure comparison algorithm and a binary method to calculate the local optimal splitting gain of different features one by one in a certain order; then, each participant continues to obtain the local optimal splitting gain and the global optimal splitting gain in the same way. During the comparison process, all possible splits will be indexed according to the participants and their characteristics, and each participant needs to record the comparison results in the process of calculating the local optimal splitting gain. Each participant uses the corresponding comparison results from top to bottom as follows Figure 5 As shown in Figure 1, the indicator vector indicates the optimal splitting point. In this vector, all the digits except the digit where the optimal splitting point is located are 0. Each participant encrypts a share of the indicator vector. The participant holding the splitting point aggregates the ciphertext and uses private information retrieval (PIR) technology locally to obtain the corresponding ciphertext of the indicator vector for how to split the current node. Each participant must jointly convert this ciphertext into a secret share form.

[0059] Step 5: Model update phase. In this phase, each participant checks whether the pre-pruning conditions are met based on the initialization phase, such as whether the tree reaches the maximum tree depth and number of leaf nodes, the minimum number of splits, and the minimum split threshold, to determine the type of the current tree node, that is, whether it is an internal node or a leaf node. If the current node is an internal node, each participant needs to use the vector indicating how to split the current node to update the vector indicating different categories, that is, it is necessary to further determine the types of the left and right child nodes of the current node through training; if the current node is a leaf node, each participant needs to update the corresponding information of the leaf node based on the training task, such as the need to count the maximum number of label information in the classification task and the mean information of the sample label in the regression task.

[0060] The above-mentioned local computing phase, secure multi-party computing phase, and model updating phase need to be executed in sequence according to the training needs of the tree model.

[0061] The following describes each step in detail:

[0062] The step 1 specifically includes:

[0063] 1.1P0, as the active participant, selects appropriate hyperparameters based on the training task to perform pre-pruning to avoid overfitting of the model, such as the maximum tree depth MAX_DEPTH, the maximum number of tree nodes MAX_NODES_NUM, the minimum number of node split samples MIN_SPLIT_NUM, the minimum split threshold MIN_SPLIT_THESH, the maximum number of allowed buckets MAX_BINS_NUM, and security-related parameters such as the Paillier key size KEY_SIZF, the computational domain F, the corresponding modulus p, and the statistical parameter κ. P0 provides other passive participants P i∈{1...m-1} Synchronize the above hyperparameters and security parameters.

[0064] 1.2P i According to the set of characteristics it holds Each feature f in the sample X is sorted by value, and the unique value val in the feature is counted unique If |val unique |≤MAX_BINS_NUM, then according to val unique Each value in determines the set of indicator vectors belonging to the samples in each bucket Otherwise, certain buckets identified by unique values ​​will be merged according to the rules agreed upon by all parties.

[0065] 1.3P0 derives vectors β indicating different categories based on the label vector y of the samples it holds k ∶=

[0066] (β k [1],...,β k [N]) and where β k [j] = 1 if and only if X[j] belongs to category k; otherwise, β k [j] = 0. P0 versus β k Secret sharing is <β k >∶=(<β k [1]>,...,<β k [N]>), each participant P i Holding share <β k > i ∶=(<β k [1]> i ,...,<β k[N]> i ), where and k [j]> i ∈ R {0...2 k}。

[0067] 1.4P i According to KEY_SIZE, the key generation protocol of (m, m) threshold Paillier is executed in a distributed manner, and in the present application, the threshold value is set to m, that is, all participants are required to correctly decrypt, and when the protocol ends, each participant obtains the public key and its partial private key pair (pk, sk u ).

[0068] The second step is specifically as follows:

[0069] 2.1P u From {0...2 k+k+v -1} uniformly randomly generate l groups of mask vectors with size N, and the set composed of the mask vectors is denoted as {mask i,l} l ; then it is compressed into a single mask vector mask i , where j∈{0...N-1} and offset∶=k+2(k+v)+logN; finally, it is encrypted with the public key pk to obtain [[mask i ]].

[0070] 2.2P i From {0...2 k+2(k+v)+logN -1{ uniformly randomly generate B groups of random secret share vectors with size l, and the set composed of the random share vectors is denoted as {share_r i,B} B , where Then, it is compressed into a single share vector share_r i , where j∈{0...B-1} and offset∶=k+2(κ+v)+logN; finally, it is encrypted with the public key pk to obtain

[0071] 2.3P i∈{1...m-1} Send and to P0; P0 calculates and P0 synchronizes and to P i∈{1...m-1} .

[0072] 2.4P i According to the indication vector of each bucket of different features in INV i , the statistical data in each bucket is calculated in a "selection-sum" manner Where b∈{0...MAX_BINS_NUM-1}.

[0073] 2.5P i Using the prefix and statistical information under different split points belonging to different features Where

[0074] 2.6P i From the re-sharing share vector Select elements to form a TEMPi matrix with MAX_BINS_NUM rows and MAX_BINS_NUM columns and calculate P i First send to P j≠i ; P j≠i uses its private key sk j to partially decrypt and returns the partially decrypted result to P i ; P i aggregates all partial decryption results from P j≠i to obtain i i and unpacks it into i,l i} l , and P j≠i only needs to set j,l j} l to the corresponding random value in the re-secret sharing vector set generated in 2.2 to complete the conversion from ciphertext to share.

[0075] The above steps 2.1-2.6 are executed in turn according to the training needs of the data model and the process can be further calculated by using linear parallel processing technology to speed up. In addition, all random values generated in the pre-computation stage are not reused and are discarded after use and are replaced by subsequent values in their positions, that is, each selection is selected from the beginning in turn. In order not to affect the understanding, the redundant subscripts are not given.

[0076] The third step is as follows:

[0077] 3.1P​​​i from {mask i,l} l k groups are chosen and <λ k > i : = <β k > i + mask i,k and <λ k > i is sent to P0. P0 aggregates all the shares to obtain λ k and synchronizes it to all other participants P i∈{1...m-1} .

[0078] 3.2 P i computes the statistical data AGG i in each bucket in a "select-sum" manner according to the indicator vector of each bucket in INV i,left , where b e {0...MAX_BINS_NUM-1}.

[0079] 3.3 P i computes the frequency statistical information FREQ i,left,k of the samples in each class in the left child node under different features and different split points using AGG i,left , where P i from {<SUM i,l > i} l k groups are chosen and combined into the corresponding matrix form <SUM i > i and <FREQ i,left > i : = FREQ i,left + <SUM i > i The corresponding frequency statistical information of the samples in each class in the right child node can be indirectly calculated as i.e. the frequency of each class label minus the frequency of the corresponding class in the left child node.

[0080] 3.4 P i computes the statistical data AGG <num>i i Set the number of samples available for splitting in the left child node of the current node to N. P i The frequency information of samples under different split points of different features can be used to calculate the number of samples available for the left child node The number of samples available for the right child node can be calculated as <NUM i,right > i ∶= <num> i -<NUM i,left > i .

[0081] 3.5P i According to where p l is the sampling probability of the sample label l, i.e. Pr(y=l), respectively, the <gini>and the impurity information of the left <Gini i,left > and right child nodes <Gini i,right > after splitting and the corresponding p l values should be replaced by p i,left,l , p i,right,l , respectively. P i ossible values of the attribute are calculated using the statistics computed in 3.3 and 3.4 <num> i <NUM i,left > i <NUM i,right > i <FREQ i,l,eft>i <FREQ i,right > i According to the relationship between probability and frequency Calculate the above required probability value.

[0082] 3.6P i According to Gain := w left · Gini left + w right · Gini right - Gini, where

[0083] and use the impurity information calculated in 3.5 <gini>< Gini i,left > and < Gini i,right > respectively calculate the gain information < Gain i > i .

[0084] 3.7 Each participant P i first indexes its gain information in natural order for identifying the user and feature it belongs to; then, it compares its gain information using binary search to obtain its locally best split gain < Gain i,best > i with respect to each feature; then, it continues to compare the locally best split gain using binary search to obtain the globally best split gain < Gain best > i Specifically, in the binary comparison, each participant P u compares two adjacent gain values one by one and utilizes max( <gain1> i , <gain2> i )∶＝ <gain1> i + <flag> i ·( <gain1> i - <gain1> i ) the comparison information is updated <flag> i * i * i <flag> i : = 1 <gain1> i≥ <gain2> i < / gain2> < / gain1> and <P * > i , <f * > i The user and feature for recording the maximum value are recorded. Each participant P i The above is executed in a loop until no more gain information is comparable, i.e. at this time the maximum gain related information has been obtained. Finally, P ∈{1...m-1} sends its share <P * > i and <f * > i to P0; P0 reconstructs P * and <f * and sends them to the other participants P ∈{1...m-1} .

[0085] The step four is specified as follows:

[0086] 4.1 P i retrieves from its recorded comparison process the features <P * and <f * which are located at different comparison levels in the comparison process <flag> i a current node is updated layer by layer from the root node top-down <flag> i ∶ <flag> i • <flag parent > i ; at this time, among all the leaf nodes, there is and only one leaf node whose <flag> i ∶= 1 and the rest of the leaf nodes are 0, each participant P i constructing a tree of leaf nodes <flag> i Composed vector <inv * > i The only record of the best split point index position in the vector. i Use the public key pk <inv * > i Encrypted to and send it to P * ;P * Will Aggregate into

[0087] 4.2P * Construct the matrix TEMP locally, whose row vectors are determined by the characteristics f * The indicator vectors derived from different splitting points in are formed; then, P * Calculation using PIR technology The multiplication between the vector and the TEMP matrix finally obtains the required indicator vector P * First The elements are evenly divided into Then, each group of ciphertexts is shifted by its plaintext value (k+κ+v)*(lz-1), and each group of ciphertexts is multiplied to obtain a vector of size N′ That is to say Where z represents the position of the ciphertext in its group.

[0088] 4.3P i From {0,...,2 (k+κ+v) }Uniformly randomly generate a random share vector of size N <r> i and packs the vector into random shares of size N' in the same way as in 4.2 to get the vector <r' i and encrypts it using the public key pk to get and sends it to P * . P * computes and broadcasts it to P i ; P i partially decrypts i using its private key sk and sends the result to P * ; P * aggregates the partially decrypted results from the other participants P i to get u' and deconstructs it into a vector u of size N. P * * set to u- <r> * while the other participants P i∈{0,...,m-1}\* will be i is configured to <r> i The conversion from the ciphertext to the secret share is completed.

[0089] The step five is specifically as follows:

[0090] 5.1 P0 checks whether the current training depth DEPTH and the number of nodes NODES_NUM are less than the preset values MAX_DEPTH and MAX_NODES_NUM respectively. i Check the current number of nodes <num> i and maximum split gain <Gain best > i whether they are less than their preset values MIN_SPLIT_NUM and MIN_SPLIT_THESH, and the checking results are sent to P0 for reconstruction, from P0 to other participants P i∈{1,...,m-1} broadcast the results.

[0091] 5.2 If all the condition checks in 5.1 pass, mark the current node as an internal node, and execute this step. i update the indicator vector of different labels in the left child node to <β left,k > i : = <β k > i ⊙ i ; accordingly, P i The indicator vectors indicating different labels in the right child node can be updated to <β right,k > i ∶=<β k > i -<β left,k > i .

[0092] 5.3 If the condition check in 5.1 fails, mark the current node as a leaf node and execute this step. i Calculate the frequency of different categories in the current node as Select the label with the maximum frequency by comparison <k> i Classification information of the current leaf node.

[0093] Those skilled in the art will appreciate that embodiments of the application can be devised for a method, a system, or a computer program product. Accordingly, the present application can take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Furthermore, the present application can take the form of a computer program product on one or more computer readable storage media (including, but not limited to, disk storage, CD-ROMs, optical storage devices, etc.) embodying computer readable program code.

[0094] The present application is described in reference to the flowchart illustrations and / or block diagrams of methods, apparatus (systems) and computer program products according to embodiments of the application. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general purpose computer, special purpose computer, embedded processing system or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in the flowchart illustrations and / or block diagrams. Figure 1 one or more functions specified in the flowchart illustrations and / or block diagrams. Figure 1 means for carrying out each of the one or more functions specified in the flowchart illustrations and / or block diagrams.

[0095] These computer program instructions can also be stored in a computer readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer readable memory produce an article of manufacture including instructions which implement the functions specified in the flowchart illustrations and / or block diagrams. Figure 1 one or more functions specified in the flowchart illustrations and / or block diagrams. Figure 1 means for carrying out each of the one or more functions specified in the flowchart illustrations and / or block diagrams.

[0096] These computer program instructions can also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart illustrations and / or block diagrams. Figure 1 one or more functions specified in the flowchart illustrations and / or block diagrams. Figure 1 means for carrying out each of the one or more functions specified in the flowchart illustrations and / or block diagrams.

[0097] The application described herein is to be considered merely as illustrative of the principles of the application and is not to be considered as limiting thereof. Various modifications and changes can be made thereto by those skilled in the art which will embody the principles of the application and fall within the spirit and scope thereof, without departing from the spirit and scope of the application.< / k> < / num> < / r> < / r> ​< / r> < / flag> < / flag> < / flag> < / flag> < / flag> < / flag> ​​​​​< / flag> < / gain1> < / flag> < / gain1> < / gain2> < / gain1> < / gini> < / num> < / gini> < / num> < / num>

Claims

1. A privacy protection method based on decision tree model training, characterized by: The following steps are involved: The active participant synchronizes the decision tree model training related hyperparameters and security parameters with the passive participant, generates multiple indicator vectors indicating different categories of samples, and secretly shares all of the above indicator vectors with the passive participant; Each participant performs pre-calculation, scatters samples into different buckets according to their characteristics and records the corresponding indicator vectors; Each participant selects its random value and aggregates it with the random vectors encrypted by other participants. Each participant calculates the statistics of the ciphertext random vector based on the channel indicator vector and converts the ciphertext into a secret sharing form. Each participant randomizes the indicator vectors of different labels and performs secure multi-party computation. Each participant compares all split gains, determines the corresponding indicator vectors, and calculates the Gini impurity and split gain of all possible split situations. Determine whether the pre-pruning conditions are met. If so, each participant calculates the current node as an internal node and updates the share vectors of different labels. If not, each participant calculates the current node as a leaf node and determines the classification information of the leaf node. Determine whether there are still splittable internal nodes. If so, return to iterative execution of the steps of randomizing the indicator vectors of different labels of each participant and performing secure multi-party computation. Otherwise, end the above process.

2. A privacy protection method based on decision tree model training as claimed in claim 1, characterized in that: The decision tree model training related hyperparameters include maximum depth, maximum number of leaf nodes in the tree, minimum number of node split samples, minimum split threshold and maximum number of buckets allowed; The security parameters include Paillier key size, calculation domain and corresponding modulus and statistical parameters.

3. The privacy protection method based on decision tree model training according to claim 1, characterized in that: The process of generating multiple indicator vectors indicating different categories of samples and secretly sharing all of the above indicator vectors with the passive participants includes: The active participant initializes the vector indicating the current node that can be used to split the sample and generates multiple indicator vectors indicating its different categories based on the sample label, and secretly shares all the above indicator vectors with the passive participant; the passive participant needs to receive and store the initialization parameters, and all participants need to sort the sample data according to each feature they hold, and then evenly scatter it into each bucket according to the maximum number of buckets of the parameter and record the corresponding indicator vector.

4. The privacy protection method based on decision tree model training according to claim 1, characterized in that: Each participant performs pre-calculation, scatters samples into different buckets according to their characteristics, and records the corresponding indicator vectors. The process includes: each participant generates a local random vector and encrypts it; each participant aggregates all local random vectors into a global random vector; each participant selects components in the global random vector based on the recorded bucket indicator vector to aggregate to obtain its corresponding aggregated statistical result for global randomness and converts it into a secret sharing form; each participant stores the generated local random mask vector and the secret share of the global randomness aggregated statistical result for subsequent calculations.

5. The privacy protection method based on decision tree model training according to claim 1, characterized in that: Each participant performs pre-calculation, scatters samples into different buckets according to their characteristics, and records the corresponding indicator vectors. All random values ​​generated in the pre-calculation phase are not reused and are discarded after use and replaced by subsequent values. The process is executed in a loop and linear parallel processing technology is used to accelerate the calculation.

6. The privacy protection method based on decision tree model training according to claim 1, characterized in that: Each participant selects its random value and aggregates it with the random vectors encrypted by other participants. Each participant calculates the statistical information of the ciphertext random vector based on the channel indicator vector. The process of converting the ciphertext into a secret sharing form includes: each participant uses the generated local randomness mask to add it to the share vector indicating different category labels held by it; each participant aggregates its calculation results, synchronizes and obtains the masked indicator vector; each participant aggregates the elements of the corresponding position of the masked indicator vector based on the indicator vectors of different buckets generated for different features and accumulates the statistical data of each bucket one by one to obtain the statistical data related to different splitting points under the corresponding feature; each participant again uses the secret share of the calculated global statistical data to eliminate the mask in the calculated statistical data and obtain the form of its secret share.

7. The privacy protection method based on decision tree model training according to claim 1, characterized in that: Each participant compares all split gains, determines the corresponding indicator vector, and calculates the Gini impurity and split gain of all possible split situations. The process includes: each participant uses the security comparison algorithm and the dichotomy method to calculate the local optimal split gain of different features one by one in a certain order; each participant continues to obtain the local optimal split gain and the global optimal split gain in the same way; During the comparison process, all possible splits will be indexed according to the participants and their characteristics, and each participant needs to record the comparison results during the calculation of the local best split gain; Each participant uses the corresponding comparison results to construct an indicator vector indicating the optimal splitting point from top to bottom. Each participant encrypts a share of the indicator vector. The participant holding the splitting point aggregates the ciphertext and uses private information retrieval technology locally to obtain the corresponding ciphertext of the indicator vector for how to split the current node. Each participant must jointly convert the ciphertext into a secret share form.

8. The privacy protection method based on decision tree model training according to claim 1, characterized in that: Determine whether the pre-pruning condition is met. If so, each participant calculates the current node as an internal node and updates the share vectors of different labels. If not, each participant calculates the current node as a leaf node and determines the classification information of the leaf node. The process includes: each participant checks whether the pre-pruning condition is met. If so, the type of the current tree node is an internal node, otherwise it is a leaf node; If the current node is an internal node, each participant uses the vector indicating how to split the current node to update the vector indicating different categories, and further determines the types of the left and right child nodes of the current node through training; if the current node is a leaf node, each participant needs to update the corresponding information of the leaf node according to the training task.

9. A computer-readable storage medium, characterized in that: Used to store computer instructions, which, when executed by a processor, complete the steps of the method according to any one of claims 1 to 8.

10. An electronic device, characterized in that: The method comprises a memory and a processor, and computer instructions stored in the memory and executed on the processor, wherein when the computer instructions are executed by the processor, the steps of the method according to any one of claims 1 to 8 are completed.

Images