A decision tree ensemble training method and device based on differential privacy

Abstract

The present application relates to a kind of decision tree ensemble training method and device based on differential privacy, the method and device of the present application are based on exponential mechanism to the feature needing privacy protection in training data is protected processing, based on laplace mechanism to the leaf node in local decision tree model after training is protected processing, avoid the problem of privacy data leakage caused by model reverse deduction.

Description

Technical Field

[0001] This invention relates to the field of materials data security technology, and in particular to a decision tree ensemble training method and apparatus based on differential privacy. Background Technology

[0002] The development of internet technology has led to an exponential increase in data, greatly facilitating machine learning. However, apart from companies with abundant data, most institutions lack sufficient data to serve their academic research or commercial projects. Furthermore, private datasets from different institutions often contain sensitive and valuable information, making sharing difficult and creating a serious "data silo" problem. Therefore, federated learning, with its secure joint training involving multiple participants, is an effective solution. While this eliminates the need for data transmission to a server, addressing data security, existing privacy protection methods combined with machine learning rely on each participant training independently locally and then transmitting the training parameters to a server for aggregation to obtain the final model. This means that the parameters used to train a particular model can be reasonably derived from the trained model of a single participant, potentially leading to the leakage of private data. Summary of the Invention

[0003] In view of this, the present invention provides a decision tree ensemble training method and apparatus based on differential privacy to avoid leakage of privacy data caused by model back-derivation.

[0004] To achieve the above objectives, the present invention provides the following solution:

[0005] A decision tree ensemble training method based on differential privacy is disclosed. The method is applied to a distributed ensemble system, which includes a server and multiple clients connected to the server. Each client possesses a different privacy data set, which is used for training the decision tree model. The privacy data in the privacy data set consists of material processing parameters and performance parameters. The method includes the following steps:

[0006] Obtain the privacy budget of the client sent by the server and the global decision tree model obtained after training on another client;

[0007] Select privacy-preserving training data from the local training dataset to construct an important feature set; the local training dataset is a subset of the privacy data set.

[0008] Based on the client's privacy budget, an exponential mechanism is used to protect the privacy-required features in the training data, resulting in processed training data.

[0009] The global decision tree model is trained using the processed training data to obtain the trained local decision tree model.

[0010] Based on the client's privacy budget, the leaf nodes in the trained local decision tree model are protected using the Laplace mechanism to obtain the processed local decision tree model, which is then sent to the server.

[0011] Optionally, the local training dataset is obtained in the following manner:

[0012] Multiple pieces of privacy data are obtained from the privacy data set using random sampling.

[0013] A local training data set is constructed by using multiple pieces of the aforementioned privacy data as training data.

[0014] Optionally, the step of selecting privacy-preserving training data from the local training dataset and constructing an important feature set specifically includes:

[0015] The first feature set is constructed using the following formula;

[0016] feature∈IP,if|V f |≤n·ζ;

[0017] Where feature is the feature in the training data, IP is the first feature set, and V f ζ represents the set of feature values ​​of the feature in all training data; n is the number of training data in the local training data set; ζ is the incomplete privacy index of the training data.

[0018] Construct a second feature set based on expert experience;

[0019] The union of the first feature set and the second feature set is obtained as the important feature set.

[0020] Optionally, the step of using an exponential mechanism to protect privacy-required features in the training data based on the client's privacy budget to obtain processed training data specifically includes:

[0021] Based on the client's privacy budget, calculate the privacy budget of the internal node using the following formula;

[0022]

[0023] Where, ε nonleaf MaxDepth represents the privacy budget for internal nodes, and it represents the maximum depth value for training the decision tree model. Let m be the privacy budget for the m-th client in the k-th training epoch;

[0024] Based on the privacy budget of the internal nodes, the probability index value of the feature requiring privacy protection is calculated using the following formula as the processed feature value of the feature requiring privacy protection.

[0025]

[0026] Among them, P bvalue G represents the probability exponent value of the feature requiring privacy protection, bvalue represents the feature value of the feature requiring privacy protection, Δ represents the increment of the feature value of the feature requiring privacy protection, and G represents the probability exponent value of the feature requiring privacy protection. bvalue Gains for features that require privacy protection.

[0027] Optionally, based on the client's privacy budget, the step of using the Laplace mechanism to protect the leaf nodes in the trained local decision tree model to obtain a processed local decision tree model specifically includes:

[0028] Based on the client's privacy budget, calculate the privacy budget of the leaf node using the following formula;

[0029]

[0030] Where, ε leaf Privacy budget for leaf nodes, Let m be the privacy budget for the m-th client in the k-th training epoch;

[0031] Based on the privacy budget of the leaf nodes, the Laplace mechanism is used to protect the leaf nodes in the trained local decision tree model using the following formula.

[0032]

[0033] Among them, w' j I represents the weight of the j-th leaf node after processing. j Let g be the set of processed training data used to train the j-th leaf node. i and h i Let ε be the first and second gradient statistics of the loss function value corresponding to the i-th processed training data. leaf Let λ be the privacy budget for the leaf nodes, and λ be a known parameter.

[0034] A decision tree ensemble training device based on differential privacy is applied to a distributed ensemble system. The distributed cluster system includes a server and multiple clients connected to the server. Each client possesses a different privacy data set, which is used for training the decision tree model. The privacy data in the privacy data set consists of material processing parameter data and performance parameter data. The device includes:

[0035] The data acquisition module is used to acquire the privacy budget of the client sent by the server and the global decision tree model obtained after training by another client;

[0036] A feature selection module requiring privacy protection is used to select training data requiring privacy protection from a local training dataset and construct an important feature set; the local training dataset is a subset of the privacy data set;

[0037] The feature protection processing module is used to protect the privacy-required features in the training data using an exponential mechanism based on the client's privacy budget, thereby obtaining the processed training data.

[0038] The training module is used to train the global decision tree model using the processed training data to obtain a trained local decision tree model. Based on the client's privacy budget, the Laplace mechanism is used to protect the leaf nodes in the trained local decision tree model to obtain a processed local decision tree model, and the processed local decision tree model is sent to the server.

[0039] Optionally, the local training dataset is obtained in the following manner:

[0040] Multiple pieces of privacy data are obtained from the privacy data set using random sampling.

[0041] A local training data set is constructed by using multiple pieces of the aforementioned privacy data as training data.

[0042] Optionally, the feature selection module requiring privacy protection specifically includes:

[0043] The first feature set construction submodule is used to construct the first feature set using the following formula;

[0044] feature∈IP,if|V f |≤n·ζ;

[0045] Where feature is the feature in the training data, IP is the first feature set, and V f Let n be the set of feature values ​​of the feature in all training data; ζ is the incomplete privacy index of the training data.

[0046] The second feature set construction submodule is used to construct the second feature set based on expert experience;

[0047] The important feature set acquisition submodule is used to obtain the union of the first feature set and the second feature set as the important feature set.

[0048] Optionally, the feature protection processing module specifically includes:

[0049] The privacy budget calculation submodule for internal nodes is used to calculate the privacy budget of internal nodes based on the privacy budget of the client using the following formula;

[0050]

[0051] Where, ε nonleaf MaxDepth represents the privacy budget for internal nodes, and it represents the maximum depth value for training the decision tree model. Let m be the privacy budget for the m-th client in the k-th training epoch;

[0052] The feature protection processing submodule is used to calculate the probability index value of the feature that needs privacy protection based on the privacy budget of the internal node, using the following formula as the processed feature value of the feature that needs privacy protection.

[0053]

[0054] Among them, P bvalue G represents the probability exponent value of the feature requiring privacy protection, bvalue represents the feature value of the feature requiring privacy protection, Δ represents the increment of the feature value of the feature requiring privacy protection, and G represents the probability exponent value of the feature requiring privacy protection. bvalue Gains for features that require privacy protection.

[0055] Optionally, the training module specifically includes:

[0056] The privacy budget calculation submodule for leaf nodes is used to calculate the privacy budget of leaf nodes based on the client's privacy budget using the following formula;

[0057]

[0058] Where, ε leaf Privacy budget for leaf nodes, Let m be the privacy budget for the m-th client in the k-th training epoch;

[0059] The leaf node protection submodule is used to protect the leaf nodes in the trained local decision tree model according to the privacy budget of the leaf nodes and using the Laplace mechanism, using the following formula.

[0060]

[0061] Among them, w' j I represents the weight of the j-th leaf node after processing. j Let g be the set of processed training data used to train the j-th leaf node. i and h i Let ε be the first and second gradient statistics of the loss function value corresponding to the i-th processed training data. leaf Let λ be the privacy budget for the leaf nodes, and λ be a known parameter.

[0062] According to specific embodiments provided by the present invention, the present invention discloses the following technical effects:

[0063] This invention discloses a decision tree ensemble training method and apparatus based on differential privacy. The method includes the following steps: obtaining the privacy budget of a client sent by a server and a global decision tree model obtained after training by another client; selecting training data requiring privacy protection from a local training dataset to construct an important feature set; according to the client's privacy budget, using an exponential mechanism to protect the privacy-requiring features in the training data to obtain processed training data; training the global decision tree model using the processed training data to obtain a trained local decision tree model; according to the client's privacy budget, using a Laplace mechanism to protect the leaf nodes in the trained local decision tree model to obtain a processed local decision tree model, and sending the processed local decision tree model to the server. This invention uses an exponential mechanism to protect the privacy-requiring features in the training data and a Laplace mechanism to protect the leaf nodes in the trained local decision tree model, avoiding the problem of privacy data leakage caused by model back-deriving.

[0064] Moreover, by applying privacy budgeting, this invention enables planned privacy protection, avoiding the waste of time and costs caused by blindly increasing the level of protection. Attached Figure Description

[0065] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0066] Figure 1 A flowchart illustrating a decision tree ensemble training method based on differential privacy, provided in an embodiment of the present invention;

[0067] Figure 2 A schematic diagram illustrating a decision tree ensemble training method based on differential privacy, provided in an embodiment of the present invention.

[0068] Figure 3 A flowchart of the distributed integration system provided in this embodiment of the invention;

[0069] Figure 4 The response principle diagram of the distributed integration system provided in the embodiment of the present invention. Detailed Implementation

[0070] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0071] The purpose of this invention is to provide a decision tree ensemble training method and apparatus based on differential privacy, so as to avoid the leakage of privacy data caused by model back-derivation.

[0072] To make the above-mentioned objects, features and advantages of the present invention more apparent and understandable, the present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0073] Example 1

[0074] Embodiment 1 of this invention provides a decision tree ensemble training method based on differential privacy, such as... Figure 1 As shown, the method includes the following steps:

[0075] Step 101: Obtain the privacy budget of the client sent by the server and the global decision tree model obtained after the other client has been trained;

[0076] Step 102: Select training data that requires privacy protection from the local training data set and construct an important feature set; the local training data set is a subset of the privacy data set.

[0077] like Figure 2 As shown, step 102 specifically involves:

[0078] The SFS metric is used to obtain the feature values ​​for dividing the left and right subtrees. The establishment of the SFS metric consists of three steps. The first is data incomplete privacy. Embodiment 1 of this invention proposes ζ to characterize data incomplete privacy. When the number of feature values ​​is less than ζ of the total number of instances, the feature has data incomplete privacy characteristics, as shown below.

[0079]

[0080] Among them, V f S is the set of feature values ​​of a feature. f It is a feature set (first feature set) established based on the principle of incomplete data privacy.

[0081] The second is the importance of expert knowledge. In many professional fields such as materials and medicine, a set of features E representing expert knowledge can be obtained based on experts' years of experience. f (Second feature set).

[0082] Third, when selecting splitting feature values ​​for nodes within the decision tree, the SFS feature set that needs to be protected by the differential privacy index mechanism is the union of features related to both data incomplete privacy and the importance of expert knowledge, as shown below:

[0083] SFS = IP∪EK.

[0084] Assume the calculated gain G bvalue The largest bvalue's features belong to the SFS feature set (important feature set). This means that the bvalue requires privacy protection due to the confidentiality of data related to materials or medicine.

[0085] The formula for calculating the gain is as follows:

[0086]

[0087]

[0088]

[0089] These are the first and second gradient statistics of the loss function, respectively.

[0090] Step 103: Based on the client's privacy budget, use an exponential mechanism to protect the privacy-required features in the training data to obtain the processed training data.

[0091] like Figure 2 As shown, step 103 specifically involves: applying an exponential mechanism to select the dividing feature value, that is, selecting a random probability. The left and right subtrees are divided based on a specific value. Conversely, instead of using an exponential mechanism, the feature value with higher gain has a higher probability of being used as the splitting value; that is, the current node is split using bvalue.

[0092] Step 104: Based on the client's privacy budget, use the Laplace mechanism to protect the leaf nodes in the trained local decision tree model to obtain the processed local decision tree model, and then send the processed local decision tree model to the server.

[0093] like Figure 2 As shown, step 105 specifically involves: for leaf nodes, applying a Laplace mechanism with saliency weights for privacy protection. When the absolute value of a leaf node's weight is greater than its saliency weight, injecting appropriate noise into the leaf node's value, using... The weights of the leaf nodes are updated to protect their privacy. Considering the distributed nature of the model, each client can obtain a saliency weight α based on its own privacy dataset. i (i = 1, ..., N).

[0094] Regarding the allocation of the privacy budget, we allocated half of the privacy budget to the leaf nodes (i.e. The remaining half of the privacy budget is evenly distributed among each layer of the internal nodes, and each level of the decision tree receives ε. nonleaf ,Right now Furthermore, considering the diversity of datasets in real-world professional fields, this invention performs a floor operation on continuous feature values.

[0095] Step 105: Use the processed training data to train the global decision tree model to obtain the trained local decision tree model.

[0096] When the method of the present invention is applied to a distributed integration system, the distributed integration system of the present invention utilizes a distributed two-level lifting framework to enable the server to effectively allocate privacy budgets within and between iterations, and organize the client's local decision trees based on differential privacy to form a decision tree integration model based on differential privacy (i.e., a global decision tree model).

[0097] like Figure 3 As shown, the working process of a distributed integration system is as follows:

[0098] At the beginning of each iteration, the aggregation server first sends a "start training" command to all data owners, defining the training order based on the order of responses, such as... Figure 4 As shown. Next, the server sends the client's privacy budget. This is given to the first client. The first client randomly samples from its own privacy dataset. Combined with the received privacy budget Train a differential privacy-based local decision tree model and local decision tree model The parameters are transmitted to the server. The server updates the global decision tree model for the k-th round. Then, the server sends the privacy budget. The second client receives the privacy budget sent by the server. And a global decision tree model, and then based on its own privacy dataset Calculate the residuals and train a local decision tree model based on differential privacy using the residual results. Will The data is transmitted to the server. The server updates the global decision tree model for the k-th round. This process continues sequentially through training until all clients have been traversed. For each data owner, this training process can be described as follows:

[0099]

[0100] Among them, w j R represents the weight value of the leaf node. j It is the representation of a leaf node, and the I function is a logical function. When w j R j If the indices are the same, I = 1; otherwise, I = 0, L f It is the set of leaf nodes of the new tree obtained by the participants during training, i.e., L f ={R j} f During a single training iteration, the privacy budget allocated to each client is distributed using a differential privacy parallel combination approach, meaning each client is allocated a privacy budget of ε. m =ε. Because the privacy data of each data owner has no overlap, it meets the definition of differential privacy parallel composition.

[0101] Furthermore, considering the time efficiency of decision tree model training, multiple iterations in the same manner are performed in parallel until the number of iterations reaches a predefined threshold, at which point training terminates. Due to the relevance of the datasets used, the privacy budget allocation for each client between iterations follows a differential privacy ordering combination, i.e. Where K is a predefined number of iterations. Ultimately, a global decision tree model based on differential privacy can be obtained, achieving differential privacy for all data owners.

[0102] Example 2

[0103] Embodiment 2 of the present invention provides a decision tree ensemble training device based on differential privacy. The device is applied to a distributed ensemble system, the distributed cluster system including a server and multiple clients connected to the server. Each client possesses a different privacy data set, which is used for training the decision tree model. The privacy data in the privacy data set includes material processing parameter data and performance parameter data. The device includes:

[0104] The data acquisition module is used to acquire the privacy budget of the client sent by the server and the global decision tree model obtained after training by another client.

[0105] A feature selection module requiring privacy protection is used to select training data requiring privacy protection from a local training dataset and construct an important feature set; the local training dataset is a subset of the privacy data set.

[0106] The local training data set is obtained in the following manner: multiple private data are obtained from the private data set by random sampling; the multiple private data are used as training data to construct the local training data set.

[0107] The feature selection module requiring privacy protection specifically includes:

[0108] The first feature set construction submodule is used to construct the first feature set using the following formula.

[0109] feature∈IP,if|V f |≤n·ζ;

[0110] Where feature is the feature in the training data, IP is the first feature set, and V f ζ represents the set of feature values ​​of all features in the training data; n represents the number of training data in the local training data set; and ζ represents the incomplete privacy index of the training data.

[0111] The second feature set construction submodule is used to construct a second feature set based on expert experience.

[0112] The important feature set acquisition submodule is used to obtain the union of the first feature set and the second feature set as the important feature set.

[0113] The feature protection processing module is used to protect the privacy-required features in the training data using an exponential mechanism based on the client's privacy budget, thereby obtaining the processed training data.

[0114] The feature protection processing module specifically includes:

[0115] The privacy budget calculation submodule for internal nodes is used to calculate the privacy budget of internal nodes based on the privacy budget of the client using the following formula.

[0116]

[0117] Where, ε nonleaf MaxDepth represents the privacy budget for internal nodes, and it represents the maximum depth value for training the decision tree model. Let m be the privacy budget for the m-th client in the k-th training cycle.

[0118] The feature protection processing submodule is used to calculate the probability index value of the feature that needs privacy protection based on the privacy budget of the internal node, using the following formula as the processed feature value of the feature that needs privacy protection.

[0119]

[0120] Among them, P bvalue G represents the probability exponent value of the feature requiring privacy protection, bvalue represents the feature value of the feature requiring privacy protection, Δ represents the increment of the feature value of the feature requiring privacy protection, and G represents the probability exponent value of the feature requiring privacy protection. bvalue Gains for features that require privacy protection.

[0121] The training module is used to train the global decision tree model using the processed training data to obtain a trained local decision tree model. Based on the client's privacy budget, the Laplace mechanism is used to protect the leaf nodes in the trained local decision tree model to obtain a processed local decision tree model, and the processed local decision tree model is sent to the server.

[0122] The training module specifically includes:

[0123] The privacy budget calculation submodule for leaf nodes is used to calculate the privacy budget of leaf nodes based on the client's privacy budget using the following formula.

[0124]

[0125] Where, ε leaf Privacy budget for leaf nodes, Let m be the privacy budget for the m-th client in the k-th training cycle.

[0126] The leaf node protection submodule is used to protect the leaf nodes in the trained local decision tree model according to the privacy budget of the leaf nodes and using the Laplace mechanism, using the following formula.

[0127]

[0128] Among them, w' j I represents the weight of the j-th leaf node after processing. j Let g be the set of processed training data used to train the j-th leaf node. i and h i Let ε be the first and second gradient statistics of the loss function value corresponding to the i-th processed training data. leaf Let λ be the privacy budget for the leaf nodes, and λ be a known parameter.

[0129] This invention discloses a differential privacy-based decision tree ensemble training method and apparatus. It employs a distributed two-level boosting framework to effectively allocate privacy budgets between decision trees within and between iterations, and organizes local decision tree models based on differential privacy into a global decision tree model. In establishing the local decision tree model based on differential privacy, this invention proposes the SFS (Significant Feature Set) metric to select the set of important features and uses an exponential mechanism to protect these important features. It also uses saliency weights to obtain the importance threshold of leaf nodes, thus assisting the Laplace mechanism in protecting leaf nodes. This invention not only ensures the secure transmission of model parameters between the client and server but also significantly improves model accuracy, achieving model accuracy comparable to traditional decision tree ensemble learning models that use data from various parties.

[0130] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on its differences from other embodiments. Similar or identical parts between embodiments can be referred to interchangeably. For the apparatus disclosed in the embodiments, since they correspond to the methods disclosed in the embodiments, the description is relatively simple; relevant parts can be referred to the method section.

[0131] This document uses specific examples to illustrate the principles and implementation methods of the present invention. The descriptions of the above embodiments are only for the purpose of helping to understand the method and core ideas of the present invention. Furthermore, those skilled in the art will recognize that, based on the ideas of the present invention, there will be changes in the specific implementation methods and application scope. Therefore, the content of this specification should not be construed as a limitation of the present invention.

Claims

1. A decision tree ensemble training method based on differential privacy, characterized in that, The method is applied to a distributed integration system, which includes a server and multiple clients connected to the server. Each client has a different set of privacy data, which is used for training the decision tree model. The privacy data in the set consists of material processing parameters and performance parameters. The method includes the following steps: Obtain the privacy budget of the client sent by the server and the global decision tree model obtained after training on another client; Select privacy-preserving training data from the local training dataset and construct an important feature set; The local training data set is a subset of the privacy data set; Based on the client's privacy budget, an exponential mechanism is used to protect the privacy-required features in the training data, resulting in processed training data. The global decision tree model is trained using the processed training data to obtain the trained local decision tree model. Based on the client's privacy budget, the Laplace mechanism is used to protect the leaf nodes in the trained local decision tree model, resulting in a processed local decision tree model, which is then sent to the server. The process involves using an exponential mechanism to protect privacy-required features in the training data based on the client's privacy budget, resulting in processed training data. Specifically, this includes: Based on the client's privacy budget, calculate the privacy budget of the internal node using the following formula; ； in, For the privacy budget of internal nodes, This represents the maximum depth value for training the decision tree model. Let m be the privacy budget for the m-th client in the k-th training epoch; Based on the privacy budget of the internal nodes, the probability index value of the feature requiring privacy protection is calculated using the following formula as the processed feature value of the feature requiring privacy protection. ； in, This represents the probability index value for features that require privacy protection. For features that require privacy protection, This represents the increment of the feature value for features that require privacy protection. Gains for features that require privacy protection; The step involves using the Laplace mechanism to protect the leaf nodes of the trained local decision tree model based on the client's privacy budget, resulting in a processed local decision tree model. Specifically, this includes: Based on the client's privacy budget, calculate the privacy budget of the leaf node using the following formula; ； in, Privacy budget for leaf nodes, Let m be the privacy budget for the m-th client in the k-th training epoch; Based on the privacy budget of the leaf nodes, the Laplace mechanism is used to protect the leaf nodes in the trained local decision tree model using the following formula. ； in, The weight of the j-th leaf node after processing. Let be the set of processed training data used to train the j-th leaf node. and Let be the first and second gradient statistics of the loss function value corresponding to the i-th processed training data, respectively. Privacy budget for leaf nodes, The parameters are known.

2. The decision tree ensemble training method based on differential privacy according to claim 1, characterized in that, The local training dataset was obtained in the following manner: Multiple pieces of privacy data are obtained from the privacy data set using random sampling. A local training data set is constructed by using multiple pieces of the aforementioned privacy data as training data.

3. The decision tree ensemble training method based on differential privacy according to claim 1, characterized in that, The step of selecting privacy-preserving training data from the local training dataset and constructing an important feature set specifically includes: The first feature set is constructed using the following formula; ； in, For features in the training data, For the first feature set, Features in all training data The set of eigenvalues; This represents the number of training data sets in the local training dataset. An incomplete privacy index for training data; Construct a second feature set based on expert experience; The union of the first feature set and the second feature set is obtained as the important feature set.

4. A decision tree ensemble training device based on differential privacy, characterized in that, The device is applied to a distributed integration system, the distributed cluster system including a server and multiple clients connected to the server, each client having a different privacy data set, the privacy data set being used for training the decision tree model, the privacy data in the privacy data set being material processing parameter data and performance parameter data, the device comprising: The data acquisition module is used to acquire the privacy budget of the client sent by the server and the global decision tree model obtained after training by another client; A feature selection module requiring privacy protection is used to select training data requiring privacy protection from a local training dataset and construct an important feature set; the local training dataset is a subset of the privacy data set; The feature protection processing module is used to protect the privacy-required features in the training data using an exponential mechanism based on the client's privacy budget, thereby obtaining the processed training data. The training module is used to train the global decision tree model using the processed training data to obtain the trained local decision tree model. Based on the client's privacy budget, the Laplace mechanism is used to protect the leaf nodes in the trained local decision tree model to obtain the processed local decision tree model, and the processed local decision tree model is sent to the server. The feature protection processing module specifically includes: The privacy budget calculation submodule for internal nodes is used to calculate the privacy budget of internal nodes based on the privacy budget of the client using the following formula; ； in, For the privacy budget of internal nodes, This represents the maximum depth value for training the decision tree model. Let m be the privacy budget for the m-th client in the k-th training epoch; The feature protection processing submodule is used to calculate the probability index value of the feature that needs privacy protection based on the privacy budget of the internal node, using the following formula as the processed feature value of the feature that needs privacy protection. ； in, This represents the probability index value for features that require privacy protection. For features that require privacy protection, This represents the increment of the feature value for features that require privacy protection. Gains for features that require privacy protection; The training module specifically includes: The privacy budget calculation submodule for leaf nodes is used to calculate the privacy budget of leaf nodes based on the client's privacy budget using the following formula; ； in, Privacy budget for leaf nodes, Let m be the privacy budget for the m-th client in the k-th training epoch; The leaf node protection submodule is used to protect the leaf nodes in the trained local decision tree model according to the privacy budget of the leaf nodes and using the Laplace mechanism, using the following formula. ； in, The weight of the j-th leaf node after processing. Let be the set of processed training data used to train the j-th leaf node. and Let be the first and second gradient statistics of the loss function value corresponding to the i-th processed training data, respectively. Privacy budget for leaf nodes, The parameters are known.

5. The decision tree ensemble training device based on differential privacy according to claim 4, characterized in that, The local training dataset was obtained in the following manner: Multiple pieces of privacy data are obtained from the privacy data set using random sampling. A local training data set is constructed by using multiple pieces of the aforementioned privacy data as training data.

6. The decision tree ensemble training device based on differential privacy according to claim 4, characterized in that, The feature selection module requiring privacy protection specifically includes: The first feature set construction submodule is used to construct the first feature set using the following formula; ； in, For features in the training data, For the first feature set, Features in all training data The set of eigenvalues; This represents the number of training data sets in the local training dataset. An incomplete privacy index for training data; The second feature set construction submodule is used to construct a second feature set based on expert experience. The important feature set acquisition submodule is used to obtain the union of the first feature set and the second feature set as the important feature set.

Images