Vertical federated learning method and apparatus, and electronic device and readable storage medium

Abstract

The present disclosure relates to a vertical federated learning method and apparatus, and an electronic device and a readable storage medium. In the method, when a plurality of participants train a random forest model on the basis of a common sample that participates in vertical federated learning, for a node to be trained on a decision tree that is included in the random forest model, each of the plurality of participants encrypts, by means of a semi-homomorphic encryption technique, an identifier for indicating whether a training sample is present in said node, and then sends the identifier to each other, and the other participant calculates a required ciphertext calculation result in a ciphertext space; in addition, the ciphertext calculation result is processed by using a differential privacy technique, and is then fed back to each other, thereby ensuring the security of the published ciphertext calculation result, preventing related information of a sample from being obtained by means of backstepping the ciphertext calculation result, and facilitating an improvement in the security of the sample.

Description

Vertical federated learning method, device, electronic device and readable storage medium

[0001] This application claims priority to the Chinese invention patent application entitled “Vertical federated learning method, device, electronic device and readable storage medium” filed on March 21, 2023, with application number 202310282556.3. The entire contents of that application are incorporated by reference into this application. Technical Field

[0002] The present disclosure relates to the field of data processing technology, and in particular to a vertical federated learning method, device, electronic device, and readable storage medium. Background Art

[0003] Federated learning is one of the most popular machine learning technologies. It solves the problem of how to jointly train a global model on a virtual "aggregated" dataset with multiple participants while protecting their data security. Federated learning can be further categorized into horizontal federation, vertical federation, and federated migration. In vertical federation, the sample data in the datasets owned by multiple participants overlap, but the features are complementary. This makes it suitable for scenarios where multiple features serve the same business label. Therefore, vertical federated learning has extremely wide applications. Random forest is one of the mainstream machine learning algorithms and is widely used in classification scenarios.

[0004] Currently, training random forest models using vertical federated learning typically involves using a trusted third-party server to collect feature data from each participant's samples, as well as information such as intermediate values ​​generated during the training process. The third-party server then iteratively updates the global model and distributes the finalized global model parameters to each participant. This approach to training random forest models using vertical federated learning results in lower sample security.

[0005] Summary of the Invention

[0006] In order to solve the above technical problems, the present disclosure provides a vertical federated learning method, device, electronic device and readable storage medium.

[0007] In a first aspect, the present disclosure provides a vertical federated learning method, comprising:

[0008] When training a random forest model based on a common sample participating in vertical federated learning, for a node to be trained on a decision tree included in the random forest model, semi-homomorphically encrypt a first identifier indicating whether the sample exists and a category label of the sample to obtain a first ciphertext, and send the first ciphertext to other participants, so that the other participants calculate a ciphertext calculation result that matches the type of the node to be trained based on the first ciphertext in the ciphertext space;

[0009] receiving a second ciphertext sent by the other participant, and training the node to be trained according to the type of the node to be trained and the second ciphertext to configure the category of the node to be trained or configure a corresponding splitting feature value; the second ciphertext includes a differential privacy processing result obtained by the other participant performing differential privacy processing on the ciphertext calculation result;

[0010] For the decision tree, the nodes to be trained are updated for training until the decision tree training end condition is met to obtain a target decision tree; and the training of all decision trees in the random forest model is completed to obtain a target random forest model.

[0011] In a second aspect, the present disclosure provides a vertical federated learning method, comprising:

[0012] When training a random forest model based on a common sample participating in vertical federated learning, receiving, for a node to be trained on a decision tree included in the random forest model, a first ciphertext sent by another participant, where the first ciphertext is obtained by semi-homomorphically encrypting, by the other participant, a first identifier indicating whether the sample exists and a category label of the sample;

[0013] Obtaining, in a ciphertext space, a ciphertext calculation result that matches the type of the node to be trained based on the first ciphertext, performing differential privacy processing on the ciphertext calculation result to obtain a second ciphertext, and sending the second ciphertext to the other participants, so that the other participants perform random forest model training based on the second ciphertext;

[0014] receiving feedback information sent by the other participants, and configuring the category of the node to be trained or configuring a corresponding splitting feature value according to the feedback information and the second ciphertext;

[0015] For the decision tree, the nodes to be trained are updated for training until the decision tree training end condition is met to obtain a target decision tree; and the training of all decision trees in the random forest model is completed to obtain a target random forest model.

[0016] In a third aspect, the present disclosure provides a vertical federated learning apparatus, comprising:

[0017] a training module configured to, when training a random forest model based on a common sample participating in longitudinal federated learning, semi-homomorphically encrypt, for a node to be trained on a decision tree included in the random forest model, a first identifier indicating whether the sample exists and a category label of the sample to obtain a first ciphertext;

[0018] a transceiver module, configured to send the first ciphertext to other participants, so that the other participants can calculate a ciphertext calculation result matching the type of the node to be trained based on the first ciphertext in the ciphertext space;

[0019] The transceiver module is further configured to receive a second ciphertext sent by the other party;

[0020] The training module is further used to train the node to be trained according to the type of the node to be trained and the second ciphertext to configure the category of the node to be trained or configure the corresponding split feature value; the second ciphertext includes the differential privacy processing result obtained by the other participants performing differential privacy processing on the ciphertext calculation result; for the decision tree, the node to be trained is updated for training until the decision tree training end condition is met to obtain the target decision tree; the training of all decision trees in the random forest model is completed to obtain the target random forest model.

[0021] In a fourth aspect, the present disclosure provides a vertical federated learning apparatus, comprising:

[0022] a transceiver module configured to receive, when training a random forest model based on a common sample participating in vertical federated learning, a first ciphertext sent by another participant for a node to be trained on a decision tree included in the random forest model, where the first ciphertext is obtained by semi-homomorphically encrypting a first identifier indicating whether the sample exists and a category label of the sample by the other participant;

[0023] a training module, configured to calculate, in a ciphertext space, based on the first ciphertext, a ciphertext calculation result that matches the type of the node to be trained, and perform differential privacy processing on the ciphertext calculation result to obtain a second ciphertext;

[0024] The transceiver module is further configured to send the second ciphertext to the other participants, so that the other participants perform random forest model training based on the second ciphertext; and receive feedback information sent by the other participants;

[0025] The training module is used to train the nodes to be trained based on the feedback information and the second ciphertext to configure the categories of the nodes to be trained or configure corresponding split feature values; for the decision tree, the nodes to be trained are updated for training until the decision tree training end condition is met to obtain a target decision tree; all decision trees in the random forest model are trained to obtain a target random forest model.

[0026] In a fifth aspect, the present disclosure provides an electronic device, comprising: a memory and a processor;

[0027] The memory is configured to: store computer program instructions;

[0028] The processor is configured to: execute the computer program instructions so that the electronic device implements the vertical federated learning method as described in the first aspect or the second aspect.

[0029] In a fourth aspect, the present disclosure provides a readable storage medium comprising: computer program instructions; at least one processor of an electronic device executes the computer program instructions, so that the electronic device implements the vertical federated learning method as described in the first aspect or the second aspect.

[0030] In a fifth aspect, the present disclosure provides a computer program product. When the computer program product is executed by an electronic device, the electronic device implements the vertical federated learning method as described in the first aspect or the second aspect. BRIEF DESCRIPTION OF THE DRAWINGS

[0031] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present disclosure and, together with the description, serve to explain the principles of the present disclosure.

[0032] In order to more clearly illustrate the embodiments of the present disclosure or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0033] FIG1 is a schematic diagram of an application scenario of the vertical federated learning method provided by the present disclosure;

[0034] FIG2 is a schematic diagram of characteristic data distribution of a sample provided by an embodiment of the present disclosure;

[0035] FIG3 is an overall framework diagram of a vertical federated learning method provided by an embodiment of the present disclosure;

[0036] FIG4 is a flow chart of a vertical federated learning method provided by an embodiment of the present disclosure;

[0037] FIG5 is a flowchart of a vertical federated learning method provided by another embodiment of the present disclosure;

[0038] FIG6 is a flowchart of a vertical federated learning method provided by another embodiment of the present disclosure;

[0039] FIG7 is a schematic diagram of the structure of a target random forest model provided by an embodiment of the present disclosure;

[0040] FIG8 is a schematic diagram of the structure of a vertical federated learning device provided by an embodiment of the present disclosure;

[0041] FIG9 is a schematic structural diagram of a vertical federated learning device provided in another embodiment of the present disclosure. DETAILED DESCRIPTION

[0042] In order to more clearly understand the above-mentioned objectives, features and advantages of the present disclosure, the scheme of the present disclosure will be further described below. It should be noted that the embodiments of the present disclosure and the features therein can be combined with each other in the absence of conflict.

[0043] In the following description, many specific details are set forth to facilitate a full understanding of the present disclosure, but the present disclosure may also be implemented in other ways different from those described herein; it is obvious that the embodiments in the specification are only part of the embodiments of the present disclosure, rather than all of the embodiments.

[0044] First, some professional terms involved in this disclosure are introduced in detail:

[0045] 1. Random Forest Model

[0046] The random forest model is a classifier that includes multiple decision trees. Each decision tree consists of nodes and directed edges. Nodes include leaf nodes and internal nodes. The parameters corresponding to leaf nodes are categories, which indicate the classification results of samples / objects to be classified that reach that node. The parameters corresponding to internal nodes are splitting features, which are used to classify training samples / objects to be processed that reach that node into the connected left or right child nodes.

[0047] 2. Semi-homomorphic encryption

[0048] For any participant, such as participant G, they have homomorphic keys pk, sk, encryption function E, and decryption function D. For any data x, y, they satisfy the following properties:

[0049] a. Homomorphic addition: as well as

[0050] b. Homomorphic multiplication:

[0051] 3. Differential Privacy

[0052] For any data x, given the privacy budget ε and sensitivity s, differentially private data that meets the privacy budget can be obtained.

[0053] It can be expressed by the formula:

[0054] The present disclosure provides a vertical federated learning method, device, electronic device and readable storage medium. In this method, when multiple participants train a random forest model based on a common sample participating in the vertical federated learning, for the nodes to be trained on the decision tree contained in the random forest model, the multiple participants encrypt an identifier used to indicate whether the training sample exists in the node to be trained using semi-homomorphic encryption technology, and then send it to the other party. The other party respectively performs the required ciphertext calculation results in the ciphertext space; in addition, the ciphertext calculation results are processed using differential privacy technology and then fed back to the other party, thereby ensuring the security of the published ciphertext calculation results, avoiding the related information of the sample obtained by reverse inference through the ciphertext calculation results, and helping to improve the security of the sample.

[0055] The vertical federated learning method provided by the present disclosure will be introduced in detail below through some embodiments combined with drawings and scenarios.

[0056] Figure 1 is a schematic diagram of an application scenario for a vertical federated learning method according to an embodiment of the present disclosure. As shown in Figure 1 , the scenario includes two participants: a first participant 101 and a second participant 102. The two participants can be servers, service clusters, cloud servers, cloud platforms, and the like. This disclosure does not limit the types of participants.

[0057] FIG2 is a schematic diagram of the data distribution of samples (also referred to as training samples) in the first participant 101 and the second participant 102. Referring to FIG2 , the first participant 101 possesses the sample's m1-dimensional first feature data and category label, while the second participant 102 possesses the sample's m2-dimensional second feature data. The first participant 101 and the second participant 102 each possess different feature data, but have the same sample identity document (ID) for the same training sample. In the present disclosure, samples are typically structured data, and may be, but are not limited to, images, voice, text, and other types of samples.

[0058] The first participant 101 and the second participant 102 can train a global random forest model based on common samples through longitudinal federated learning and use the trained random forest model to perform classification tasks.

[0059] During the model training phase, when calculating the number of samples present at a node to be trained on a decision tree included in the random forest model or the splitting gain of each eigenvalue, the first and second parties 101 and 102 each semi-homomorphically encrypt an identifier indicating whether a sample exists at the current node and send it to the other party. The other party then processes the encrypted result in the ciphertext space to obtain the required ciphertext calculation result. Before publishing the ciphertext calculation result to the other party, the ciphertext calculation result is processed using differential privacy technology to ensure the security of the published ciphertext calculation result.

[0060] In the prediction stage, for the object to be classified, the first participant 101 and the second participant 102 each use a trained random forest model to output classification results based on different feature values ​​of the object to be classified, and then encrypt the classification results through semi-homomorphic encryption technology and send them to the object to ensure the security of the object to be classified.

[0061] In the scenario shown in Figure 1, the global random forest model is trained through vertical federated learning. There is no need for a third-party server to collect information about samples owned by the first participant 101 and the second participant 102 respectively, as well as the intermediate information generated during the training process, which can well protect the security of samples and training-related data.

[0062] FIG3 is a schematic diagram of the overall framework of the vertical federated learning method provided by an embodiment of the present disclosure. Referring to FIG3, when training begins, the first participant and the second participant each determine whether the node type is a leaf node or an internal node based on the depth of the node to be trained; or, the first participant and the second participant interactively determine the number of training samples existing on the current node, and then, based on the depth of the node to be trained and the number of training samples existing on the node to be trained, determine whether the node type is a leaf node or an internal node; if the node type of the node to be trained is a leaf node, the first participant and the second participant use semi-homomorphic encryption technology and differential privacy technology to interactively calculate the category of the node to be trained; if the node type of the node to be trained is an internal node, the first participant and the second participant use semi-homomorphic encryption technology and differential privacy technology to interactively calculate the split feature value of the node type of the node to be trained. Afterwards, if the node to be trained is an internal node, the samples are divided into its left and right child nodes according to the calculated split eigenvalues, and the node to be trained is updated to any child node for training; if the node to be trained is a leaf node, the node to be trained is updated to any child node of its sibling node for training, until the nodes of all decision trees contained in the random forest model are trained.

[0063] FIG4 is a flow chart of a method for vertical federated learning provided by an embodiment of the present disclosure. In the following embodiment, a first participant and a second participant perform vertical federated learning as an example, where the first participant has the category label of the sample. Referring to FIG4 , the method of this embodiment includes:

[0064] S401. When training a random forest model based on a common sample participating in vertical federated learning, the first participant performs semi-homomorphic encryption on a first identifier indicating whether the sample exists and a category label of the sample for a node to be trained on a decision tree contained in the random forest model to obtain a first ciphertext.

[0065] The number of common samples participating in vertical federated learning can be multiple, and the first participant has the first feature data and category label of the sample. The sample can come from any field, any content, and any storage format, and this disclosure does not limit them.

[0066] S402: The first participant sends a first ciphertext to the second participant.

[0067] The first identifier is used to indicate whether the sample exists on the node to be trained of the random forest model of the first participant. It should be noted that, during the model training process, as the nodes to be trained are continuously updated, the sample will be continuously split until it reaches the leaf node, and the first identifier of the sample will also be continuously updated. For example, when the first identifier takes a value of 0, it indicates that the sample does not exist in the current node to be trained, and when the first identifier takes a value of 1, it indicates that the sample exists in the current node to be trained. Assuming that node 1 is the root node, and node 2 and node 3 are the left and right child nodes of node 1, when the sample is input to node 1, the first identifier value of all samples is 1. If the root node has a splitting feature value, then according to the first feature data of the sample and the splitting feature value of node 1, the sample is distinguished to node 2 or node 3. The first identifier value of the sample distinguished to node 2 on node 2 is 1, the first identifier value of the sample distinguished to node 3 on node 3 is 1, the first identifier of the sample distinguished to node 3 on node 2 is 0, and the first identifier of the sample distinguished to node 2 on node 3 is 0.

[0068] The class label of a sample is used to indicate the class to which the sample belongs. Common samples participating in vertical federated learning can belong to one of multiple classes.

[0069] As a possible implementation method, for the node to be trained, the first participant can generate a category label vector for each sample based on the category label of the sample and the first identifier. The category label vector can indicate whether the sample exists in the node to be trained and the category of the sample. Afterwards, the first participant uses its own homomorphic encryption key to encrypt the category label vector to obtain an encrypted category label vector, and the first ciphertext includes the encrypted category label vector corresponding to each sample.

[0070] In addition, the first participant can concatenate the encrypted category label vectors corresponding to each sample into one vector and send it to the second participant. The second participant can obtain the ciphertext calculation result that matches the type of the current node to be trained by calculating the concatenated vector, thereby reducing the data processing amount of the second participant and improving training efficiency.

[0071] Since the node to be trained may be an internal node or a leaf node of the decision tree, the second party needs to obtain different ciphertext calculation results based on the first ciphertext for different types and feed them back to the first party. The first party can determine the type of the current node based on the depth of the node to be trained in the decision tree and the number of samples of the node to be trained.

[0072] For example, before training, the maximum tree depth and minimum number of split samples of the decision tree can be initialized. During training, the first participant can determine the type of node to be trained based on the initialized maximum tree depth and minimum number of split samples, which can include the following situations:

[0073] 1. If the depth of the node to be trained is greater than or equal to the maximum tree depth of the decision tree, the node to be trained is a leaf node; 2. If the depth of the node to be trained is less than the maximum tree depth of the decision tree, and the number of training samples for the node to be trained is less than the minimum number of split samples, the node to be trained is a leaf node; 3. If the depth of the node to be trained is less than the maximum tree depth of the decision tree, and the number of training samples for the node to be trained is greater than or equal to the minimum number of split samples, the node to be trained is an internal node.

[0074] S403. The second participant calculates a ciphertext calculation result that matches the type of the node to be trained in the ciphertext space based on the first ciphertext, and performs differential privacy processing on the ciphertext calculation result to obtain a second ciphertext.

[0075] The first participant may send node type information to the second participant to indicate whether the type of the node to be trained is a leaf node or an internal node, so that the second participant can obtain the correct ciphertext calculation result.

[0076] 1. If the node to be trained is a leaf node, the second party calculates the number of encrypted samples for each category based on the first ciphertext and the second identifier indicating whether the sample exists. The number of encrypted samples for each category is the ciphertext calculation result. The number of encrypted samples for a particular category refers to the number of samples for that category that exist on the nodes to be trained in both the first and second parties. For example, if nodes a1 and a2 are the nodes to be trained in the first and second parties, respectively, and the random forest model contains five categories, the number of samples that exist simultaneously on nodes a1 and a2 is calculated for each of the five categories.

[0077] As a possible implementation, when the first ciphertext includes an encrypted category label vector corresponding to each sample, the second participant may homomorphically multiply each element of the encrypted category label vector for the same sample by a second identifier of the sample, where the second identifier indicates whether the sample exists on the corresponding node to be trained on the second participant. Then, for each category, the elements corresponding to the category in each sample are homomorphically added to obtain the initial number of encrypted samples corresponding to each category, where the initial number of encrypted samples has not been differentially private processed. Next, differential privacy noise is generated based on a preset sensitivity and differential privacy budget, and then added to the initial encrypted sample via homomorphic addition to obtain the number of encrypted samples corresponding to each category included in the second ciphertext. The second participant obtains the number of encrypted samples corresponding to each category by performing a calculation in the ciphertext space. The second participant perturbs the ciphertext calculation result to be sent to the first participant using differential privacy processing, preventing the first participant from obtaining the number of plaintext samples corresponding to each category, thereby protecting the security of the training samples.

[0078] 2. If the node to be trained is an internal node, the second participant splits the sample based on the first ciphertext and multiple first random eigenvalues ​​to obtain a first sample splitting result; and, based on the second sample splitting result sent by the first participant, obtains a ciphertext splitting gain for the second candidate splitting eigenvalue. The second sample splitting result is the ciphertext result obtained by the first participant by splitting the sample based on a pre-received third ciphertext, multiple second random eigenvalues, and the sample's category label and adding noise; the third ciphertext is the encrypted tag vector obtained by the second participant by semi-homomorphically encrypting the second identifier. The second participant decrypts the second sample splitting result using the private key in the semi-homomorphic key pair to obtain multiple data items. For each data item, the second participant calculates an initial splitting gain for each second random feature using a weighted approach. Then, using differential privacy technology, after adding noise to the initial splitting gain for each second random eigenvalue, the optimal local splitting gain is selected. The second random eigenvalue corresponding to the optimal local splitting gain is the second candidate splitting feature. If the node to be trained is a second ciphertext, the ciphertext includes: the first sample splitting result and the ciphertext splitting gain for the second candidate splitting eigenvalue.

[0079] S404: The second participant sends a second ciphertext to the first participant.

[0080] S405: The first participant trains the node to be trained according to the type of the node to be trained and the second ciphertext to configure the category of the node to be trained or configure a corresponding splitting feature value.

[0081] This step configures the nodes to be trained on the decision tree contained in the random forest model corresponding to the first participant.

[0082] If the type of the node to be trained is a leaf node, the first participant decrypts the second ciphertext based on the semi-homomorphic private key to obtain the number of encrypted samples corresponding to each category, and configures the category corresponding to the maximum number of encrypted samples as the category of the node to be trained.

[0083] If the type of the node to be trained is an internal node, the first participant calculates the ciphertext splitting gain of the first candidate splitting feature based on the first sample splitting result included in the second ciphertext, and the first sample splitting result is the ciphertext result obtained by the second participant by splitting the sample based on multiple first random feature values ​​and the first ciphertext; the splitting feature value of the node to be trained is configured by comparing the ciphertext splitting gain of the first candidate splitting feature and the second candidate splitting feature indicated by the second ciphertext.

[0084] S406: The second participant configures the category of the corresponding node to be trained or configures the corresponding split feature value according to the feedback information sent by the first participant and the second ciphertext.

[0085] This step configures the nodes to be trained on the decision tree contained in the random forest model corresponding to the second participant.

[0086] If the type of the node to be trained corresponding to the first participant is a leaf node, after the first participant configures the category of the leaf node, it will send the category identification information of the leaf node (such as the category's digital number, category ID, etc.) as feedback information to the second participant so that it can configure the corresponding node to be trained in the second participant.

[0087] If the type of the node to be trained corresponding to the first participant is an internal leaf node, the first participant generates a fourth ciphertext by performing differential privacy processing on the ciphertext splitting gain of the first candidate split feature, and sends the fourth ciphertext as feedback information to the second participant. The second participant configures the split feature value of the node to be trained corresponding to the second participant by comparing the ciphertext splitting gain of the first candidate split feature and the second candidate split feature after differential privacy processing.

[0088] It should be noted that in steps S405 and S406, the ciphertext splitting gain of the first candidate splitting feature value is the local optimal splitting gain calculated by the first participant for the second participant, and the ciphertext splitting gain of the second candidate splitting feature value is the local optimal splitting gain calculated by the second participant for the first participant. By comparing the size of the local optimal splitting gain calculated by the first participant and the second participant for each other, the optimal splitting feature of the node to be trained can be determined. In some embodiments, the smaller the splitting gain value, the better the performance of distinguishing the sample into the left and right child nodes of the node to be trained based on the corresponding splitting feature value; the candidate splitting feature value with a smaller splitting gain can be determined as the target splitting feature data corresponding to the current node. In this disclosure, in order to ensure the security of the feature data of the sample, the feature value will not be sent to the other party. Therefore, if the party with a smaller ciphertext splitting gain sets the corresponding candidate splitting feature value as the splitting feature value of the node to be trained, the party with a larger ciphertext splitting gain sets the corresponding node to be trained as an empty node, where an empty node means that the sample of the node will be passed to the left and right child nodes respectively without any change.

[0089] In other embodiments, a larger splitting gain indicates better performance in distinguishing training samples into the left and right child nodes of the current node based on the corresponding splitting feature value; the candidate splitting feature value with a larger splitting gain can be determined as the target splitting feature data corresponding to the current node. The configuration method is similar and will not be repeated here.

[0090] S407: The first participant and the second participant update the nodes to be trained and perform training until the decision tree training end condition is met to obtain a target decision tree; all decision trees in the random forest model are trained to obtain a target random forest model.

[0091] In the method provided in this embodiment, when the first participant and the second participant train a random forest model based on a common sample participating in vertical federated learning, for the nodes to be trained on the decision tree contained in the random forest model, multiple participants encrypt the identifier used to indicate whether the training sample exists in the node to be trained through semi-homomorphic encryption technology, and then send it to the other party. The other participant respectively calculates the ciphertext calculation results required in the ciphertext space; in addition, the ciphertext calculation results are processed using differential privacy technology and then fed back to the other party, thereby ensuring the security of the published ciphertext calculation results, avoiding obtaining relevant information of the sample by reverse deduction through the ciphertext calculation results, and helping to improve the security of the sample.

[0092] In the federated learning method provided by the present disclosure, when the node to be trained is an internal node, it is necessary to determine the split feature value of the node to be trained based on the ciphertext splitting gains of the two candidate split features calculated by the first participant and the second participant respectively. Among them, the splitting gain of the first candidate split feature data is calculated by the first participant, and the splitting gain of the second candidate split feature data is obtained by the second participant. Next, the implementation method of calculating the two candidate split feature values ​​is explained through the embodiments shown in Figures 5 and 6.

[0093] FIG5 is a flow chart of a vertical federated learning method provided by an embodiment of the present disclosure. The method of this embodiment mainly describes how the first participant obtains the ciphertext splitting gain corresponding to the second candidate splitting feature value. Referring to FIG5, the method of this embodiment includes:

[0094] S501: The first participant selects multiple second random feature values.

[0095] The first participant determines first feature data of multiple dimensions based on the first feature data of each sample, where the first feature data of each dimension includes multiple feature values, i.e., second random feature values. Specifically, based on the first feature data of each sample, a range of feature values ​​of the first feature data of each dimension is first determined. First feature data of multiple dimensions that meet the value range requirements can be selected, and then each second random feature value is determined randomly or by other means (e.g., sorting by feature value).

[0096] S502: The first participant splits the sample based on multiple second random feature values, the category label of the sample, and the pre-received third ciphertext and adds random noise to obtain a second sample splitting result.

[0097] As a possible implementation method, the following steps may be included:

[0098] Step a1: The first participant divides each sample into left and right subnodes of the node to be trained according to the selected multiple second random feature values, and obtains the number of left and right subnode samples respectively divided by each second random feature value.

[0099] For example, assuming that the node to be trained is node 1, node 2 and node 3 are the left and right child nodes of node 1 respectively, the first feature data of three dimensions are selected, and three eigenvalues ​​are selected for the first feature data of each dimension based on the value range of the first feature data of each dimension, that is, a total of 9 second random eigenvalues ​​are selected, and a sample differentiation (i.e., splitting) is performed based on each second random eigenvalue, and the samples in node 1 are differentiated into node 2 and node 3, thereby obtaining the number of left and right child node samples corresponding to the 9 second eigenvalues.

[0100] As a possible implementation method, when the first participant distinguishes samples into left and right child nodes based on the selected second random feature value, it can generate a vector corresponding to the left child node and a vector corresponding to the right child node respectively based on the first identifier of the sample, whether the first feature data of the sample satisfies the selected second random feature value, and the category label of the sample.

[0101] Step b1: The first participant performs homomorphic multiplication and homomorphic addition processing on the third ciphertext and the number of left and right child node samples corresponding to each second eigenvalue to obtain a second sample splitting result, wherein the second sample splitting result includes the number of encrypted samples of the left and right child nodes corresponding to each second random eigenvalue.

[0102] Combined with the example in step a, the first participant selected 9 second random feature values. Assuming that the total number of categories of the random forest model is 5, for each second random feature value, the first participant can distinguish each training sample into one of node 2 and node 3. Each sample existing in node 2 may belong to one or more of the above 5 categories. Similarly, each sample existing in node 3 may belong to one or more of the above 5 categories. Since some samples of the second participant may not exist in node 1, these training samples cannot be distinguished into node 2 or node 3. It is necessary to correct the number of samples distinguished into left and right sub-nodes by the first participant based on each second random feature value according to the second identifier of each sample in the second participant on the corresponding node corresponding to node 1 to obtain a higher number of samples with higher accuracy. Taking the left child node as an example, for each selected second random eigenvalue, the first participant performs homomorphic multiplication and homomorphic addition on the samples distinguished to the left child node based on the second random eigenvalue, the category labels of each sample distinguished to the left child node, and the third ciphertext, so as to obtain the number of samples in each category in the corrected left child node; the processing method of the right child node is similar to that of the left child node.

[0103] S503: The first participant sends the second sample splitting result to the second participant.

[0104] S504. The second participant uses the semi-homomorphic private key to decrypt the second sample splitting result to obtain multiple data items; and performs weighted calculation based on the multiple data items to obtain initial splitting gains corresponding to multiple second random eigenvalues.

[0105] S505. The second participant performs differential privacy processing on the initial splitting gains corresponding to the multiple second random eigenvalues, and determines the ciphertext splitting gain of the second candidate splitting eigenvalue based on the differential privacy processing result.

[0106] The second party can add differential privacy noise to the initial splitting gain corresponding to each second random eigenvalue based on the pre-set sensitivity and differential privacy budget, and determine the locally optimal ciphertext splitting gain from the ciphertext splitting gain with the added differential privacy noise. The second random eigenvalue corresponding to the locally optimal ciphertext splitting gain is the second candidate splitting eigenvalue.

[0107] S506: The second participant sends the ciphertext splitting gain of the second candidate splitting eigenvalue to the first participant. Correspondingly, the first participant receives the ciphertext splitting gain corresponding to the second candidate splitting eigenvalue.

[0108] In this embodiment, the first participant and the second participant process the interaction data through semi-homomorphic encryption and differential privacy to calculate the optimal split eigenvalue for the node to be trained. Since the first participant will not send the information of the second random eigenvalue to the second participant, the security of the characteristic data of the sample during the interaction process is effectively protected.

[0109] FIG6 is a flow chart of a vertical federated learning method provided by another embodiment of the present disclosure. The method of this embodiment mainly describes how the second participant obtains the ciphertext splitting gain corresponding to the first candidate splitting feature value. Referring to FIG6, the method of this embodiment includes:

[0110] S601: The second participant selects multiple first random feature values.

[0111] The second participant determines second feature data of multiple dimensions based on the second feature data of each sample, where the second feature data of each dimension includes multiple feature values, namely, first random feature values. Specifically, based on the second feature data of each sample, a range of feature values ​​of the second feature data of each dimension is first determined. Second feature data of multiple dimensions that meet the value range requirements can be selected, and then the multiple first random feature values ​​are determined randomly or by other means (e.g., sorting by feature value).

[0112] S602: The second participant splits the sample based on the multiple first random feature values ​​and the first ciphertext to obtain a first sample splitting result.

[0113] As a possible implementation method, the following steps may be included:

[0114] Step a2: The second participant classifies each sample into a left child node and a right child node of a corresponding node to be trained in the second participant according to the selected multiple first random feature values.

[0115] As a possible implementation method, when the second participant makes a distinction based on the selected first random feature value, it can generate a vector corresponding to the left child node and a vector corresponding to the right child node based on the second identifier of the sample and whether the second feature data of the sample satisfies the selected first random feature value.

[0116] Step b2: The second participant calculates the number of encrypted samples of the left and right sub-nodes distinguished by each first random feature value based on the first ciphertext and the result obtained in step a2, that is, the first sample splitting result.

[0117] In step b2, since the first ciphertext can indicate that the sample is on the corresponding node to be trained in the first participant, when calculating the number of encrypted samples, using the first ciphertext for calculation is equivalent to realizing the need to correct the number of samples of the second participant that are divided into left and right sub-nodes based on each first random feature value according to the first identifier of the sample, thereby obtaining a sample number with higher accuracy.

[0118] S603: The second participant sends the first sample splitting result to the first participant.

[0119] S604. The first participant uses the semi-homomorphic private key to decrypt the first sample splitting result to obtain multiple data items; and obtains initial splitting gains corresponding to multiple first random eigenvalues ​​through weighted calculation based on the multiple decrypted data items.

[0120] S605. The first participant performs differential privacy processing on the initial splitting gains corresponding to the multiple first random eigenvalues, and determines the ciphertext splitting gain of the first candidate splitting eigenvalue based on the differential privacy processing result.

[0121] The first participant can add differential privacy noise to the initial splitting gain corresponding to each first random eigenvalue based on the pre-set sensitivity and differential privacy budget, and determine the locally optimal ciphertext splitting gain from the ciphertext splitting gain with added differential privacy noise. The first random eigenvalue corresponding to the locally optimal ciphertext splitting gain is the first candidate splitting eigenvalue.

[0122] S606: The first participant sends the ciphertext splitting gain of the first candidate splitting eigenvalue to the second participant. Correspondingly, the second participant receives the ciphertext splitting gain corresponding to the first candidate splitting eigenvalue.

[0123] In this embodiment, the first participant and the second participant process the interaction data through semi-homomorphic encryption and differential privacy to calculate the optimal split eigenvalue for the node to be trained. Since the second participant will not send the information of the first random eigenvalue to the first participant, the security of the characteristic data of the sample during the interaction process is effectively protected.

[0124] In a specific embodiment, it is assumed that the two participants are Guest (hereinafter referred to as G) and Host (hereinafter referred to as H), where the data format of the sample owned by G is: <id,f1,f2,....,f n1 >, the data form of the samples owned by H is: <id,f1′,f2′,....,f n2 ′>, that is, each sample contains an id and multiple feature values, and the features owned by G and H are different. G and H have N samples in common.

[0125] It should be noted that the id of the same sample in G is the same as that in H, which also provides a basis for identifying relevant information of the same sample when the two interact in the future.

[0126] The training phase may include the following steps:

[0127] Step 1: Initialize relevant parameters.

[0128] For example, the user can set the differential privacy budget ε, the number of feature data selected p, the number of feature values ​​selected for each dimension feature data q, the minimum number of split samples I min , the maximum depth d of the decision tree max .

[0129] Step 2: G and H add a marker bit t / t' (i.e., first identifier / second identifier) ​​to each sample. A marker bit value of 1 indicates that the sample exists, and a marker bit value of 0 indicates that the sample does not exist. Therefore, the data format of the samples owned by G is: <id,f1,f2,....,f n1 ,t>, the data form of the samples owned by H is: <id,f1′,f2′,....,f n2 ′,t′>. Initially, the value of the mark bit of all samples on the root node is 1. In addition, G and H respectively initialize their corresponding public and private keys (pk G ,sk G )、(pk H ,sk H ).

[0130] Step 3: Determine the depth d of the node to be trained. If the depth d of the node to be trained is less than the maximum depth d max, then jump to step 4; if the depth d of the node to be trained is equal to the maximum depth d max , then follow the steps 3.1 to 3.5 below.

[0131] Step 3.1, for each category c1,c2,....,c l (l represents the total number of categories), G obtains the category label vector of the sample based on the category label of the sample in, c id Indicates the category label of the sample. ∧ represents the logical AND operation.

[0132] Step 3.2, G uses the public key pk G The category tag vector of each sample is semi-homomorphically encrypted to obtain the ciphertext category tag vector of each sample and sent to H. The ciphertext category tag vector of N samples can be expressed as follows:

[0133] In step 3.3, H obtains the number of encrypted samples corresponding to each category through homomorphic addition and homomorphic multiplication based on the sample tag vector (the vector composed of the tag bits t′) of the local sample and the ciphertext category tag vector sent by G. It should be noted that the encrypted sample number obtained at this time has not been differentially privately processed.

[0134] Among them, the category label vector of the samples owned by H is expressed by the following formula:<id1,t1′> ,<id2,t2′> ,..., <id N ,t N ′>

[0135] The process of obtaining the number of encrypted samples of each category is expressed by the following formula:

[0136] Next, H generates differential privacy noise based on the preset sensitivity s and differential privacy budget ε, and adds differential privacy noise to the number of encrypted samples in each category. The differential privacy processing process can be expressed as follows:

[0137] H will Send to G.

[0138] Step 3.4, G reception And use the private key sk G Decryption yields: And set the category of the node to be trained to

[0139] Step 3.5: Return the node. The node to be trained is a leaf node, and the category of the leaf node is the category determined in step 3.4.

[0140] Step 4: G and H interactively calculate the number of samples existing in the node to be trained.

[0141] Exemplarily, this can be achieved through the following steps 4.1 to 4.3:

[0142] Step 4.1: G uses the public key in the homomorphic key pair to semi-homomorphically encrypt the tag bit of each sample in the sample set to obtain the ciphertext tag vector corresponding to each sample, which is expressed as: G sends the ciphertext tag vector to H.

[0143] In step 4.2, H searches for the tag bit of each sample based on its ID, and obtains the number of encrypted samples of the node to be trained through homomorphic addition and multiplication. The formula is as follows:

[0144] H generates differential privacy noise based on the pre-set sensitivity s and differential privacy budget ε, and uses the differential privacy noise to perturb the number of encrypted samples of the training node. This process can be expressed as follows: And [I * ]Send to G.

[0145] Step 4.3, G decryption to get the number of samples of the node to be trained

[0146] If I * Less than I min , then execute steps 3.1 to 3.5 to set the category for the node to be trained; if I * Greater than or equal to I min , then execute step 5 to set the split feature value for the node to be trained.

[0147] Step 5: G and H interactively calculate the split eigenvalues ​​of the nodes to be trained.

[0148] Exemplarily, the following steps 5.1 to 5.11 may be included.

[0149] Step 5.1, H uses the key pk H Perform semi-homomorphic encryption on the tag bit of each sample in H at the currently trained node in H to obtain the ciphertext tag vector of each sample: And send the ciphertext tag vector to G.

[0150] Step 5.2, for each category c1,c2,....,c l (l represents the total number of categories), G obtains the category label vector of the sample based on the category label of each sample in, c id Indicates the category to which the sample belongs, that is, the category label corresponding to the sample, and t indicates the value of the sample's mark bit.

[0151] G performs semi-homomorphic encryption on the category label vector of each sample to obtain the ciphertext category label vector of each sample and sends it to H. Please refer to the description of step 3.2.

[0152] Step 5.3, H randomly selects feature data of p dimensions, and selects q eigenvalues ​​for each dimension of feature data, expressed as: Where 1≤x i ≤n2,1≤y j ≤N. For each selected eigenvalue H judges each sample based on the eigenvalue Distinguish into left and right child nodes and generate corresponding results: <id,t′ left >、 <id,t′ right >, where

[0153] Step 5.4: H calculates the number of encrypted samples of the left and right child nodes distinguished by each eigenvalue based on the information received in step 5.2. The formula can be expressed as:

[0154] in,

[0155] H is the number of encrypted samples of the left and right child nodes corresponding to all eigenvalues Send to G.

[0156] Step 5.5: G randomly selects feature data of p dimensions, and selects q eigenvalues ​​for each dimension, expressed as: Where 1≤x i ≤n2,1≤y j ≤N. For each selected eigenvalue G judges each sample based on the eigenvalue Distinguish into left and right child nodes, and combine the category labels of the samples to generate corresponding results:

[0157] in,

[0158] Step 5.6: G calculates the number of encrypted samples of the left and right child nodes distinguished by each eigenvalue based on the information received in step 5.1. The formula can be expressed as:

[0159] in,

[0160] G is the number of encrypted samples of the left and right child nodes corresponding to all eigenvalues Send to H.

[0161] Step 5.7, G and H use their own homomorphic private keys to decrypt and Each item in .

[0162] In step 5.8, G and H calculate the splitting gain of each eigenvalue according to the following formula:

[0163] in, When calculating G, use I′ instead of I.

[0164] In the present disclosure, the smaller the splitting gain is, the better the performance of splitting the sample into the left and right child nodes is, and the closer the corresponding eigenvalue is to the optimal local splitting gain.

[0165] In step 5.9, G and H use differential privacy technology to add the split gain of each eigenvalue to the differential privacy noise, which can be expressed as: Afterwards, the optimal local split gain is selected G and H both compare the optimal local splitting gain sent by the other end and the optimal local splitting gain calculated by the local end, and select the eigenvalue corresponding to the smaller splitting gain as the splitting eigenvalue of the node to be trained.

[0166] In step 5.9, the sensitivity to differential privacy noise can be obtained as follows:

[0167] Step a1: Set the sensitivity s to 0 initially and calculate the initial splitting gain g0 according to the formula in step 5.8.

[0168] Step a2: For any Modify I to I = I ± 1 and calculate the variable gain g according to the formula in step 5.8. + 、g - ; If |g + -g0|>s, then s=|g + -g0|; if|g - -g0|>s, then s=|g - -g0|.

[0169] Step 5.10, according to The optimal splitting gain in the training node is obtained, and the corresponding participant (the party with the eigenvalue corresponding to the optimal splitting gain) uses the eigenvalue corresponding to the splitting gain as the splitting eigenvalue of the node to be trained to split the sample of the node to be trained into the left and right child nodes. For example, the splitting eigenvalue corresponding to the node to be trained is If the eigenvalues ​​of the sample Then set the mark bit of the sample in the left child node to t=t∧1, and the mark bit of the sample in the right child node to t=0; if the characteristic value of the sample Then the mark bit of the sample in the left child node is set to t=0, and the mark bit of the sample in the right child node is set to t=t∧1.

[0170] For the party that does not have the eigenvalue corresponding to the optimal split gain, the node to be trained can be set to an empty node, and then all samples of the node to be trained can be passed to the left and right child nodes respectively.

[0171] Step 5.11: Return to the node.

[0172] Since the node to be trained has left and right child nodes, the left child node or the right child node can be the node to be trained, and the process returns to step 3 to train the updated node to be trained.

[0173] By continuously updating and training the nodes to be trained until all nodes in all decision trees contained in the random forest models in G and H are trained, the trained target random forest model is obtained.

[0174] Based on steps 1 to 5 of this embodiment, G and H can be trained to obtain a global random forest model. In combination with the description in the aforementioned step 5.10, it can be seen that for the party with the target split feature value, the sample can be divided into left and right child nodes according to the target split feature value. In order to ensure the security of the sample, the target split feature value will not be sent to the other party, and the party that does not have the target split feature value will set the node to be trained to an empty node. For example, therefore, the same decision tree in the trained target random forest model is different in the two participating parties. For example, the embodiment shown in Figure 7 shows a schematic diagram of the structure of the same decision tree in G and H respectively. The structure of a decision tree in G is shown as decision tree 1 in Figure 7, and the structure of the corresponding decision tree in H is shown as decision tree 2 in Figure 7. Node a is an internal node in the decision tree. In decision tree 1, node a has a corresponding split feature value A, and in decision tree 2, node a is an empty node. Node b is another internal node in the decision tree. In decision tree 1, node b is an empty node, and in decision tree 2, node b has a corresponding split feature value B. That is, G has the split eigenvalue A of node a, and H has the split eigenvalue B of node b.

[0175] In addition, G has the category information of the leaf node. In order to ensure the security of the samples and the objects to be classified, G can send the category identifier (such as a digital number) corresponding to the leaf node to H. H configures the received category identifier as the category of the leaf node. During prediction, H can return the category identifier (such as a digital number) and the id of the sample reaching the node or the id of the object to be classified to G.

[0176] Next, the two parties can predict the category of the object to be classified based on the trained random forest model. For example, the following steps can be included:

[0177] Assume that the first participant (for example, G) has the first feature data of the object to be classified. In the case of including the mark bit, the data form can be expressed as: <id,f1,f2,....,f n1 ,t>;The second feature data of the object to be classified owned by the second participant (for example, H), when including the mark bit, the data format is: <id,f1′,f2′,....,f n2 ′,t′>.

[0178] Starting from the root node, if the node in the decision tree has a split feature value, the object to be classified is divided into the left and right child nodes based on the split feature value of the node and the feature value of the object to be classified; if the node in the decision tree is an empty node, the object to be classified is passed to the left and right child nodes respectively.

[0179] When the object to be classified reaches the leaf node, the first participant encrypts the tag bit of the object to be classified to obtain The ciphertext classification result is sent to the second party.

[0180] The second participant performs homomorphic multiplication based on the tag bit t′ of the object to be classified in the corresponding leaf node to obtain the fusion result. Afterwards, the second participant sends the fusion result to the first participant. It should be noted that the first participant can also send the identifier of the leaf node (such as the numerical number of the category corresponding to the leaf node) to the second participant, so that the second participant can make a judgment based on the identifier of the leaf node and perform the correct fusion.

[0181] The first participant decrypts the received fusion using the private key in the homomorphic key pair. If the decryption result t′□t=1, the category of the object to be classified is set to the category of the currently reached leaf node; if the decryption result t′□t≠1, the next leaf node is judged until the category of the object to be classified is determined.

[0182] Exemplarily, an embodiment of the present disclosure also provides a vertical federated learning device.

[0183] FIG8 is a schematic diagram of the structure of a vertical federated learning device provided in one embodiment of the present disclosure. Referring to FIG8 , the vertical federated learning device 700 provided in this embodiment includes:

[0184] A training module 801 is configured to, when training a random forest model based on a common sample participating in longitudinal federated learning, semi-homomorphically encrypt, for a node to be trained on a decision tree included in the random forest model, a first identifier indicating whether the sample exists and a category label of the sample to obtain a first ciphertext;

[0185] The transceiver module 802 is configured to send the first ciphertext to other participants, so that the other participants can calculate a ciphertext calculation result that matches the type of the node to be trained based on the first ciphertext in the ciphertext space;

[0186] The transceiver module 802 is further configured to receive a second ciphertext sent by the other party;

[0187] The training module 801 is also used to train the node to be trained according to the type of the node to be trained and the second ciphertext to configure the category of the node to be trained or configure the corresponding split feature value; the second ciphertext includes the differential privacy processing result obtained by the other participants performing differential privacy processing on the ciphertext calculation result; for the decision tree, the node to be trained is updated for training until the decision tree training end condition is met to obtain the target decision tree; the training of all decision trees in the random forest model is completed to obtain the target random forest model.

[0188] In some possible designs, the training module 801 is specifically configured to:

[0189] If the type of the node to be trained is a leaf node, configuring the type of the node to be trained according to the number of encrypted samples of each category indicated by the second ciphertext;

[0190] If the type of the training node is an internal node, the ciphertext splitting gain of the first candidate splitting feature is calculated based on the first sample splitting result included in the second ciphertext, and the first sample splitting result is the ciphertext result obtained by the other participants splitting the sample based on multiple first random feature values ​​and the first ciphertext; the splitting feature value of the node to be trained is configured by comparing the ciphertext splitting gain of the first candidate splitting feature and the second candidate splitting feature indicated by the second ciphertext.

[0191] In some possible designs, before semi-homomorphic encryption is performed on the first identifier indicating whether the sample exists and the category label of the sample to obtain the first ciphertext, the training module 801 is also used to: obtain the node depth of the node to be trained in the decision tree to which it belongs; if the node depth is equal to the preset maximum tree depth, determine that the node to be trained is a leaf node of the decision tree to which it belongs; if the node depth is less than the preset maximum tree depth, obtain the total number of encrypted samples of the node to be trained returned by the other participants; if the total number of encrypted samples is less than the preset minimum number of split samples, determine that the node to be trained is a leaf node of the decision tree to which it belongs; if the total number of encrypted samples is greater than or equal to the preset minimum number of split samples, determine that the node to be trained is an internal node of the decision tree to which it belongs.

[0192] In some possible designs, the training module 801 is specifically used to: obtain the category label vector of the sample through a logical AND operation based on the first identifier and category label of the sample; and perform semi-homomorphic encryption on the category label vector of the sample based on a semi-homomorphic public key to obtain the first ciphertext.

[0193] In some possible designs, the training module 801 is specifically used to: decrypt the second ciphertext based on the semi-homomorphic private key to obtain the number of encrypted samples corresponding to each category, and configure the category corresponding to the maximum number of encrypted samples as the category of the node to be trained.

[0194] In some possible designs, the training module 801 is specifically used to: decrypt the first sample splitting result based on a semi-homomorphic private key to obtain multiple data items; the multiple data items are used to indicate the number of left and right node encrypted samples obtained by the other participants splitting the sample based on multiple first random eigenvalues; perform weighted calculation based on the multiple data items to obtain the initial splitting gains corresponding to the multiple first random eigenvalues; perform differential privacy processing on the initial splitting gains corresponding to the multiple first random eigenvalues ​​to obtain corresponding ciphertext splitting gains, and determine the first candidate eigenvalue from the multiple first random eigenvalues, and obtain the ciphertext splitting gain of the first candidate eigenvalue.

[0195] In some possible designs, before receiving the second ciphertext sent by the other participants, the training module 801 is also used to: select multiple second random feature values, and split the sample based on the multiple second random feature values, the category label of the sample and the pre-received third ciphertext and add random noise to obtain a second sample splitting result; send the second sample splitting result to the other participants, so that the other participants calculate the ciphertext splitting gain of the second candidate splitting feature based on the second sample splitting result.

[0196] In some possible designs, the transceiver module 802 is also used to: if the type of the node to be trained is a leaf node, send the category identification information of the node to be trained to the other participants; if the type of the node to be trained is an internal node, perform differential privacy processing on the ciphertext splitting gain of the first candidate split eigenvalue to obtain a fourth ciphertext, and send the fourth ciphertext to the other participants, so that the other participants configure the corresponding nodes to be trained in the random forest model corresponding to the other participants based on the fourth ciphertext.

[0197] In some possible designs, it also includes: a classification module 803, which is used to input the object to be classified into the target random forest model. When the object to be classified reaches a leaf node, the first identifier of the object to be classified at the leaf node is semi-homomorphically encrypted to obtain a ciphertext classification result.

[0198] The transceiver module, 802, is also used to send the ciphertext classification result to the other participants; and receive the fusion result fed back by the other participants, wherein the fusion result is obtained by fusing the ciphertext classification result and the second identifier of the object to be classified on the corresponding node in the other participants.

[0199] The classification module 803 is also used to determine that the category of the object to be classified is the category of the leaf node currently reached if the fusion result meets the preset conditions; if the fusion result does not meet the preset conditions, continue to judge the next leaf node reached by the object to be classified until the category of the object to be classified is determined.

[0200] The device provided in this embodiment can be used to implement the steps performed by the first participant in the aforementioned method embodiment. Its implementation principle and technical effects are similar. Please refer to the detailed description of the aforementioned method embodiment. For the sake of brevity, it will not be repeated here.

[0201] FIG9 is a schematic diagram of the structure of an electronic device provided in an embodiment of the present disclosure. Referring to FIG9 , the electronic device 900 provided in this embodiment includes:

[0202] Transceiver module 901 is configured to receive, when training a random forest model based on a common sample participating in vertical federated learning, a first ciphertext sent by another participant for a node to be trained on a decision tree included in the random forest model, where the first ciphertext is obtained by semi-homomorphically encrypting a first identifier indicating whether the sample exists and a category label of the sample by the other participant;

[0203] A training module 902 is configured to calculate, in a ciphertext space, based on the first ciphertext, a ciphertext calculation result that matches the type of the node to be trained, and perform differential privacy processing on the ciphertext calculation result to obtain a second ciphertext;

[0204] The transceiver module 901 is further configured to send the second ciphertext to the other participants, so that the other participants perform random forest model training based on the second ciphertext; and receive feedback information sent by the other participants;

[0205] The training module 902 is used to train the nodes to be trained based on the feedback information and the second ciphertext to configure the categories of the nodes to be trained or configure corresponding split feature values; for the decision tree, the nodes to be trained are updated for training until the decision tree training end condition is met to obtain a target decision tree; all decision trees in the random forest model are trained to obtain a target random forest model.

[0206] In some possible designs, the training module 902 is specifically used to, if the type of the node to be trained is a leaf node, calculate the number of encrypted samples of each category based on the first ciphertext and the second identifier indicating whether the sample exists; if the type of the node to be trained is an internal node, split the sample according to the first ciphertext and multiple first random eigenvalues ​​to obtain a first sample splitting result; and, obtain the initial splitting gain of multiple second random eigenvalues ​​according to the second sample splitting result sent by the other participants; the second sample splitting result is the ciphertext result obtained by the other participants by splitting the sample and adding noise according to the pre-received third ciphertext, multiple second random eigenvalues ​​and the category label of the sample; the third ciphertext is obtained by semi-homomorphic encryption of the second identifier.

[0207] In some possible designs, the training module 902 is specifically configured to configure the category of the node to be trained according to the category identification information indicated by the feedback information if the type of the node to be trained is a leaf node.

[0208] In some possible designs, if the type of the node to be trained is an internal node, the feedback information includes a fourth ciphertext, and the fourth ciphertext is the ciphertext splitting gain of the first candidate splitting feature calculated by the other participants based on the first sample splitting result and obtained through differential privacy processing; the training module 902 is specifically used to configure the splitting feature value of the node to be trained by comparing the ciphertext splitting gain of the first candidate splitting feature and the second candidate splitting feature indicated by the second ciphertext.

[0209] In some possible designs, the transceiver module 901 is further configured to receive a ciphertext classification result sent by the other participant; the ciphertext classification result is obtained by performing semi-homomorphic encryption on the first identifier of the object to be classified at the leaf node reached after the object to be classified is input into the random forest model corresponding to the other participant;

[0210] The method further includes: a classification module 903 for inputting the object to be classified into the target random forest model to obtain a second identifier of the object to be classified on the corresponding node; and fusing the ciphertext classification result and the second identifier in the ciphertext space to obtain a fusion result;

[0211] The transceiver module 901 is further configured to send the fusion result to the other participants, so that the other participants determine the category of the object to be classified according to the fusion result.

[0212] The device provided in this embodiment can be used to implement the steps performed by the second participant in the aforementioned method embodiment. Its implementation principle and technical effects are similar. Please refer to the detailed description of the aforementioned method embodiment. For the sake of brevity, it will not be repeated here.

[0213] Exemplarily, the present disclosure provides an electronic device comprising: one or more processors; a memory; and one or more computer programs; wherein the one or more computer programs are stored in the memory; when the one or more processors execute the one or more computer programs, the electronic device implements the vertical federated learning method executed by the first party or the second in the foregoing embodiments.

[0214] Exemplarily, the present disclosure provides a chip system, which is applied to an electronic device including a memory and a sensor; the chip system includes: a processor; when the processor executes the vertical federated learning method executed by the first participant or the second in the above embodiment.

[0215] Exemplarily, the present disclosure provides a computer-readable storage medium having a computer program stored thereon. When the computer program is executed by a processor in an electronic device, it implements the vertical federated learning method performed by the first participant or the second in the above embodiment.

[0216] Exemplarily, the present disclosure provides a computer program product, which, when executed on a computer, enables the computer to execute the vertical federated learning method executed by the first participant or the second participant in the foregoing embodiment.

[0217] In the above embodiments, all or part of the functions can be implemented by software, hardware, or a combination of software and hardware. When implemented using software, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the process or function according to the embodiment of the present disclosure is generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that includes one or more available media integrated. The available medium can be a magnetic medium (for example, a floppy disk, a hard disk, a tape), an optical medium (for example, a DVD), or a semiconductor medium (for example, a solid state drive (SSD)), etc.

[0218] It should be noted that, in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any actual relationship or order between these entities or operations. Moreover, the terms "comprises," "comprising," or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, article, or device comprising a series of elements includes not only those elements, but also other elements not explicitly listed, or elements inherent to such process, method, article, or device. In the absence of further limitations, an element defined by the phrase "comprising a ..." does not exclude the presence of other identical elements in the process, method, article, or device comprising the element.

[0219] The foregoing description is intended only to provide specific embodiments of the present disclosure, intended to enable those skilled in the art to understand and implement the present disclosure. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present disclosure. Therefore, the present disclosure is not intended to be limited to the embodiments described herein, but rather to be construed in the broadest manner consistent with the principles and novel features disclosed herein.

Claims

1. A vertical federated learning method, comprising: When training a random forest model based on a common sample participating in vertical federated learning, for a node to be trained on a decision tree included in the random forest model, a first identifier indicating whether the sample exists and a category label of the sample are semi-homomorphically encrypted to obtain a first ciphertext, and the first ciphertext is sent to other participants, so that the other participants calculate a ciphertext calculation result matching the type of the node to be trained based on the first ciphertext in the ciphertext space; Receiving a second ciphertext sent by the other participant, and training the node to be trained according to the type of the node to be trained and the second ciphertext to configure the category of the node to be trained or configure a corresponding split feature value; The second ciphertext includes a result obtained by the other participants performing differential privacy processing on the ciphertext calculation result; For the decision tree, the nodes to be trained are updated for training until the decision tree training end condition is met to obtain a target decision tree; and the training of all decision trees in the random forest model is completed to obtain a target random forest model.

2. According to the method of claim 1, the step of training the node to be trained according to the type of the node to be trained and the second ciphertext to configure the category of the node to be trained or to configure a corresponding split feature value comprises: If the type of the node to be trained is a leaf node, configuring the type of the node to be trained according to the number of encrypted samples of each category indicated by the second ciphertext; If the type of the training node is an internal node, the ciphertext splitting gain of the first candidate splitting feature is calculated according to the first sample splitting result included in the second ciphertext, where the first sample splitting result is the ciphertext result obtained by the other participants splitting the sample based on the multiple first random feature values ​​and the first ciphertext; The split feature value of the node to be trained is configured by comparing the ciphertext splitting gain of the first candidate split feature and the second candidate split feature indicated by the second ciphertext.

3. According to the method of claim 1, before semi-homomorphic encryption is performed on the first identifier indicating whether the sample exists and the category label of the sample to obtain the first ciphertext, the method further comprises: Obtaining the node depth of the node to be trained in the decision tree to which it belongs; If the node depth is equal to the preset maximum tree depth, determining that the node to be trained is a leaf node of the decision tree to which it belongs; If the node depth is less than the preset maximum tree depth, obtaining the total number of encrypted samples of the nodes to be trained returned by the other participants; If the total number of encrypted samples is less than the preset minimum number of split samples, the node to be trained is determined to be a leaf node of the decision tree to which it belongs; if the total number of encrypted samples is greater than or equal to the preset minimum number of split samples, the node to be trained is determined to be an internal node of the decision tree to which it belongs.

4. According to the method of claim 1, the step of performing semi-homomorphic encryption on the first identifier indicating whether the sample exists and the category label of the sample to obtain the first ciphertext comprises: Obtaining a category label vector of the sample through a logical AND operation according to the first identifier and the category label of the sample; The category label vector of the sample is semi-homomorphically encrypted based on a semi-homomorphic public key to obtain the first ciphertext.

5. The method according to claim 2, wherein configuring the category of the node to be trained according to the number of encrypted samples of each category indicated by the second ciphertext comprises: The second ciphertext is decrypted based on the semi-homomorphic private key to obtain the number of encrypted samples corresponding to each category, and the category corresponding to the maximum number of encrypted samples is configured as the category of the node to be trained.

6. The method according to claim 2, wherein the step of calculating the ciphertext splitting gain of the first candidate splitting feature according to the first sample splitting result included in the second ciphertext comprises: Decrypting the first sample splitting result based on the semi-homomorphic private key to obtain multiple data items; The multiple data items are used to indicate the number of left and right node encrypted samples obtained by the other participants splitting the samples based on the multiple first random feature values; Performing weighted calculation according to the multiple data items to obtain initial splitting gains corresponding to the multiple first random eigenvalues ​​respectively; Differential privacy processing is performed on the initial splitting gains corresponding to the multiple first random eigenvalues ​​to obtain corresponding ciphertext splitting gains, and the first candidate eigenvalue is determined from the multiple first random eigenvalues ​​to obtain the ciphertext splitting gain of the first candidate eigenvalue.

7. The method according to claim 2, wherein the type of the node to be trained is an internal node, and before receiving the second ciphertext sent by the other party, the method further comprises: Selecting a plurality of second random feature values, splitting the sample based on the plurality of second random feature values, the category label of the sample and the pre-received third ciphertext and adding random noise to obtain a second sample splitting result; The second sample splitting result is sent to the other participants, so that the other participants calculate the ciphertext splitting gain of the second candidate splitting feature based on the second sample splitting result.

8. The method according to claim 2, further comprising: If the type of the node to be trained is a leaf node, sending the category identification information of the node to be trained to the other participants; If the type of the node to be trained is an internal node, differential privacy processing is performed on the ciphertext splitting gain of the first candidate split eigenvalue to obtain a fourth ciphertext, and the fourth ciphertext is sent to the other participants, so that the other participants configure the corresponding nodes to be trained in the random forest model corresponding to the other participants based on the fourth ciphertext.

9. The method according to claim 1, further comprising: Inputting the object to be classified into the target random forest model, when the object to be classified reaches a leaf node, performing semi-homomorphic encryption on the first identifier of the object to be classified at the leaf node to obtain a ciphertext classification result; Sending the ciphertext classification result to the other participants; receiving a fusion result fed back by the other participants, wherein the fusion result is obtained by fusing the ciphertext classification result and the second identifier of the object to be classified on the corresponding node of the other participants; If the fusion result meets the preset condition, the category of the object to be classified is determined to be the category of the currently reached leaf node; if the fusion result does not meet the preset condition, Continue to determine the next leaf node that the object to be classified reaches, until the category of the object to be classified is determined.

10. A vertical federated learning method, comprising: When training a random forest model based on a common sample participating in vertical federated learning, for a node to be trained on a decision tree included in the random forest model, receiving a first ciphertext sent by another participant, where the first ciphertext is obtained by semi-homomorphic encryption by the other participant of a first identifier indicating whether the sample exists and a category label of the sample; Obtaining a ciphertext calculation result matching the type of the node to be trained according to the first ciphertext in the ciphertext space, performing differential privacy processing based on the ciphertext calculation result to obtain a second ciphertext, and sending the second ciphertext to the other participants, so that the other participants perform random forest model training based on the second ciphertext; receiving feedback information sent by the other participants, and configuring the category of the node to be trained or configuring a corresponding splitting feature value according to the feedback information and the second ciphertext; For the decision tree, the nodes to be trained are updated for training until the decision tree training end condition is met to obtain a target decision tree; and the training of all decision trees in the random forest model is completed to obtain a target random forest model.

11. The method according to claim 10, wherein the step of calculating in the ciphertext space according to the first ciphertext to obtain a ciphertext calculation result that matches the type of the node to be trained comprises: If the type of the node to be trained is a leaf node, the number of encrypted samples of each category is calculated according to the first ciphertext and a second identifier indicating whether the sample exists; If the type of the node to be trained is an internal node, the sample is split according to the first ciphertext and the multiple first random eigenvalues ​​to obtain a first sample splitting result; and the initial splitting gain of the multiple second random eigenvalues ​​is obtained according to the second sample splitting result sent by the other participants; the second sample splitting result is the ciphertext result obtained by the other participants by splitting the sample according to the pre-received third ciphertext, the multiple second random eigenvalues ​​and the category label of the sample and adding noise; The third ciphertext is obtained by performing semi-homomorphic encryption on the second identifier.

12. The method according to claim 11, wherein configuring the category of the node to be trained or configuring the corresponding split feature value according to the feedback information and the second ciphertext comprises: If the type of the node to be trained is a leaf node, the category of the node to be trained is configured according to the category identification information indicated by the feedback information.

13. The method according to claim 11, if the type of the node to be trained is an internal node, the feedback information includes a fourth ciphertext, and the fourth ciphertext is the ciphertext splitting gain of the first candidate splitting feature calculated by the other participants based on the first sample splitting result and obtained through differential privacy processing; The configuring the category of the node to be trained or configuring the corresponding split feature value according to the feedback information and the second ciphertext includes: The split feature value of the node to be trained is configured by comparing the ciphertext splitting gain of the first candidate split feature and the second candidate split feature indicated by the second ciphertext.

14. The method according to claim 10, further comprising: Receiving the ciphertext classification results sent by the other participants; The ciphertext classification result is obtained by performing semi-homomorphic encryption on the first identifier of the object to be classified on the leaf node reached after the object to be classified is input into the random forest model corresponding to the other participants; Inputting the object to be classified into the target random forest model to obtain a second identifier of the object to be classified on the corresponding node; Fusing the ciphertext classification result and the second identifier in the ciphertext space to obtain a fusion result; The fusion result is sent to the other participants, so that the other participants determine the category of the object to be classified according to the fusion result.

15. A vertical federated learning device, comprising: A training module, for training a random forest model based on a common sample participating in longitudinal federated learning, for a node to be trained on a decision tree included in the random forest model, performing semi-homomorphic encryption on a first identifier indicating whether the sample exists and a category label of the sample to obtain a first ciphertext; A transceiver module is used to send the first ciphertext to other participants, so that the other participants can calculate the node to be trained based on the first ciphertext in the ciphertext space. The ciphertext calculation results of the matching types; The transceiver module is further used to receive the second ciphertext sent by the other party; The training module is further used to train the node to be trained according to the type of the node to be trained and the second ciphertext to configure the category of the node to be trained or configure a corresponding split feature value; The second ciphertext includes the differential privacy processing result obtained by the other participants performing differential privacy processing on the ciphertext calculation result; for the decision tree, the nodes to be trained are updated for training until the decision tree training end condition is met to obtain the target decision tree; the training of all decision trees in the random forest model is completed to obtain the target random forest model.

16. A vertical federated learning device, comprising: A transceiver module, used for, when training a random forest model based on a common sample participating in vertical federated learning, receiving a first ciphertext sent by other participants for a node to be trained on a decision tree included in the random forest model, wherein the first ciphertext is obtained by semi-homomorphic encryption by the other participants of a first identifier indicating whether the sample exists and a category label of the sample; A training module, configured to obtain a ciphertext calculation result matching the type of the node to be trained in the ciphertext space according to the first ciphertext, and perform differential privacy processing based on the ciphertext calculation result to obtain a second ciphertext; The transceiver module is further configured to send the second ciphertext to the other participants, so that the other participants perform random forest model training based on the second ciphertext; and receive feedback information sent by the other participants; The training module is used to train the node to be trained according to the feedback information and the second ciphertext to configure the category of the node to be trained or configure the corresponding split feature value; For the decision tree, the nodes to be trained are updated for training until the decision tree training end condition is met to obtain a target decision tree; and the training of all decision trees in the random forest model is completed to obtain a target random forest model.

17. An electronic device comprising: Memory and processor; The memory is configured to: store computer program instructions; The processor is configured to: execute the computer program instructions so that the electronic device implements the vertical federated learning method as described in any one of claims 1 to 9, or any one of claims 10 to 14.

18. A readable storage medium, comprising: Computer program instructions; The computer program instructions are executed by at least one processor of an electronic device, so that the electronic device implements the vertical federated learning method as described in any one of claims 1 to 9, or any one of 10 to 14.

19. A computer program product, when the computer program product is executed by an electronic device, enables the electronic device to implement the vertical federated learning method as described in any one of claims 1 to 9, or any one of claims 10 to 14.

Images