Vertical associative learning method, device, electronic device, readable storage medium, and computer program

Abstract

The present disclosure relates to a method, device, electronic device, and readable storage medium for vertical federated learning, in which multiple participants train a random forest model based on common samples participating in vertical federated learning. When the method is used, the participants use partially homomorphic encryption technology to encrypt a node waiting to be trained in a decision tree included in the random forest model, indicating whether the training sample exists in the identifier of the node waiting to be trained, and then send the encrypted result to the other party. The other participants calculate the required ciphertext in the ciphertext space, and process the ciphertext calculation result using differential privacy technology before feeding it back to the other party. This ensures the security of the distributed ciphertext calculation result and prevents the ciphertext calculation result from being used to obtain related information about the sample, which is advantageous to improving the security of the sample.

Description

[Technical Field]

[0001] [CROSS-REFERENCE TO RELATED APPLICATIONS] This application claims priority to Chinese Patent Application No. 202310282556.3, filed on March 21, 2023, entitled "Vertical associative learning method, device, electronic device and readable storage medium," the entire contents of which are incorporated herein by reference.

[0002] [Technical content] The present disclosure relates to the field of data processing technology, and in particular to a method, device, electronic device, and readable storage medium for longitudinal association learning. [Background technology]

[0003] Federated learning is a popular machine learning technique that solves the problem of jointly training a global model on a virtual "aggregated" dataset among multiple participants, while ensuring the security of each participant's data. Federated learning can be further divided into horizontal federation, vertical federation, and federated transition. Vertical federation is a technique where the sample data in the datasets owned by multiple participants overlap, but the features are complementary. This makes vertical federation extremely versatile, as it can be applied to situations where multiple features serve the same business label. Random forests are currently one of the mainstream machine learning algorithms and are widely used in classification scenarios. Summary of the Invention [Problem to be solved by the invention]

[0004] Currently, random forest models are trained using vertical federated learning technology, which typically involves using a trusted third-party server to collect information such as the feature data of each participant's sample and intermediate values ​​generated during the training process. The third-party server then iteratively updates the global model, and then sends the determined global model parameters to each participant. This method of training a random forest model using vertical federated learning has low sample security. [Means for solving the problem]

[0005] To solve the above technical problems, the present disclosure provides a vertical associative learning method, device, electronic device, and readable storage medium.

[0006] In a first aspect, the present disclosure provides a longitudinal associative learning method, the method comprising: When training a random forest model based on common samples participating in vertical federated learning, for a node waiting to be trained in a decision tree included in the random forest model, partially homomorphically encrypting a first identifier indicating whether the sample exists and a class label of the sample to obtain a first ciphertext, and transmitting the first ciphertext to other participants so that the other participants can calculate a ciphertext matching the type of the node waiting to be trained based on the first ciphertext in a ciphertext space; receiving a second ciphertext sent from the other participant, and training the node waiting to be trained according to the type of the node waiting to be trained and the second ciphertext to set a class of the node waiting to be trained or a corresponding split feature value, wherein the second ciphertext includes a result of differential privacy processing performed by the other participant on a calculation result of the ciphertext; The method includes updating and training the training waiting node for the decision tree until a training termination condition for the decision tree is met and a target decision tree is obtained, thereby completing training of all decision trees in the random forest model and obtaining a target random forest model.

[0007] In a second aspect, the present disclosure provides a longitudinal associative learning method, the method comprising: When training a random forest model based on common samples participating in vertical federated learning, receiving a first ciphertext sent by another participant to a node waiting to be trained in a decision tree included in the random forest model, wherein the first ciphertext is obtained by the other participant partially homomorphically encrypting a first identifier indicating whether the sample exists and a class label of the sample; Calculating a ciphertext that matches the type of the node waiting to be trained according to the first ciphertext in a ciphertext space, performing differential privacy processing based on the ciphertext calculation result to obtain a second ciphertext, and sending the second ciphertext to the other participants so that the other participants train a random forest model based on the second ciphertext; receiving feedback information transmitted from the other participants, and setting a class or a corresponding split feature value of the node waiting to be trained according to the feedback information and the second ciphertext; The method includes updating and training the training waiting node for the decision tree until a training termination condition for the decision tree is met and a target decision tree is obtained, thereby completing training of all decision trees in the random forest model and obtaining a target random forest model.

[0008] In a third aspect, the present disclosure provides a longitudinal associative learning apparatus, the apparatus comprising: a training module for, when training a random forest model based on common samples participating in vertical federated learning, partially homomorphically encrypting a first identifier indicating whether the sample exists and a class label of the sample for a node waiting to be trained in a decision tree included in the random forest model to obtain a first ciphertext; a transceiver module for transmitting the first ciphertext to other participants so that the other participants can calculate a calculation result of a ciphertext that matches the type of the node waiting to be trained based on the first ciphertext in a ciphertext space; the transceiver module for receiving a second ciphertext sent from the other participant; The training module trains the nodes waiting to be trained according to the type of the nodes waiting to be trained and the second ciphertext to set the class of the nodes waiting to be trained or the corresponding split feature value, the second ciphertext including the result of differential privacy processing performed by the other participant on the calculation result of the ciphertext, and updates and trains the nodes waiting to be trained for the decision tree until a training termination condition for the decision tree is met and a target decision tree is obtained, thereby completing training of all decision trees in the random forest model to obtain a target random forest model.

[0009] In a fourth aspect, the present disclosure provides a longitudinal associative learning apparatus, the apparatus comprising: a transceiver module for receiving a first ciphertext sent by another participant to a node waiting to be trained in a decision tree included in the random forest model, when the random forest model is trained based on a common sample participating in vertical federated learning, the first ciphertext being obtained by the other participant performing partially homomorphic encryption on a first identifier indicating whether the sample exists and a class label of the sample; a training module for calculating a calculation result of a ciphertext matching a type of a node waiting to be trained in a ciphertext space according to the first ciphertext, and performing differential privacy processing based on the calculation result of the ciphertext to obtain a second ciphertext; the transceiver module for transmitting the second ciphertext to the other participants so that the other participants train a random forest model based on the second ciphertext and receiving feedback information transmitted from the other participants; The training module trains the nodes waiting to be trained according to the feedback information and the second ciphertext to set the classes or corresponding split feature values ​​of the nodes waiting to be trained, updates and trains the nodes waiting to be trained for the decision trees until a training termination condition for the decision trees is met and a target decision tree is obtained, and completes training of all decision trees in the random forest model to obtain a target random forest model.

[0010] According to a fifth aspect, the present disclosure provides an electronic device comprising a memory and a processor, the memory configured to store computer program instructions; The processor is configured to execute the computer program instructions to cause the electronic device to implement the longitudinal associative learning method according to the first aspect or the second aspect.

[0011] According to a fourth aspect, the present disclosure provides a readable storage medium comprising computer program instructions that, when executed by at least one processor of an electronic device, cause the electronic device to implement the longitudinal associative learning method according to the first or second aspect.

[0012] According to a fifth aspect, the present disclosure provides a computer program product that, when executed by at least one processor of an electronic device, causes the electronic device to implement the longitudinal associative learning method described in the first or second aspect. [Brief explanation of the drawings]

[0013] The drawings herein are incorporated into and constitute a part of this specification, and are illustrative of the embodiments of the present disclosure, and together with the specification serve to explain the principles of the present disclosure.

[0014] In order to more clearly explain the technical solutions in the embodiments of the present disclosure or the prior art, the following briefly introduces the drawings that need to be used in the description of the embodiments or the prior art, and it is obvious that those skilled in the art can obtain other drawings based on these drawings without any creative labor.

[0015] [Figure 1] FIG. 1 is a schematic diagram of an application scenario of the vertical association learning method provided by the present disclosure; [Figure 2] FIG. 10 is a feature data distribution diagram of a sample according to an embodiment of the present disclosure. [Figure 3] FIG. 1 is an overall frame diagram of a vertical associative learning method according to an embodiment of the present disclosure. [Figure 4] 1 is a flowchart of a vertical associative learning method according to an embodiment of the present disclosure. [Figure 5] 10 is a flowchart of a vertical associative learning method according to another embodiment of the present disclosure. [Figure 6] 10 is a flowchart of a vertical associative learning method according to another embodiment of the present disclosure. [Figure 7] FIG. 1 is a structural schematic diagram of a target random forest model according to an embodiment of the present disclosure. [Figure 8] 1 is a structural schematic diagram of a vertical associative learning device according to an embodiment of the present disclosure; [Figure 9] FIG. 10 is a structural schematic diagram of a vertical associative learning device according to another embodiment of the present disclosure. DETAILED DESCRIPTION OF THE INVENTION

[0016] In order to make the above objectives, features and advantages of the present disclosure more clearly understood, the technical solution of the present disclosure will be further described below. It should be noted that, where not inconsistent, the embodiments and features in the embodiments of the present disclosure can be combined with each other.

[0017] In the following description, numerous specific details are set forth to provide a thorough understanding of the present disclosure; however, the present disclosure may be embodied in other forms different from those described herein, and the embodiments in this specification are merely some embodiments of the present disclosure, but not all embodiments.

[0018] First, technical terms used in the present disclosure will be explained in detail.

[0019] 1. Random forest model A random forest model is a classifier that includes multiple decision trees, each of which consists of nodes and directed edges, where the nodes include leaf nodes and internal nodes, the parameters corresponding to the leaf nodes are classes that indicate the classification results of the samples / objects to be classified that arrive at the node, and the parameters corresponding to the internal nodes are split feature values ​​that are used to differentiate the training samples / objects to be processed that arrive at the node into connected left or right subnodes.

[0020] 2. Partially homomorphic encryption For any participant, say participant G, we have homomorphic keys pk and sk, encryption function E, and decryption function D. For any data x and y, the following properties are satisfied. a, homomorphic addition:

number

number

number

[0021] 3. Differential Privacy For any data x, given a privacy budget ε and a sensitivity s, it is possible to obtain differentially private data that satisfies the privacy budget. This can be expressed as follows using the following formula:

number

[0022] The present disclosure provides a method, device, electronic device, and readable storage medium for vertical federated learning, in which multiple participants train a random forest model based on common samples participating in vertical federated learning. For nodes waiting to be trained in a decision tree included in the random forest model, the multiple participants use partially homomorphic encryption technology to indicate whether the training sample exists in the identifier of the node waiting to be trained, encrypt the training sample, and then send it to the other party. The other participants each calculate the required ciphertext calculation results in the ciphertext space, and process the ciphertext calculation results using differential privacy technology before feeding them back to the other party. This ensures the security of the distributed ciphertext calculation results and prevents the ciphertext calculation results from being used to obtain related information about the samples, which is advantageous to improving the security of the samples.

[0023] Hereinafter, the vertical associative learning method according to the present disclosure will be described in detail with reference to drawings and scenes through several examples.

[0024] 1 is a schematic diagram of an application scenario of a vertical federated learning method according to an embodiment of the present disclosure. As shown in FIG. 1, the scene involves two participants, a first participant 101 and a second participant 102, respectively. The two participants may be servers, service clusters, cloud servers, cloud platforms, etc., and the present disclosure does not limit the type of participants.

[0025] FIG. 2 is a diagram illustrating the data distribution of samples (also referred to as training samples) for a first participant 101 and a second participant 102. As shown in FIG. 2, the first participant 101 has m1-dimensional first feature data and class labels of the sample, and the second participant 102 has m2-dimensional second feature data of the sample. Here, the first participant 101 and the second participant 102 each possess different feature data, but possess the same sample identity document (ID) for the same training sample. In the present disclosure, samples are typically structured data and may be, but are not limited to, image, audio, text, and other types of samples.

[0026] Here, the first participant 101 and the second participant 102 can train a global random forest model based on common samples through longitudinal federated learning and perform classification tasks using the trained random forest model.

[0027] In the model training stage, when calculating the number of samples present in a node waiting to be trained or the split gain of each feature value for a node waiting to be trained in a decision tree included in a random forest model, the first participant 101 and the second participant 102 partially homomorphically encrypt an identifier indicating whether the current node is present in the sample and transmit it to the other participant, who then processes the identifier in the ciphertext space to obtain the desired ciphertext calculation result. Before distributing the ciphertext calculation result to the other participant, the ciphertext calculation result is processed using differential privacy technology to ensure the security of the distributed ciphertext calculation result.

[0028] In the prediction stage, for the object to be classified, the first participant 101 and the second participant 102 use the random forest model they have trained to output classification results based on different feature values ​​of the object to be classified, and then encrypt the classification results using partially homomorphic encryption technology and send them to the object, ensuring the security of the object to be classified.

[0029] In the scenario shown in Figure 1, a global random forest model is trained using longitudinal federated learning, and the third-party server does not need to collect sample information owned by the first participant 101 and the second participant 102 respectively and intermediate information generated during the training process, which can effectively protect the security of the sample and training-related data.

[0030] 3 is a schematic diagram of an overall frame of a vertical federated learning method according to an embodiment of the present disclosure. As shown in FIG. 3, when training begins, the first and second participants each determine whether the node type is a leaf node or an internal node based on the depth of the node waiting for training, or alternately determine the number of training samples present in the current node. Then, based on the depth of the node waiting for training and the number of training samples present in the node waiting for training, determine whether the node type is a leaf node or an internal node. If the node type of the node waiting for training is a leaf node, the first and second participants alternately calculate the class of the node waiting for training using partially homomorphic encryption and differential privacy techniques. If the node type of the node waiting for training is an internal node, the first and second participants alternately calculate the partition feature value of the node type of the node waiting for training using partially homomorphic encryption and differential privacy techniques. Then, if the node waiting to be trained is an internal node, the sample is classified into its left and right child nodes according to the calculated split feature value, and the node waiting to be trained is updated to any child node and training is performed. If the node waiting to be trained is a leaf node, the node waiting to be trained is updated to any child node that is its sibling node and training is performed, and this is continued until node training for all decision trees included in the random forest model is completed.

[0031] 4 is a flowchart of a vertical associative learning method according to an embodiment of the present disclosure. In the following embodiment, a first participant and a second participant perform vertical associative learning, where the first participant owns the class label of the sample. As shown in FIG. 4, the method of this embodiment includes the following steps:

[0032] S401: When training a random forest model based on common samples participating in vertical federated learning, a first participant obtains a first ciphertext by partially homomorphically encrypting a first identifier indicating whether the sample exists and a class label of the sample for a node waiting to be trained in a decision tree included in the random forest model.

[0033] The number of common samples participating in the longitudinal federated learning may be multiple, and the first participant owns the first feature data and class label of the sample. The samples may come from any field, any content, or any storage format, and the present disclosure is not limited in any way.

[0034] S402, the first participant transmits a first ciphertext.

[0035] The first identifier is used to indicate whether the sample is present in the training waiting node of the random forest model of the first participant. During the model training process, as the training waiting node is constantly updated, the sample is constantly split until it reaches a leaf node, and the first identifier of the sample is also constantly updated. For example, if the value of the first identifier is 0, it indicates that the sample does not exist in the current training waiting node, and if the value of the first identifier is 1, it indicates that the sample exists in the current training waiting node. Node 1 is the root node, and nodes 2 and 3 are the left and right child nodes of node 1. When a sample is input to node 1, the values ​​of the first identifier of all samples are 1. If a splitting feature value exists in the root node, based on the first feature data of the sample and the splitting feature value of node 1, when the sample is classified into node 2 or node 3, the value of the first identifier in node 2 of the sample classified into node 2 is 1, the value of the first identifier in node 3 of the sample classified into node 3 is 1, the first identifier in node 2 of the sample classified into node 3 is 0, and the first identifier in node 3 of the sample classified into node 2 is 0.

[0036] The class label of a sample is used to indicate the class to which the sample belongs. Each common sample participating in longitudinal associative learning can belong to one of multiple classes.

[0037] In one possible embodiment, for the node waiting to train, the first participant can generate a classmark vector for each sample based on the class label of the sample and the first identifier, where the classmark vector can indicate whether the sample exists in the node waiting to train and can also indicate the class of the sample; then, the first participant encrypts the classmark vector using its homomorphic encryption key to obtain an encrypted classmark vector, and the first ciphertext includes the encrypted classmark vector corresponding to each sample.

[0038] In addition, the first participant stitches the encrypted classmark vectors corresponding to each sample into one vector and sends it to the second participant. The second participant can then calculate the vector obtained by stitching to obtain the calculation result of the ciphertext that matches the type of the current node waiting to be trained, thereby reducing the amount of data processing by the second participant and improving training efficiency.

[0039] The type of the node waiting to train may be an internal node of the decision tree to which it belongs, or it may be a leaf node of the decision tree to which it belongs. For different types, the second participant needs to obtain a calculation result of a different ciphertext based on the first ciphertext and feed it back to the first participant. Here, the first participant can determine the type of the current node based on the depth of the node waiting to train in the decision tree and the number of samples present in the node waiting to train.

[0040] For example, before training, the maximum tree depth and minimum number of split samples of the decision tree can be initialized. During training, the first participant can determine the type of the node waiting to be trained based on the maximum tree depth and minimum number of split samples of the initialization, which can include the following situations:

[0041] 1. If the depth of a node waiting to be trained is greater than or equal to the maximum tree depth of the decision tree, the node waiting to be trained is a leaf node; 2. If the depth of a node waiting to be trained is less than the maximum tree depth of the decision tree and the number of training samples in the node waiting to be trained is less than the minimum number of split samples, the node waiting to be trained is a leaf node; and if the depth of a node waiting to be trained is less than the maximum tree depth of the decision tree and the number of training samples in the node waiting to be trained is greater than or equal to the minimum number of split samples, the node waiting to be trained is an internal node.

[0042] S403: The second participant calculates a calculation result of a ciphertext that matches the type of the node waiting to be trained according to the first ciphertext in the ciphertext space, and performs differential privacy processing based on the calculation result of the ciphertext to obtain a second ciphertext.

[0043] The first participant can send node type information to the second participant, indicating that the type of the node waiting to be trained is a leaf node or an internal node, so that the second participant can obtain the correct ciphertext calculation result.

[0044] 1. If the type of the node waiting to train is a leaf node, the second participant calculates the number of encrypted samples of each obtained class based on the first ciphertext and the second identifier indicating whether a sample exists, where the number of encrypted samples of each class is the calculation result of the ciphertext. The number of encrypted samples of a certain class is the number of samples for that class that simultaneously exist in the nodes waiting to train of the first participant and the second participant. For example, if nodes a1 and a2 are nodes waiting to train of the first participant and the second participant, respectively, and the random forest model includes five classes, the number of samples simultaneously existing in nodes a1 and a2 for these five classes is calculated.

[0045] In one possible embodiment, when the first ciphertext includes an encrypted classmark vector corresponding to each sample, the second participant homomorphically multiplies each element in the encrypted classmark vector of the same sample by a second identifier of the sample, where the second identifier is used to indicate whether the sample exists in the corresponding training node of the second participant, and then, for each class, performs homomorphic addition on the elements in each sample corresponding to the class to obtain an initial number of encrypted samples corresponding to each class, where the initial number of encrypted samples is the number of samples that have not undergone differential privacy processing. Next, differential privacy noise is generated based on the preset sensitivity and differential privacy budget, and the differential privacy noise is further added to the initial encrypted samples using homomorphic addition to obtain an encrypted number of samples corresponding to each class included in the second ciphertext. The second participant obtains the encrypted number of samples corresponding to each class and performs calculations in the ciphertext space, and the calculation results of the ciphertext that the second participant sends to the first participant are perturbed by differential privacy processing, thereby preventing the first participant from obtaining the plaintext number of each class, thereby protecting the security of the training samples.

[0046] Second, if the type of the node waiting to train is an internal node, the second participant divides the samples based on the first ciphertext and the plurality of first random feature values ​​to obtain a first sample division result, and obtains a ciphertext division gain of the second candidate division feature value based on the second sample division result sent by the first participant, where the second sample division result is a ciphertext result obtained by dividing the samples and adding noise based on the third ciphertext, the plurality of second random feature values, and the class labels of the samples previously received by the first participant, and the third ciphertext is an encrypted mark vector obtained by the second participant by performing partially homomorphic encryption on the second identifier. The second participant uses the private key in the partially homomorphic key pair to decrypt the second sample division result to obtain multiple data items, calculates the initial division gain of each second random feature for each data item using a weighted method, and further adds noise to the initial division gain of each second random feature value using differential privacy technology, and then selects an optimal local division gain, and the second random feature value corresponding to the optimal local division gain is a second candidate division feature. If the type of the node waiting to train is a second ciphertext, the first sample division result and the ciphertext division gain of the second candidate division feature value are included.

[0047] S404, the second participant transmits the second ciphertext to the first participant.

[0048] S405: The first participant trains the nodes waiting to be trained according to the types of the nodes waiting to be trained and the second ciphertext to set the classes or corresponding split feature values ​​of the nodes waiting to be trained.

[0049] This step sets a waiting-to-train node in the decision tree included in the random forest model corresponding to the first participant.

[0050] If the type of the node waiting to train is a leaf node, the first participant decrypts the second ciphertext based on the partially homomorphic private key to obtain the number of encrypted samples corresponding to each class, and sets the class corresponding to the maximum number of encrypted samples as the class of the node waiting to train.

[0051] If the type of the node waiting to train is an internal node, the first participant calculates a ciphertext splitting gain of a first candidate splitting feature based on a first sample splitting result included in the second ciphertext, where the first sample splitting result is a ciphertext result obtained by the second participant splitting the sample based on a plurality of first random feature values ​​and the first ciphertext, and sets a splitting feature value of the node waiting to train by comparing the magnitude of the ciphertext splitting gain of the first candidate splitting feature and the second candidate splitting feature indicated by the second ciphertext.

[0052] S406, the second participant sets the class of the corresponding node waiting to be trained or the corresponding split feature value according to the feedback information and the second ciphertext sent by the first participant.

[0053] This step sets a waiting-to-train node in the decision tree included in the random forest model corresponding to the second participant.

[0054] The type of the training waiting node corresponding to the first participant is a leaf node, and after the first participant sets the class of the leaf node, it sends the class identification information of the leaf node (e.g., the numeric number of the class, the class ID, etc.) as feedback information to the second participant to set the corresponding training waiting node in the second participant.

[0055] According to the type internal leaf node of the training waiting node corresponding to the first participant, the first participant generates a fourth ciphertext by performing differential privacy processing on the ciphertext splitting gain of the first candidate splitting feature and sends the fourth ciphertext to the second participant as feedback information, and the second participant sets the splitting feature value of the training waiting node corresponding to the second participant by comparing the magnitude of the ciphertext splitting gain of the first candidate splitting feature with the second candidate splitting feature that has been subjected to differential privacy processing.

[0056] In steps S405 and S406, the ciphertext splitting gain of the first candidate splitting feature value is calculated by the first participant using the locally optimal splitting gain calculated by the second participant, and the ciphertext splitting gain of the second candidate splitting feature value is calculated by the second participant using the locally optimal splitting gain calculated by the first participant. The optimal splitting feature of the node waiting for training can be determined by comparing the magnitudes of the locally optimal splitting gains calculated by the first participant and the second participant for each other. In some embodiments, the smaller the splitting gain value, the better the performance of distinguishing samples into left and right child nodes of the node waiting for training based on the corresponding splitting feature value. A candidate splitting feature value with a smaller splitting gain can be determined as the target splitting feature data corresponding to the current node. In the present disclosure, to ensure the security of the sample feature data, the feature values ​​are not transmitted to the other party. Therefore, the candidate splitting feature value corresponding to the smaller ciphertext splitting gain is set as the splitting feature value of the node waiting for training, and the node waiting for training corresponding to the larger ciphertext splitting gain is set as an empty node. Here, an empty node means that the sample of the node is directly transmitted to the left and right child nodes, respectively.

[0057] In some other embodiments, the larger the splitting gain value, the better the performance of distinguishing the training samples into left and right child nodes of the current node based on the corresponding splitting feature value, and the candidate splitting feature value with the larger splitting gain can be determined as the target splitting feature data corresponding to the current node. The setting methods are similar and will not be described here.

[0058] S407: The first participant and the second participant update the training waiting nodes and perform training until the training termination condition of the decision tree is met and the target decision tree is obtained, and the training of all decision trees in the random forest model is completed to obtain the target random forest model.

[0059] According to the method of this embodiment, when a first participant and a second participant train a random forest model based on a common sample participating in vertical federated learning, for a node waiting to be trained in a decision tree included in the random forest model, the multiple participants use partially homomorphic encryption technology to encrypt the identifier of the node waiting to be trained to indicate whether the training sample exists, and then send it to the other participant. The other participants each calculate the ciphertext required for the calculation in the ciphertext space, and also use differential privacy technology to process the ciphertext calculation result and then feed it back to the other participant. This ensures the security of the distributed ciphertext calculation result and prevents the ciphertext calculation result from being used to obtain related information about the sample, which is advantageous to improving the security of the sample.

[0060] In the federated learning method according to the present disclosure, if the node waiting for training is an internal node, it is necessary to determine the split feature value of the node waiting for training based on the ciphertext split gains of two candidate split features calculated by the first participant and the second participant, respectively. Here, the split gain of the first candidate split feature data is calculated by the first participant, and the split gain of the second candidate split feature data is obtained by the second participant. Next, an implementation of calculating two candidate split feature values ​​will be described using the examples shown in Figures 5 and 6.

[0061] 5 is a flowchart of a longitudinal federated learning method according to an embodiment of the present disclosure. The method of this embodiment mainly describes how a first participant obtains a ciphertext splitting gain corresponding to a second candidate splitting feature value. As shown in FIG. 5, the method of this embodiment includes the following steps:

[0062] S501, a first participant selects a plurality of second random feature values.

[0063] The first participant determines first feature data of multiple dimensions based on the first feature data of each sample, and the first feature data of each dimension includes multiple second random feature values. Here, based on the first feature data of each sample, the first participant can first determine a range of values ​​that the feature value of the first feature data of each dimension can take, select first feature data of multiple dimensions that meet the value range requirements, and then determine each second random feature value in a random manner or other manner (for example, by sorting according to the magnitude of the feature value).

[0064] S502, the first participant divides the samples based on a plurality of second random feature values, the class labels of the samples and the third ciphertext received in advance, and adds random noise to obtain a second sample division result.

[0065] One possible embodiment may include the following steps:

[0066] Step a1: The first participant classifies each sample into a left and right child node of the node waiting to be trained based on the selected second random feature values, and obtains the number of left and right child node samples that are respectively classified for each second random feature value.

[0067] For example, the node waiting for training is node 1, and node 2 and node 3 are the left and right child nodes of node 1, respectively. Three dimensions of first feature data are selected, and three feature values ​​are selected for the first feature data of each dimension based on the value range of the first feature data of each dimension, i.e., a total of nine second random feature values ​​are selected. One sample distinction (i.e., division) is performed based on each second random feature value, and the samples in node 1 are distinguished into node 2 and node 3, and the number of samples in the left and right child nodes corresponding to the nine second feature values ​​is obtained.

[0068] In one possible embodiment, when the first participant classifies the sample into left and right subnodes based on the selected second random feature value, the first participant can generate a vector corresponding to the left subnode and a vector corresponding to the right subnode, respectively, based on the first identifier of the sample and whether the first feature data of the sample satisfies the selected second random feature value and the class label of the sample.

[0069] Step b1: The first participant performs a homomorphic multiplication operation and a homomorphic addition operation based on the third ciphertext and the left and right subnode sample numbers corresponding to each second feature value, respectively, to obtain a second sample division result, where the second sample division result includes the encrypted sample numbers of the left and right subnodes corresponding to each second random feature value, respectively.

[0070] In combination with the example in step a, suppose the first participant selects nine second random feature values ​​and the total number of classes in the random forest model is five. For each second random feature value, the first participant can classify each training sample into one of node 2 and node 3. Each sample present in node 2 may belong to one or more of the above five classes. Similarly, each sample present in node 3 may belong to one or more of the above five classes. Some samples from the second participant may not be present in node 1, so these training samples cannot also be classified into node 2 or node 3. In this case, based on the second identifier in the node corresponding to node 1 for each sample from the second participant, the number of samples that the first participant classifies into left and right subnodes based on each second random feature value needs to be corrected to obtain a more accurate number of samples. Taking the left child node as an example, for each selected second random feature value, the first participant can obtain the number of samples of each class in the modified left child node by performing homomorphic multiplication and homomorphic addition on the samples classified in the left child node based on the second random feature value, the class labels of each sample classified in the left child node, and the third ciphertext, and the processing method of the right child node is similar to that of the left child node.

[0071] S503, the first participant transmits the second sample division result to the second participant.

[0072] S504, the second participant uses the partially homomorphic private key to decrypt the second sample division result to obtain a plurality of data items, and performs weighting calculation based on the plurality of data items to obtain initial division gains respectively corresponding to a plurality of second random feature values.

[0073] S505: The second participant performs differential privacy processing on the initial division gains corresponding to the plurality of second random feature values, respectively, and determines the ciphertext division gains of the second candidate division feature values ​​based on the differential privacy processing results.

[0074] The second participant adds differential privacy noise to the initial splitting gains corresponding to each second random feature value based on a predetermined sensitivity and differential privacy budget, and determines a locally optimal ciphertext splitting gain from the ciphertext splitting gains to which the differential privacy noise has been added, and the second random feature value corresponding to the locally optimal ciphertext splitting gain is a second candidate splitting feature value.

[0075] In step S506, the second participant transmits the ciphertext split gain of the second candidate split feature value to the first participant, and in response, the first participant receives the ciphertext split gain corresponding to the second candidate split feature value.

[0076] In this embodiment, the first participant and the second participant use partially homomorphic encryption and differential privacy processing of interaction data to calculate optimal split feature values ​​for the training-waiting node being trained, and the first participant does not send information about the second random feature value to the second participant, thereby effectively protecting the security of the sample feature data during the interaction.

[0077] 6 is a flowchart of a vertical federated learning method according to another embodiment of the present disclosure. The method of this embodiment mainly describes how the second participant obtains the ciphertext splitting gain corresponding to the first candidate splitting feature value. As shown in FIG. 6, the method of this embodiment includes the following steps:

[0078] S601, a second participant selects a plurality of first random feature values.

[0079] The second participant determines second feature data of multiple dimensions based on the second feature data of each sample, and the second feature data of each dimension includes multiple first random feature values. Here, based on the second feature data of each sample, the second participant can first determine a range of values ​​that the feature value of the second feature data of each dimension can take, select second feature data of multiple dimensions that meet the required value range, and then determine the multiple first random feature values ​​in a random manner or other manner (for example, by sorting according to the magnitude of the feature value).

[0080] S602, a second participant divides samples based on a plurality of first random feature values ​​and a first ciphertext to obtain a first sample division result.

[0081] One possible embodiment may include the following steps:

[0082] Step a2: The second participant classifies each sample into a left subnode and a right subnode of the corresponding waiting-to-train node in the second participant based on the selected first random feature values.

[0083] In one possible embodiment, when the second participant distinguishes based on the selected first random feature value, the second participant can generate a vector corresponding to a left subnode and a vector corresponding to a right subnode, respectively, based on the second identifier of the sample and whether the second feature data of the sample satisfies the selected first random feature value.

[0084] Step b2: The second participant calculates the number of encrypted samples of the left and right child nodes of each first random feature value distinction, i.e., the first sample division result, based on the first ciphertext and the result obtained in step a2.

[0085] In step b2, the first ciphertext can indicate that the sample is in the corresponding training waiting node of the first participant, so when calculating the number of encrypted samples, using the first ciphertext is equivalent to correcting the number of samples that the second participant needs to distinguish into left and right child nodes based on each first random feature value based on the first identifier of the sample, thereby obtaining a more accurate number of samples.

[0086] S603, the second participant sends the first sample division result to the first participant.

[0087] S604: The first participant uses the partially homomorphic private key to decrypt the first sample division result to obtain a plurality of data items, and obtains initial division gains corresponding to a plurality of first random feature values ​​by weighting calculation based on the plurality of data items obtained by decryption.

[0088] S605: The first participant performs differential privacy processing on the initial division gains corresponding to the plurality of first random feature values, respectively, and determines the ciphertext division gains of the first candidate division feature values ​​based on the differential privacy processing results.

[0089] The first participant adds differential privacy noise to the initial splitting gains corresponding to each first random feature value based on a preset sensitivity and differential privacy budget, and determines a locally optimal ciphertext splitting gain from the ciphertext splitting gains to which the differential privacy noise has been added, and the first random feature value corresponding to the locally optimal ciphertext splitting gain is the first candidate splitting feature value.

[0090] In step S606, the first participant transmits the ciphertext split gain of the first candidate split feature value to the second participant, and in response, the second participant receives the ciphertext split gain corresponding to the first candidate split feature value.

[0091] In this embodiment, the first participant and the second participant use partially homomorphic encryption and differential privacy processing of interaction data to calculate optimal split feature values ​​for the training-waiting node being trained, and the second participant does not send information about the first random feature value to the first participant, thereby effectively protecting the security of the sample feature data during the interaction.

[0092] In one specific embodiment, two participants are Guest (hereinafter abbreviated as G) and Host (hereinafter abbreviated as H), respectively, where the data format of the sample owned by G is:

number

number

[0093] Note that the id of the same sample in G is the same as the id in H, which also serves as a basis for providing relevant information to identify the same sample during subsequent interactions between the two.

[0094] The training phase may include the following steps: In step 1, the relevant parameters are initialized. For example, the user selects a differential privacy budget ε, the number of feature data p to be selected, the number of feature values ​​included in each dimension of feature data q to be selected, and the minimum number of split samples I min , and the maximum depth of the decision tree, d max can be set.

[0095] In step 2, G and H respectively add label bits t / t' (i.e., first identifier / second identifier) ​​to each sample, and a label bit value of 1 indicates that the sample exists, and a label bit value of 0 indicates that the sample does not exist. Therefore, the data format of the samples owned by G is:

number

number

[0096] In step 3, the depth d of the nodes waiting for training is determined, and the depth d of the nodes waiting for training is the maximum depth d max If it is smaller, go to step 4 and the depth d of the nodes waiting for training is increased to the maximum depth d max If it is equal to , perform steps 3.1 to 3.5 below.

[0097] In step 3.1, for each class c1, c2, … c l (where l denotes the total number of classes), G is the classmark vector of the sample based on the class label of the sample.

number

number

[0098] In step 3.2, G receives the public key pk G The classmark vector of each sample is partially homomorphically encrypted using to obtain the ciphertext classmark vector of each sample, which is then sent to H. The ciphertext classmark vector of N samples can be expressed as follows by the following formula:

number

[0099] In step 3.3, H obtains the number of encrypted samples corresponding to each class by homomorphic addition and homomorphic multiplication according to the sample mark vector (vector consisting of label bit t') of the sample of the terminal and the ciphertext class mark vector sent by G. Note that the obtained number of encrypted samples is not subjected to differential privacy processing.

[0100] Here, the classmark vector of the samples owned by H is expressed as follows:

number

number

[0101] Next, H generates differential privacy noise based on the preset sensitivity s and differential privacy budget ε, and adds the differential privacy noise to the number of encrypted samples of each class. The differential privacy process can be expressed as follows:

number

[0102] In step 3.4, G is [I * c1 ],[I * c2 ],…,[I * cN ] and receive the private key sk G Decryption is performed using

number

[0103] In step 3.5, returning to the node, the node waiting to be trained is a leaf node, and the class of the leaf node is the class determined in step 3.4.

[0104] In step 4, G and H alternately calculate the number of samples present in the nodes waiting to be trained. For example, this can be achieved by the following steps 4.1 to 4.3. In step 4.1, G performs partially homomorphic encryption on the label bits of each sample in the sample set using the public key in the homomorphic key pair to obtain the ciphertext label vector corresponding to each sample, which is expressed as follows by the following equation:

number

[0105] In step 4.2, H searches for the local label bits for the same sample based on the id of each sample, and calculates and obtains the number of encrypted samples of the node waiting for training through homomorphic addition and homomorphic multiplication processes, which is expressed as follows:

number

[0106] H generates differential privacy noise based on a preset sensitivity s and a differential privacy budget ε, and uses the differential privacy noise to scramble the encrypted samples of the nodes waiting to train. This process can be expressed as follows:

number

[0107] In step 4.3, G is decoded and the number of samples waiting for training, I * =D skG ([I * ]) to get the I * I min If it is smaller than , execute steps 3.1 to 3.5, set the class for the training waiting node, and then * I min If it is greater than or equal to, execute step 5 and set the split feature value to the node waiting to be trained.

[0108] In step 5, G and H alternately calculate the split feature values ​​of the nodes waiting to be trained. For example, the following steps 5.1 to 5.11 may be included. In step 5.1, H is the key ph H Using the above, each sample in H is partially homomorphically encrypted with the label bits of the currently trained node in H, and the ciphertext label vector for each sample is:

number

[0109] In step 5.2, for each class c1, c2, … c l(where l denotes the total number of classes), G is the classmark vector of the samples based on the class label of each sample.

number

number

[0110] G partially homomorphically encrypts the classmark vector of each sample to obtain the ciphertext classmark vector of each sample, and sends it to H. See the explanation in step 3.2.

[0111] In step 5.3, H randomly selects p-dimensional feature data, and each dimension of the feature data selects q feature values, which are expressed as follows:

number

[0112] In step 5.4, H calculates the number of encrypted samples of the left and right subnodes distinguished by each feature value based on the information received in step 5.2, and can be expressed as follows:

number

number

number

[0113] In step 5.5, G randomly selects p-dimensional feature data, and each dimension of the feature data selects q feature values, which are expressed as follows:

number

number

number

[0114] In step 5.6, G calculates the number of encrypted samples of the left and right subnodes distinguished by each feature value based on the information received in step 5.1, which is expressed as follows by the following formula:

number

number

number

[0115] In step 5.7, G and H each receive their respective homomorphic private keys, [J xi,yj ] and [J * xi,yj ] is decrypted.

[0116] In step 5.8, calculate the splitting gains G and H for each feature value according to the following formulas, respectively:

number

number

[0117] In the present disclosure, a smaller splitting gain indicates a better performance of splitting a sample into left and right child nodes, and the corresponding feature value approaches the optimal local splitting gain.

[0118] In step 5.9, G and H are respectively calculated by adding differential privacy noise to the split gain of each feature value using differential privacy techniques, as follows:

number

number

[0119] In step 5.9, the sensitivity of differential privacy noise can be obtained by the following method: In step a1, the sensitivity s is initialized to 0, and the initial division gain g0 is calculated based on the formula in step 5.8.

[0120] In step a2, for any I∈J xi,yj For , change I to I = I ± 1 and change gain g based on the formula in step 5.8. + , g - Calculate |g + -If g0|>s, then s=|g + -g0| and |g - -If g0|>s, then s=|g - -g0|.

[0121] In step 5.10,

number

[0122] For the node that does not have a feature value corresponding to the optimal splitting gain, the node waiting to be trained can be set to an empty node, and then all samples present in the node waiting to be trained can be transmitted to the left and right child nodes, respectively.

[0123] In step 5.11, the node is returned.

[0124] Since the node waiting to be trained has left and right child nodes, the left child node or the right child node is the node waiting to be trained, and we can return to step 3 and train the updated node waiting to be trained.

[0125] A trained target random forest model is obtained by constantly updating and training the nodes waiting to be trained until training of all nodes in all decision trees included in the random forest models in G and H is completed.

[0126] Based on steps 1 to 5 of this embodiment, G and H can obtain global random forest models through training. As can be seen from the description of step 5.10 above, for one participant with a target split feature value, samples can be separated into left and right subnodes based on the target split feature value. To ensure sample safety, the other participant without the target split feature value can set the training waiting node to an empty node without transmitting the target split feature value to the other participant. For example, the same decision tree in the trained target random forest model is distinguished between two participants. For example, in the example shown in FIG. 7, the same decision tree is shown as a structural diagram for G and H, respectively. The structure of the decision tree for G is shown as Decision Tree 1 in FIG. 7, and the corresponding structure of the decision tree for H is shown as Decision Tree 2 in FIG. 7. Node a is an internal node in the decision tree. In Decision Tree 1, node a has a corresponding split feature value A, and in Decision Tree 2, node a is an empty node. Node b is another internal node in the decision tree. In Decision Tree 1, node b is an empty node, and in Decision Tree 2, node b has a corresponding split feature value B. That is, G has a division feature A of node a, and H has a division feature B of node b.

[0127] In addition, G has class information of the leaf node, and in order to ensure the safety of the sample and the safety of the object waiting to be classified, G sends the class identifier (e.g., a numeric number) corresponding to the leaf node to H, and H sets the received class identifier to the class of the leaf node. During prediction, H can return to G the class identifier (e.g., a numeric number) and the ID of the sample that arrived at the node or the ID of the object waiting to be classified.

[0128] Then, the two participants can predict the class of the object to be classified according to the trained random forest model. Illustratively, this may include the following steps:

[0129] If the first participant (e.g., G) owns the first feature data of the object to be classified and includes the label bit, the data format is

number

number

[0130] From the root node, if a node in the decision tree has a split feature value, the object to be classified is separated into left and right child nodes based on the split feature value of the node and the feature value of the object to be classified, and if the node in the decision tree is an empty node, the object to be classified is transmitted to the left and right child nodes, respectively.

[0131] When the object to be classified reaches the leaf node, the first participant encrypts the label bits of the object to be classified and

number

[0132] The second participant performs homomorphic multiplication based on the label bits of the objects to be classified in the corresponding leaf nodes to obtain a fusion result, and then sends the fusion result to the first participant. Note that the first participant sends the identifier of the leaf node (e.g., the digital number of the class corresponding to the leaf node) to the second participant, and the second participant can determine and perform accurate fusion based on the identifier of the leaf node.

[0133] The first participant decrypts the received fusion based on the private key in the homomorphic key pair, and the decrypted result is

number

number

[0134] Illustratively, an embodiment of the present disclosure further provides a longitudinal associative learning device. 8 is a structural schematic diagram of a vertical associative learning device according to an embodiment of the present disclosure. As shown in FIG. 8, a vertical associative learning device 700 according to this embodiment includes: a training module 801 for, when training a random forest model based on common samples participating in vertical federated learning, partially homomorphically encrypting a first identifier indicating whether the sample exists and a class label of the sample for a node waiting to be trained in a decision tree included in the random forest model to obtain a first ciphertext; a transceiver module 802 for transmitting the first ciphertext to other participants, so that the other participants can calculate a ciphertext matching the type of the node waiting to be trained based on the first ciphertext in a ciphertext space; the transceiver module 802 for receiving a second ciphertext sent from the other participant; The training module 801 trains the nodes waiting to be trained according to the type of the nodes waiting to be trained and the second ciphertext to set the class of the nodes waiting to be trained or the corresponding split feature value, the second ciphertext including the differential privacy processing result obtained by the other participant performing differential privacy processing on the calculation result of the ciphertext, and updates and trains the nodes waiting to be trained for the decision tree until a training termination condition for the decision tree is met and a target decision tree is obtained, thereby completing the training of all decision trees in the random forest model to obtain a target random forest model.

[0135] In some possible designs, the training module 801 specifically sets the class of the node waiting to be trained according to the number of encrypted samples of each class indicated by the second ciphertext when the type of the node waiting to be trained is a leaf node, and calculates the ciphertext splitting gain of a first candidate splitting feature according to a first sample splitting result included in the second ciphertext when the type of the node waiting to be trained is an internal node, where the first sample splitting result is a ciphertext result obtained by the other participant splitting samples based on multiple first random feature values ​​and the first ciphertext, and is used to set the splitting feature value of the node waiting to be trained by comparing the magnitude of the ciphertext splitting gain of the first candidate splitting feature and the second candidate splitting feature indicated by the second ciphertext.

[0136] In some possible designs, before obtaining a first ciphertext by partially homomorphically encrypting a first identifier indicating whether the sample exists and a class label of the sample, the training module 801 is further used to: obtain a node depth of the node waiting to be trained in a decision tree to which the node waiting to be trained belongs; and determine that the node waiting to be trained is a leaf node of the decision tree to which the node belongs if the node depth is equal to a predetermined maximum tree depth; and obtain a total number of encrypted samples of the node waiting to be trained returned from the other participants if the node depth is less than the predetermined maximum tree depth; and determine that the node waiting to be trained is a leaf node of the decision tree to which the node belongs if the total number of encrypted samples is less than a predetermined minimum number of divided samples; and determine that the node waiting to be trained is an internal node of the decision tree to which the node waiting to be trained belongs if the total number of encrypted samples is equal to or greater than the predetermined minimum number of divided samples.

[0137] In some possible designs, the training module 801 is specifically used for obtaining a classmark vector of the sample by a logical AND operation according to a first identifier and class label of the sample, and partially homomorphically encrypting the classmark vector of the sample based on a partially homomorphic public key to obtain the first ciphertext.

[0138] In some possible designs, the training module 801 is specifically used to decrypt the second ciphertext based on a partially homomorphic private key to obtain the number of encrypted samples corresponding to each class, and set the class corresponding to the maximum number of encrypted samples as the class of the node waiting to be trained.

[0139] In some possible designs, the training module 801 is specifically used for: decrypting the first sample division result based on a partially homomorphic private key to obtain a plurality of data items indicating the numbers of encrypted samples of left and right nodes obtained by the other participant dividing samples based on a plurality of first random feature values; performing weighting calculations according to the plurality of data items to obtain initial division gains respectively corresponding to the plurality of first random feature values; performing differential privacy processing on the initial division gains respectively corresponding to the plurality of first random feature values ​​to obtain corresponding ciphertext division gains; determining first candidate feature values ​​from the plurality of first random feature values; and obtaining the ciphertext division gains for the first candidate feature values.

[0140] In some possible designs, before receiving the second ciphertext sent from the other participant, the training module 801 is further used for selecting a plurality of second random feature values, dividing the sample based on the plurality of second random feature values, the class label of the sample, and the previously received third ciphertext, adding random noise to obtain a second sample division result, and transmitting the second sample division result to the other participant so that the other participant calculates a ciphertext division gain of a second candidate division feature based on the second sample division result.

[0141] In some possible designs, the transceiver module 802 is further used for: if the type of the node waiting to be trained is a leaf node, sending class identification information of the node waiting to be trained to the other participants; and if the type of the node waiting to be trained is an internal node, performing differential privacy processing on the ciphertext splitting gain of the first candidate split feature value to obtain a fourth ciphertext, and sending the fourth ciphertext to the other participants, so that the other participants set corresponding nodes waiting to be trained in the random forest model corresponding to the other participants based on the fourth ciphertext.

[0142] In some possible designs, the apparatus further includes a classification module 803 for inputting an object to be classified into the target random forest model, and, when the object to be classified reaches a leaf node, partially homomorphically encrypting a first identifier of the object to be classified at the leaf node to obtain a ciphertext classification result.

[0143] The transceiver module 802 is further used for sending the ciphertext classification result to the other participants and receiving a fusion result fed back from the other participants, the fusion result being obtained by fusing the ciphertext classification result and a second identifier of the object to be classified at the corresponding node in the other participants.

[0144] The classification module 803 further determines that the class of the object to be classified is the class of the currently arrived leaf node if the fusion result satisfies the preset condition; if the fusion result does not satisfy the preset condition, it is used to continue to determine the next leaf node at which the object to be classified arrives until the class of the object to be classified is determined.

[0145] The device of this embodiment may be used to realize the steps performed by the first participant in the method embodiment, and its realization principle and technical effect are similar, and reference may be made to the detailed description of the method embodiment, and for the sake of brevity, the description will be omitted here.

[0146] 9 is a structural schematic diagram of an electronic device according to an embodiment of the present disclosure. As shown in FIG. 9, an electronic device 900 according to this embodiment includes: a transceiver module 901 for receiving a first ciphertext sent by another participant to a node waiting to be trained in a decision tree included in the random forest model, when the random forest model is trained based on common samples participating in vertical federated learning, the first ciphertext being obtained by the other participant performing partially homomorphic encryption on a first identifier indicating whether the sample exists and a class label of the sample; a training module 902 for calculating a calculation result of a ciphertext matching a type of a node waiting to be trained according to the first ciphertext in a ciphertext space, and performing differential privacy processing based on the calculation result of the ciphertext to obtain a second ciphertext; the transceiver module 901 for sending the second ciphertext to the other participants and receiving feedback information sent from the other participants so that the other participants train a random forest model based on the second ciphertext; and a training module 902 for training the nodes waiting to be trained according to the feedback information and the second ciphertext to set the classes or corresponding split feature values ​​of the nodes waiting to be trained, updating and training the nodes waiting to be trained for the decision trees until a training termination condition for the decision trees is met and a target decision tree is obtained, and completing the training of all decision trees in the random forest model to obtain a target random forest model.

[0147] In some possible designs, the training module 902 is specifically used for: if the type of the node waiting for training is a leaf node, calculating the number of encrypted samples of each class according to the first ciphertext and a second identifier indicating whether the samples exist; if the type of the node waiting for training is an internal node, dividing the samples according to the first ciphertext and a plurality of first random feature values ​​to obtain first sample division results, and obtaining initial division gains of a plurality of second random feature values ​​according to second sample division results sent by the other participants, where the second sample division results are ciphertext results obtained by dividing the samples according to a third ciphertext, the plurality of second random feature values, and class labels of the samples and adding noise, and the third ciphertext is obtained by partially homomorphically encrypting the second identifier.

[0148] In some possible designs, the training module 902 is specifically used to set the class of the node waiting to be trained according to the class identification information indicated by the feedback information when the type of the node waiting to be trained is a leaf node.

[0149] In some possible designs, when the type of the node waiting to train is an internal node, the feedback information includes a fourth ciphertext, and the fourth ciphertext is obtained by the other participant calculating a ciphertext splitting gain of a first candidate splitting feature based on the first sample splitting result and performing differential privacy processing, and the training module 902 is specifically used to set the splitting feature value of the node waiting to train by comparing the magnitude of the ciphertext splitting gain of the first candidate splitting feature and the second candidate splitting feature indicated by the second ciphertext.

[0150] In some possible designs, the transceiver module 901 is further used to receive a ciphertext classification result sent by the other participant, the ciphertext classification result being obtained by performing partially homomorphic encryption on a first identifier at a leaf node where the object to be classified arrives after the object to be classified is input into a random forest model corresponding to the other participant.

[0151] a classification module 903 for inputting the object to be classified into the target random forest model to obtain a second identifier of the object to be classified at a corresponding node, and fusing the ciphertext classification result and the second identifier in a ciphertext space to obtain a fusion result;

[0152] The transceiver module 901 is used for transmitting the fusion result to the other participants so that the other participants can determine the class of the object to be classified according to the fusion result.

[0153] The device of this embodiment may be used to realize the steps performed by the second participant in the method embodiment, and its realization principle and technical effect are similar, and reference may be made to the detailed description of the method embodiment, and for the sake of brevity, the description will be omitted here.

[0154] Illustratively, the present disclosure provides an electronic device that includes one or more processors, a memory, and one or more computer programs, wherein the one or more computer programs are stored in the memory, and when the one or more processors execute the one or more computer programs, the electronic device realizes the longitudinal associative learning method performed by the first participant or the second participant in the above-described embodiments.

[0155] Illustratively, the present disclosure provides a chip system applied to an electronic device including a memory and a sensor, the chip system including a processor, and the processor executes the longitudinal associative learning method executed by the first participant or the second participant in the above-mentioned embodiments.

[0156] Illustratively, the present disclosure provides a computer-readable storage medium having a computer program stored thereon, the computer program, when executed by a processor, realizing the longitudinal associative learning method performed by the first participant or the second participant in the aforementioned embodiments.

[0157] Illustratively, the present disclosure provides a computer program product, which, when executed on a computer, causes the computer to perform the longitudinal associative learning method performed by the first participant or the second participant in the aforementioned embodiments.

[0158] In the above embodiments, all or part of the functions may be implemented by software, hardware, or a combination of software and hardware. When implemented by software, all or part of the functions may be implemented in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded into a computer and executed, all or part of the procedures or functions according to the embodiments of the present disclosure occur. The computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions may be stored in a computer-readable storage medium. The computer-readable storage medium may be any available medium accessible by a computer, or a data storage device such as a server or data center that includes one or more available media. The available medium may be a magnetic medium (e.g., a floppy disk, a hard disk, a magnetic tape), an optical medium (e.g., a DVD), or a semiconductor medium (e.g., a solid-state disk (SSD)).

[0159] It should be noted that, in this context, relational terms such as "first" and "second" are merely used to distinguish one entity or operation from another and do not necessarily require or imply the existence of any actual relationship or sequence between those entities or operations. Furthermore, the terms "comprise," "have," or any other variation thereof are intended to cover a non-exclusive inclusion, such that a process, method, article, or facility that includes a set of elements includes not only those elements but also other elements not expressly listed, or further elements inherent in such process, method, article, or facility. Absent more limitations, an element qualified by the phrase "comprises a ..." does not exclude the presence of other identical elements in the process, method, article, or facility that includes said element.

[0160] The foregoing are merely specific embodiments of the present disclosure, and those skilled in the art will be able to understand or realize the present disclosure. Various modifications of these embodiments will be apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present disclosure. Therefore, the present disclosure is not limited to the embodiments described herein, but is accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. When training a random forest model based on common samples participating in vertical federated learning, for a node awaiting training in a decision tree included in the random forest model, partially homomorphically encrypting a first identifier indicating whether the sample exists and a class label of the sample to obtain a first ciphertext; transmitting the first ciphertext and the node type information to the other participants so that the other participants calculate a calculation result of a ciphertext matching the type of the node waiting to train based on the first ciphertext and node type information of the node waiting to train in a ciphertext space, wherein the node type information indicates that the type of the node waiting to train is a leaf node or an internal node; receiving a second ciphertext transmitted from the other participant, and training the node waiting to be trained according to the type of the node waiting to be trained and the second ciphertext to set a class of the node waiting to be trained or a corresponding split feature value, wherein the second ciphertext includes a result of differential privacy processing performed by the other participant on a calculation result of the ciphertext; updating and training the training-waiting nodes for the decision tree until a training termination condition for the decision tree is met and a target decision tree is obtained, thereby completing the training of all decision trees in the random forest model and obtaining a target random forest model; A vertical associative learning method comprising:

2. training the nodes waiting to be trained according to the types of the nodes waiting to be trained and the second ciphertext to set classes or corresponding split feature values ​​of the nodes waiting to be trained, If the type of the node waiting for training is a leaf node, setting a class of the node waiting for training according to the number of encrypted samples of each class indicated by the second ciphertext; If the type of the node waiting for training is an internal node, calculating a ciphertext splitting gain of a first candidate splitting feature according to a first sample splitting result included in the second ciphertext, wherein the first sample splitting result is a ciphertext result obtained by the other participant splitting a sample based on a plurality of first random feature values ​​and the first ciphertext, and setting a splitting feature value of the node waiting for training by comparing the magnitude of the ciphertext splitting gain of the first candidate splitting feature and a second candidate splitting feature indicated by the second ciphertext; 2. The method of claim 1, comprising:

3. before partially homomorphically encrypting a first identifier indicating whether the sample exists and a class label of the sample to obtain a first ciphertext, obtaining a node depth of the awaiting training node in the decision tree to which it belongs; If the node depth is equal to a preset maximum tree depth, determine that the node waiting to be trained is a leaf node of the decision tree to which it belongs; if the node depth is less than the preset maximum tree depth, obtain the total number of encrypted samples of the node waiting to be trained returned by the other participants; If the total number of the encrypted samples is smaller than a predetermined minimum number of divided samples, determining that the node waiting for training is a leaf node of a decision tree to which the node belongs, and if the total number of the encrypted samples is equal to or greater than the predetermined minimum number of divided samples, determining that the node waiting for training is an internal node of a decision tree to which the node belongs; The method of claim 1 further comprising:

4. Obtaining a first ciphertext by partially homomorphically encrypting a first identifier indicating whether the sample exists and a class label of the sample, Obtaining a classmark vector of the sample by a logical AND operation according to a first identifier and a class label of the sample; partially homomorphically encrypting the sample classmark vector based on a partially homomorphic public key to obtain the first ciphertext; 2. The method of claim 1, comprising:

5. setting a class of the nodes waiting for training according to the number of encrypted samples of each class indicated by the second ciphertext, decrypting the second ciphertext based on the partially homomorphic private key to obtain the number of encrypted samples corresponding to each class, and setting the class corresponding to the maximum number of encrypted samples as the class of the node waiting to train; 3. The method of claim 2, comprising:

6. Calculating a ciphertext splitting gain of a first candidate splitting feature according to a first sample splitting result included in the second ciphertext includes: decrypting the first sample division result based on the partially homomorphic private key to obtain a plurality of data items indicating the numbers of encrypted samples of the left and right nodes obtained by dividing the samples by the other participants based on the plurality of first random feature values; performing a weighting calculation according to the plurality of data items to obtain initial division gains corresponding to the plurality of first random feature values; performing a differential privacy process on initial split gains corresponding to the plurality of first random feature values ​​to obtain corresponding ciphertext split gains; determining first candidate feature values ​​from the plurality of first random feature values; and obtaining ciphertext split gains for the first candidate feature values; 3. The method of claim 2, comprising:

7. The type of the training-waiting node is an internal node, and before receiving a second ciphertext transmitted from the other participant, selecting a plurality of second random feature values, dividing the samples according to the plurality of second random feature values, the class labels of the samples, and the previously received third ciphertext, and adding random noise to obtain a second sample division result; sending the second sample division result to the other participants so that the other participants calculate a ciphertext division gain of a second candidate division feature based on the second sample division result; 3. The method of claim 2, further comprising:

8. The method comprises: If the type of the node waiting to train is a leaf node, transmitting class identification information of the node waiting to train to the other participants; If the type of the node waiting to be trained is an internal node, performing a differential privacy process on the ciphertext splitting gain of the first candidate split feature to obtain a fourth ciphertext, and sending the fourth ciphertext to the other participants so that the other participants set corresponding nodes waiting to be trained in the random forest model corresponding to the other participants based on the fourth ciphertext; 3. The method of claim 2, comprising:

9. Inputting an object to be classified into the target random forest model, and when the object to be classified reaches a leaf node, partially homomorphically encrypting a first identifier of the object to be classified at the leaf node to obtain a ciphertext classification result; transmitting the ciphertext classification result to the other participants; receiving a fusion result fed back from the other participants, the fusion result being obtained by fusing the ciphertext classification result and a second identifier of the object to be classified at a corresponding node in the other participants; If the fusion result satisfies a preset condition, determine that the class of the object to be classified is the class of the currently arrived leaf node; if the fusion result does not satisfy the preset condition, continue to determine the next leaf node at which the object to be classified arrives until the class of the object to be classified is determined; The method of claim 1 further comprising:

10. When training a random forest model based on common samples participating in vertical federated learning, receiving a first ciphertext sent by another participant for a node waiting to be trained in a decision tree included in the random forest model and node type information of the node waiting to be trained, wherein the first ciphertext is obtained by the other participant partially homomorphically encrypting a first identifier indicating whether the sample exists and a class label of the sample, and the node type information indicates that the type of the node waiting to be trained is a leaf node or an internal node; Calculating a calculation result of a ciphertext that matches the type of the node waiting to be trained in a ciphertext space according to the first ciphertext and the node type information, performing differential privacy processing based on the calculation result of the ciphertext to obtain a second ciphertext, and sending the second ciphertext to the other participants so that the other participants can train a random forest model based on the second ciphertext; receiving feedback information transmitted from the other participants, and setting a class or a corresponding split feature value of the node waiting to be trained according to the feedback information and the second ciphertext; updating and training the training-waiting nodes for the decision tree until a training termination condition for the decision tree is met and a target decision tree is obtained, thereby completing the training of all decision trees in the random forest model and obtaining a target random forest model; A vertical associative learning method comprising:

11. calculating a calculation result of a ciphertext matching a type of a node waiting to be trained in a ciphertext space according to the first ciphertext and the node type information, If the type of the node waiting to be trained is a leaf node, calculating the number of encrypted samples of each class according to the first ciphertext and a second identifier indicating whether the sample exists; If the type of the node waiting to be trained is an internal node, obtain a first sample division result by dividing samples according to the first ciphertext and a plurality of first random feature values, and obtain initial division gains of a plurality of second random feature values ​​according to a second sample division result transmitted by the other participant, wherein the second sample division result is a ciphertext result obtained by dividing samples according to a third ciphertext, the plurality of second random feature values, and class labels of the samples and adding noise, the third ciphertext being obtained by partially homomorphically encrypting the second identifier; 11. The method of claim 10, comprising:

12. setting a class of the node waiting to be trained or a corresponding split feature value according to the feedback information and the second ciphertext, If the type of the node waiting to be trained is a leaf node, setting a class of the node waiting to be trained according to class identification information indicated by the feedback information.

12. The method of claim 11 .

13. If the type of the node waiting to be trained is an internal node, the feedback information includes a fourth ciphertext, and the fourth ciphertext is obtained by the other participants calculating a ciphertext splitting gain of a first candidate splitting feature based on the first sample splitting result and performing differential privacy processing; setting a class of the node waiting to be trained or a corresponding split feature value according to the feedback information and the second ciphertext, setting a division feature value of the node waiting to be trained by comparing a magnitude of a ciphertext division gain of the first candidate division feature and a magnitude of a ciphertext division gain of a second candidate division feature indicated by the second ciphertext; 12. The method of claim 11 .

14. The method comprises: receiving a ciphertext classification result transmitted by the other participant, wherein the ciphertext classification result is obtained by performing partially homomorphic encryption on a first identifier at a leaf node at which the object to be classified arrives after the object to be classified is input into a random forest model corresponding to the other participant; inputting the object to be classified into the target random forest model to obtain a second identifier of the object to be classified at a corresponding node; and fusing the ciphertext classification result and the second identifier in a ciphertext space to obtain a fusion result; transmitting the fusion result to the other participants so that the other participants determine the class of the object to be classified according to the fusion result; 11. The method of claim 10, further comprising:

15. a training module for, when training a random forest model based on common samples participating in vertical federated learning, partially homomorphically encrypting a first identifier indicating whether the sample exists and a class label of the sample for a node waiting to be trained in a decision tree included in the random forest model to obtain a first ciphertext; a transmitting / receiving module for transmitting the first ciphertext and the node type information to the other participants so that the other participants can calculate a calculation result of a ciphertext matching a type of the node waiting to train based on the first ciphertext and node type information of the node waiting to train in a ciphertext space, the node type information indicating that the type of the node waiting to train is a leaf node or an internal node; the transceiver module for receiving a second ciphertext sent from the other participant; a training module for training the nodes waiting to be trained according to the type of the nodes waiting to be trained and the second ciphertext to set the class of the nodes waiting to be trained or the corresponding split feature value, the second ciphertext including a differential privacy processing result obtained by the other participant performing a differential privacy processing on the calculation result of the ciphertext, updating and training the nodes waiting to be trained for the decision tree until a training termination condition for the decision tree is satisfied and a target decision tree is obtained, and completing the training of all decision trees in the random forest model to obtain a target random forest model; A vertical associative learning device comprising:

16. a transmitting / receiving module for receiving, when training a random forest model based on common samples participating in vertical federated learning, a first ciphertext transmitted by another participant for a node waiting to be trained in a decision tree included in the random forest model and node type information of the node waiting to be trained, the first ciphertext being obtained by the other participant performing partially homomorphic encryption on a first identifier indicating whether the sample exists and a class label of the sample, and the node type information indicating that the type of the node waiting to be trained is a leaf node or an internal node; a training module for calculating a calculation result of a ciphertext matching a type of a node waiting to be trained in a ciphertext space according to the first ciphertext and the node type information, and performing differential privacy processing based on the calculation result of the ciphertext to obtain a second ciphertext; the transceiver module for transmitting the second ciphertext to the other participants so that the other participants train a random forest model based on the second ciphertext and receiving feedback information transmitted from the other participants; a training module for training the nodes waiting to be trained according to the feedback information and the second ciphertext to set the classes or corresponding split feature values ​​of the nodes waiting to be trained, updating and training the nodes waiting to be trained for the decision trees until a training termination condition for the decision trees is met and a target decision tree is obtained, and completing the training of all decision trees in the random forest model to obtain a target random forest model; A vertical associative learning device comprising:

17. An electronic device comprising a memory and a processor, the memory configured to store computer program instructions; The processor is configured to execute the computer program instructions to cause the electronic device to implement the longitudinal associative learning method according to any one of claims 1 to 9 or 10 to 14. An electronic device characterized by:

18. A readable storage medium comprising computer program instructions that, when executed by at least one processor of an electronic device, cause the electronic device to implement the longitudinal associative learning method of any one of claims 1 to 9 or 10 to 14.

19. A computer program, which, when executed by at least one processor of an electronic device, causes the electronic device to implement the longitudinal associative learning method according to any one of claims 1 to 9 or 10 to 14.

Images