Model training method and device and electronic equipment

Abstract

Embodiments of a model training method, device and electronic equipment are provided. The method comprises: according to feature data, splitting a sample identification set into a plurality of subsets, receiving first gradient value ciphertext and second gradient value ciphertext corresponding to sample identification; within a subset, homomorphically adding the first gradient value ciphertext of a plurality of sample identifications to obtain the first feature value ciphertext of the subset, and homomorphically adding the second gradient value ciphertext of a plurality of sample identifications to obtain the second feature value ciphertext of the subset; using a random number to mask the first feature value ciphertext and the second feature value ciphertext to obtain the masked first feature value ciphertext and the masked second feature value ciphertext; and sending the masked first feature value ciphertext and the masked second feature value ciphertext corresponding to the subset to a second party. According to the embodiments of the present specification, the feature value ciphertext is masked using a random number according to a homomorphic encryption algorithm, which can enhance the privacy protection of data in the process of multi-party cooperation modeling.

Description

Technical Field

[0001] The embodiments in this specification relate to the field of computer technology, and in particular to a model training method, apparatus, and electronic device. Background Technology

[0002] In practice, a single data provider often lacks complete data and typically needs to leverage data from other providers to collaboratively train data processing models. This multi-party collaborative modeling process frequently raises concerns about privacy leaks. Summary of the Invention

[0003] This specification provides a model training method, apparatus, and electronic device to enhance data privacy protection during multi-party collaborative modeling.

[0004] To achieve the above objectives, one or more embodiments in this specification provide the following technical solutions.

[0005] According to a first aspect of one or more embodiments of this specification, a model training method is provided, applied to a first party, the first party holding feature data of samples, the method comprising: dividing a sample identifier set into multiple subsets according to the feature data, the sample identifier set including multiple sample identifiers; receiving a first gradient value ciphertext and a second gradient value ciphertext corresponding to each sample identifier, the first gradient value ciphertext and the second gradient value ciphertext being calculated by a homomorphic encryption algorithm; within each subset, homomorphically adding the first gradient value ciphertexts of the multiple sample identifiers to obtain a first feature value ciphertext of the subset, and homomorphically adding the second gradient value ciphertexts of the multiple sample identifiers to obtain a second feature value ciphertext of the subset; masking the first feature value ciphertext and the second feature value ciphertext using random numbers respectively, to obtain masked first feature value ciphertext and masked second feature value ciphertext; and sending the masked first feature value ciphertext and the masked second feature value ciphertext corresponding to each subset to a second party to facilitate training of non-leaf nodes of a data processing model.

[0006] According to a second aspect of one or more embodiments of this specification, a model training method is provided, applied to a second party holding labeled data of samples. The method includes: receiving a masked first feature value ciphertext and a masked second feature value ciphertext corresponding to a subset, the subset being obtained by segmenting a sample identifier set, the sample identifier set including multiple sample identifiers; decrypting the masked first feature value ciphertext and the masked second feature value ciphertext respectively to obtain a masked first feature value and a masked second feature value; and calculating a segmentation gain factor using the masked first feature value and the masked second feature value, the segmentation gain factor being used to calculate the segmentation gain of the subset, the segmentation gain being used to train the non-leaf nodes of a data processing model.

[0007] According to a third aspect of one or more embodiments of this specification, a model training apparatus is provided, applied to a first party holding feature data of samples. The apparatus includes: a segmentation unit, configured to segment a sample identifier set into multiple subsets based on the feature data, the sample identifier set including identifiers of multiple samples; a receiving unit, configured to receive a first gradient value ciphertext and a second gradient value ciphertext corresponding to each sample identifier, the first gradient value ciphertext and the second gradient value ciphertext being calculated by a homomorphic encryption algorithm; an addition unit, configured to homomorphically add the first gradient value ciphertexts of multiple sample identifiers within each subset to obtain a first feature value ciphertext of the subset, and homomorphically add the second gradient value ciphertexts of multiple sample identifiers to obtain a second feature value ciphertext of the subset; a masking unit, configured to mask the first feature value ciphertext and the second feature value ciphertext using random numbers respectively, to obtain masked first feature value ciphertext and masked second feature value ciphertext; and a sending unit, configured to send the masked first feature value ciphertext and the masked second feature value ciphertext corresponding to each subset to a second party, so as to facilitate training of non-leaf nodes of a data processing model.

[0008] According to a fourth aspect of one or more embodiments of this specification, a model training apparatus is provided, applied to a second party holding labeled data of samples. The apparatus includes: a receiving unit for receiving a masked first feature value ciphertext and a masked second feature value ciphertext corresponding to a subset, the subset being obtained by segmenting a sample identifier set, the sample identifier set including multiple sample identifiers; a decryption unit for decrypting the masked first feature value ciphertext and the masked second feature value ciphertext respectively to obtain a masked first feature value and a masked second feature value; and a calculation unit for calculating a segmentation gain factor using the masked first feature value and the masked second feature value, the segmentation gain factor being used to calculate the segmentation gain of the subset, the segmentation gain being used to train non-leaf nodes of a data processing model.

[0009] According to a fifth aspect of one or more embodiments of this specification, an electronic device is provided, including a memory and a processor; the memory is used to store computer instructions; the processor is used to perform the steps of the method described in the first aspect.

[0010] According to a sixth aspect of one or more embodiments of this specification, an electronic device is provided, including a memory and a processor; the memory is used to store computer instructions; the processor is used to perform the steps of the method described in the second aspect.

[0011] As can be seen from the technical solutions provided in the embodiments of this specification above, by using a homomorphic encryption algorithm to mask the ciphertext of feature values ​​with random numbers, the privacy protection of data in the process of multi-party collaborative modeling can be enhanced. Attached Figure Description

[0012] To more clearly illustrate the technical solutions in the embodiments or prior art of this specification, the drawings used in the description of the embodiments or prior art will be briefly introduced below. The drawings described below are only some embodiments recorded in this specification. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0013] Figure 1 This is a schematic diagram of a decision tree model according to an embodiment of this specification;

[0014] Figure 2 This is a flowchart of a model training method according to an embodiment of this specification;

[0015] Figure 3 This is a flowchart of a model training method according to an embodiment of this specification;

[0016] Figure 4 This is a flowchart of a model training method according to an embodiment of this specification;

[0017] Figure 5 This is a functional structure diagram of a model training device according to an embodiment of this specification;

[0018] Figure 6 This is a functional structure diagram of a model training method according to an embodiment of this specification;

[0019] Figure 7 This is a functional structure diagram of an electronic device according to an embodiment of this specification. Detailed Implementation

[0020] The technical solutions in the embodiments of this specification will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of this specification, and not all of them. Based on the embodiments in this specification, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of this specification. Furthermore, it should be understood that although the terms "first," "second," "third," etc., may be used in this specification to describe various information, this information should not be limited to these terms. These terms are only used to distinguish information of the same type from each other. For example, without departing from the scope of this specification, first information may also be referred to as second information, and similarly, second information may also be referred to as first information.

[0021] The technical terms used in the embodiments of this specification will be explained below.

[0022] Tree model: A supervised machine learning model. The tree model can be, for example, a binary tree. The tree model can include a decision tree model, which may include regression decision trees and classification decision trees, etc. The tree model includes multiple nodes. Each node may correspond to a position identifier, which can be used to identify the node's position in the tree model, specifically, for example, the node's number. The multiple nodes can form multiple prediction paths. The starting node of the prediction path is the root node of the tree model, and the ending node is a leaf node of the tree model.

[0023] Leaf node: A node in a tree model that cannot split downwards is called a leaf node. Each leaf node corresponds to a leaf value. Different leaf nodes in the tree model may have the same or different leaf values. Each leaf value can represent a prediction result. The leaf value can be a numerical value or a vector, etc.

[0024] Non-leaf nodes: When a node in a tree model can split downwards, that node can be called a non-leaf node. Specifically, a non-leaf node can include the root node, and all other nodes except the leaf nodes and the root node (hereinafter referred to as internal nodes). Each non-leaf node has a corresponding splitting condition, which can be used to select a prediction path.

[0025] One or more tree models can constitute a forest model. The forest model can be a supervised machine learning model, specifically including regression decision forests and classification decision forests. Algorithms used to integrate multiple tree models into a forest model can include Random Forest, Extreme Gradient Boosting (XGBoost), Gradient Boosting Decision Tree (GBDT), and other algorithms.

[0026] The following is a scenario example of a tree model.

[0027] Please see Figure 1In this example scenario, the tree model Tree1 can include 11 nodes: 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, and 11. Node 1 is the root node; nodes 2, 3, 4, and 5 are internal nodes; and nodes 6, 7, 8, 9, 10, and 11 are leaf nodes. Nodes 1, 2, 4, and 8 can form a prediction path; nodes 1, 2, 4, and 9 can form a prediction path; nodes 1, 2, 5, and 10 can form a prediction path; nodes 1, 2, 5, and 11 can form a prediction path; nodes 1, 3, and 6 can form a prediction path; and nodes 1, 3, and 7 can form a prediction path.

[0028] The splitting conditions corresponding to nodes 1, 2, 3, 4 and 5 are shown in Table 1 below.

[0029] Table 1

[0030] node Splitting conditions Node 1 Age over 30 Node 2 Annual income greater than 50,000 Node 3 Own a house Node 4 Have a car Node 5 Married

[0031] The splitting criteria "age greater than 20 years old", "annual income greater than 50,000", "owns a house", "owns a car", and "married" can be used to select the prediction path. When the splitting criteria are not met (i.e., the judgment result is 0), the prediction path on the left can be selected; when the splitting criteria are met (i.e., the judgment result is 0), the prediction path on the right can be selected.

[0032] The leaf values ​​corresponding to nodes 6, 7, 8, 9, 10, and 11 are shown in Table 2 below.

[0033] Table 2

[0034] node Leaf value Node 6 20 Node 7 40 Node 8 80 Node 9 100 Node 10 200 Node 11 250

[0035] A loss function measures the degree of discrepancy between the predicted and actual values ​​of a data processing model. A smaller loss function value indicates better robustness of the data processing model. Loss functions include, but are not limited to, logarithmic loss and square loss.

[0036] This specification provides an embodiment of a model training system.

[0037] The model training system may include a first party and a second party. The first party may be a device such as a server, mobile phone, tablet computer, or personal computer; or it may be a system composed of multiple devices, such as a server cluster composed of multiple servers. The second party may be a device such as a server, mobile phone, tablet computer, or personal computer; or it may be a system composed of multiple devices, such as a server cluster composed of multiple servers.

[0038] In some embodiments, the first party holds the sample's feature data but not its label data. The second party holds the sample's label data. The second party may not hold the sample's feature data, or it may hold some of the sample's feature data. The first and second parties may perform collaborative security modeling. During collaborative security modeling, for data privacy protection, the first party may not disclose the sample's feature data to the second party, and the second party may not disclose the sample's label data to the first party.

[0039] The model obtained through collaborative security modeling can include a forest model, which may include at least one tree model. In practical applications, the first party and the second party can recursively train the nodes in the forest model. Algorithms used for recursive training include, but are not limited to, XGBoost, ID3, C4.5, C5.0, etc.

[0040] by Figure 1 Taking the tree model shown as an example, a non-leaf node 1 can correspond to a set of sample identifiers. The samples corresponding to each sample identifier in the set are used to train the non-leaf node 1. The first party can hold the feature data of the samples corresponding to each sample identifier, and the second party can hold the label data of the samples corresponding to each sample identifier. The first party can train the non-leaf node 1 based on its own feature data, and the second party can train the non-leaf node 1 based on its own label data, thereby obtaining the splitting condition of the non-leaf node 1. After training the non-leaf node 1, the splitting condition corresponding to the non-leaf node 1 can be obtained, and the set of sample identifiers is divided into a first subset and a second subset.

[0041] The first subset can correspond to non-leaf node 2. The samples corresponding to each sample identifier in the first subset are used to train non-leaf node 2. The first party can hold the feature data of the samples corresponding to each sample identifier, and the second party can hold the label data of the samples corresponding to each sample identifier. The first party can train non-leaf node 2 based on its own feature data, and the second party can train non-leaf node 2 based on its own label data, to obtain the splitting condition for non-leaf node 2. After training non-leaf node 2 is completed, the splitting condition corresponding to non-leaf node 2 can be obtained, and the first subset is further divided into two subsets to facilitate further training of non-leaf node 4 and non-leaf node 5. Subsequent processes will not be elaborated further.

[0042] The second subset can correspond to non-leaf node 3. The samples corresponding to each sample identifier in the second subset are used to train non-leaf node 3. The first party can hold the feature data of the samples corresponding to each sample identifier, and the second party can hold the label data of the samples corresponding to each sample identifier. The first party can train non-leaf node 3 based on its own feature data, and the second party can train non-leaf node 3 based on its own label data, to obtain the splitting condition for non-leaf node 3. After training non-leaf node 3 is completed, the splitting condition corresponding to non-leaf node 3 can be obtained. The second subset is further divided into two subsets to facilitate further training of leaf nodes 6 and 7, obtaining the leaf values ​​of leaf node 6 and leaf node 7.

[0043] In some embodiments, a sample identifier can be used to identify a sample. For example, a sample can be data of a business object, and the sample identifier can be an identifier of the business object. Specifically, for example, a sample can be user data, and the sample identifier can be a user's identity identifier. As another example, a sample can be product data, and the sample identifier can be a product identifier.

[0044] A sample may include feature data and label data. The feature data may include P sub-data points across P dimensions, where P is a positive integer. For example, a sample x1 can be represented as a vector [x11, x12, ..., x1...]. i ,...,x1 p [,Y1]. x11, x12, ..., x1 i ... x1 p The feature data consists of P sub-data points across P dimensions. Y1 represents the label data. For example, for sample x1, the feature data includes: loan amount data in the loan amount dimension, social security base data in the social security base dimension, marital status data in the marital status dimension, and property ownership data in the property ownership dimension. The label data includes: whether the user is a defaulter.

[0045] The following is a scenario example. In this scenario, the first party is a big data company, and the second party is a credit reporting agency. The big data company holds data such as the user's loan amount, the user's social security contribution base, whether the user is married, and whether the user owns a house. The credit reporting agency holds data such as whether the user is a defaulter. The big data company and the credit reporting agency can collaborate on secure modeling based on their respective user data to obtain a forest model. The forest model can be used to predict whether a user is a defaulter. During the collaborative secure modeling process, to protect data privacy, the big data company cannot disclose its data to the credit reporting agency, and the credit reporting agency cannot disclose its data to the big data company.

[0046] This specification provides an embodiment of a model training method.

[0047] The described model training method can be used to train a non-leaf node in a forest model, where the non-leaf node can be the root node or an internal node. In practical applications, using this model training method, a recursive approach can be adopted to train each non-leaf node in the forest model, thereby achieving cooperative and safe modeling.

[0048] Please see Figure 2 The model training method may include the following steps.

[0049] Step S101: The first party divides the sample identifier set into multiple subsets based on the feature data.

[0050] In some embodiments, the sample identifier set may include multiple sample identifiers. The samples corresponding to each sample identifier in the sample identifier set are used for training non-leaf nodes. Specifically, when the non-leaf node is the root node, the sample identifier set may be the original sample identifier set, which may include sample identifiers of samples used to train the forest model. When the non-leaf node is an internal node, the sample identifier set may be a subset obtained after training the previous non-leaf node.

[0051] In some embodiments, the first party may hold feature data of the samples corresponding to each sample identifier in the sample identifier set. The feature data may include P sub-data points across P dimensions, where P is a positive integer. The first party may divide the sample identifier set into multiple subsets based on the sub-data points in at least one dimension. In practical applications, the first party may divide the sample identifier set into multiple subsets based on the sub-data points in each dimension.

[0052] For example, the sample identifier set may include sample identifiers for N samples, such as x1, x2, ..., xi, ..., xN, and the feature data of each sample may include P sub-data points in P dimensions. The sub-data points of samples x1, x2, ..., xi, ..., xN in the i-th dimension are x1, x2, ..., xi, ..., xN respectively. i x2 i ,...,xi i ,...,xN i Therefore, based on sub-data x1 i x2 i ,...,xi i ,...,xN iThe first party can divide the sample identifiers of samples x1, x2, ..., xi, ..., xN into multiple subsets. Specifically, for example, the i-th dimension can be age. The sub-data of samples x1, x2, ..., xi, ..., xN in the age dimension are x1, x2, ..., xi, ..., xN respectively. i =30,x2 i =35,...,xi i =20,...,xN i =50. Therefore, the first party can divide the sample identifiers of samples x1, x2, ..., xi, ..., xN into three subsets: T1, T2, and T3. In subset T1, the sample identifiers correspond to age sub-data points of 0-20 years old; in subset T2, they correspond to age sub-data points of 21-30 years old; and in subset T3, they correspond to age sub-data points of 31-50 years old.

[0053] Step S103: The second party calculates the first gradient value ciphertext and the second gradient value ciphertext corresponding to the sample identifier.

[0054] In some embodiments, the first gradient value ciphertext and the second gradient value ciphertext can be calculated by the loss function of the forest model. Specifically, the second party may hold the label data of the samples corresponding to each sample identifier in the sample identifier set. Based on the label data, the second party may calculate the first gradient value and the second gradient value corresponding to each sample identifier in the sample identifier set. The first gradient value may be the first gradient value of the loss function, and the second gradient value may be the second gradient value of the loss function. It is worth noting that the second party may hold the sample label data but not the sample feature data. Therefore, the second party may calculate the first gradient value and the second gradient value corresponding to each sample identifier in the sample identifier set based only on the label data. Alternatively, the second party may hold the sample label data and some feature data. Therefore, the second party may calculate the first gradient value and the second gradient value corresponding to each sample identifier in the sample identifier set based on the label data and some feature data.

[0055] Taking the XGBoost algorithm as an example, the second party can... Calculate the first gradient value corresponding to the sample identifier; it can be based on Calculate the second gradient value corresponding to the sample identifier. Here, g represents the first gradient value, h represents the second gradient value, l represents the loss function, and represents the label data. This represents the predicted value of the labeled data, where t represents the current iteration round. This represents the predicted value after the (t-1)th iteration. Those skilled in the art should understand that the formulas used here to calculate the first and second gradient values ​​are merely examples, and other variations or modifications are possible in practice. Furthermore, the XGBoost algorithm described here is also just an example; other training algorithms can be used in practice.

[0056] In some embodiments, the second party can encrypt the first gradient value and the second gradient value to obtain ciphertext of the first gradient value and the second gradient value corresponding to each sample identifier in the sample identifier set. Specifically, the second party can use a homomorphic encryption algorithm to encrypt the first gradient value and the second gradient value. The homomorphic encryption algorithm may include the Paillier algorithm, the Okamoto-Uchiyama algorithm, the Damgard-Jurik algorithm, etc. Homomorphic encryption is an encryption technique. It allows direct operation on ciphertext data to obtain an encrypted result, and the result obtained by decrypting it is the same as the result of performing the same operation on plaintext data. The homomorphic encryption algorithm may include additive homomorphic encryption algorithms and multiplicative homomorphic encryption algorithms, etc. For example, the second party can generate a public-private key pair for homomorphic encryption; the public key in the public-private key pair can be used to encrypt the first gradient value and the second gradient value.

[0057] Step S105: The second party sends the first gradient value ciphertext and the second gradient value ciphertext corresponding to each sample identifier to the first party.

[0058] Step S107: The first party receives the first gradient value ciphertext and the second gradient value ciphertext corresponding to each sample identifier.

[0059] Step S109: Within each subset, the first party homomorphically adds the first gradient value ciphertexts of multiple sample identifiers to obtain the first feature value ciphertext of the subset, and homomorphically adds the second gradient value ciphertexts of multiple sample identifiers to obtain the second feature value ciphertext of the subset.

[0060] In some embodiments, after step S101, the first party can obtain multiple subsets, each subset including multiple sample identifiers. For each subset, the first party can homomorphically add the ciphertexts of the first gradient values ​​corresponding to the multiple sample identifiers in the subset to obtain the ciphertext of the first feature value of the subset; and can homomorphically add the ciphertexts of the second gradient values ​​corresponding to the multiple sample identifiers in the subset to obtain the ciphertext of the second feature value of the subset.

[0061] For example, a subset may include m sample identifiers x1, x2, ..., xi, ..., xm. The ciphertexts of the first gradient values ​​corresponding to the sample identifiers x1, x2, ..., xi, ..., xm are E(g(x1)), E(g(x2)), ..., E(g(xi)), ..., E(g(xm)), respectively, and the ciphertexts of the second gradient values ​​corresponding to the sample identifiers x1, x2, ..., xi, ..., xm are E(h(x1)), E(h(x2)), ..., E(h(xi)), ..., E(h(xm)), respectively. Then, the first party can calculate E(g(x1))+E(g(x2))+,...,+E(g(xi))+,...,+E(g(xm))=E(g(x1)+g(x2)+,...,+g(xi)+,...,+g(xm)) as the first feature value ciphertext of the subset; and can calculate E(h(x1))+E(h(x2))+,...,+E(h(xi))+,...,+E(h(xm))=E(h(x1)+h(x2)+,...,+h(xi)+,...,+h(xm)) as the second feature value ciphertext of the subset.

[0062] Step S111: The first party uses random numbers to mask the first feature value ciphertext and the second feature value ciphertext respectively, to obtain the masked first feature value ciphertext and the masked second feature value ciphertext.

[0063] In some embodiments, by masking the first feature value ciphertext and the second feature value ciphertext, the second party can be prevented from obtaining the first feature value ciphertext and the second feature value ciphertext, thereby preventing the second party from obtaining the first feature value and the second feature value from the first feature value ciphertext and the second feature value ciphertext, thus enhancing privacy protection.

[0064] In some embodiments, for each subset of first and second feature value ciphertexts, the first party may use any of the following methods to mask the first and second feature value ciphertexts corresponding to that subset.

[0065] Method 1:

[0066] The first feature value ciphertext is masked using only random numbers, resulting in a masked first feature value ciphertext; the second feature value ciphertext is also masked using only random numbers, resulting in a masked second feature value ciphertext. Thus, in subsequent step S119, the second party can calculate the segmentation gain factor.

[0067] The first party can use homomorphic encryption to perform homomorphic encryption on the random number, obtaining a ciphertext random number. The ciphertext random number can then be homomorphically processed with a first feature value ciphertext and a second feature value ciphertext to obtain masked first and second feature value ciphertexts. The homomorphic processing can include homomorphic addition, homomorphic multiplication, and any combination thereof. For example, the first party can use the second party's public key to perform homomorphic encryption on the random number.

[0068] For example, the first feature value ciphertext can be E(g), and the masked first feature value ciphertext can be E(gr). The second feature value ciphertext can be E(h), and the masked second feature value ciphertext can be E((h+λ)×r). 2 r represents a random number, and λ represents the regularization coefficient.

[0069] Method 2:

[0070] The first feature value ciphertext is masked using random numbers and first noise data to obtain the masked first feature value ciphertext; the second feature value ciphertext is then masked using random numbers to obtain the masked second feature value ciphertext. The first noise data can be a random number with a small value. This allows the second party to calculate a segmentation gain factor with limited precision in subsequent step S119. It is worth noting that because the first noise data is a random number with a small value, the segmentation gain factor with limited precision can meet business requirements.

[0071] The specific concealment process is similar to the previous method one, and will not be repeated here.

[0072] Method 3:

[0073] The first feature value ciphertext is masked using a random number and first noise data to obtain the masked first feature value ciphertext; the second feature value ciphertext is masked using a random number and second noise data to obtain the masked second feature value ciphertext. The first noise data can be a random number with a small value, and the second noise data can be another random number with a small value. In this way, the second party can calculate a precision-limited segmentation gain factor in the subsequent step S119. It is worth noting that because the first noise data is a random number with a small value, and the second noise data is another random number with a small value, the precision-limited segmentation gain factor can meet the business requirements.

[0074] The specific concealment process is similar to the previous method one, and will not be repeated here.

[0075] For example, the first feature value ciphertext can be E(g), and the masked first feature value ciphertext can be E(gr+s1). The second feature value ciphertext can be E(h), and the masked second feature value ciphertext can be E((h+λ)×r). 2 +s2). r represents a random number, λ represents the regularization coefficient, s1 represents the first noisy data, and s2 represents the first noisy data.

[0076] Method 4:

[0077] The first feature value ciphertext is masked using a random number, resulting in a masked first feature value ciphertext. The second feature value ciphertext is then masked using a random number and second noise data, resulting in a masked second feature value ciphertext. The second noise data can be a random number with a small value. This allows the second party to calculate a precision-limited segmentation gain factor in subsequent step S119. It is worth noting that because the first noise data is a random number with a small value, the precision-limited segmentation gain factor can meet the business requirements.

[0078] The specific concealment process is similar to the previous method one, and will not be repeated here.

[0079] Step S113: The first party sends the first feature value ciphertext and the second feature value ciphertext corresponding to each subset to the second party.

[0080] Step S115: The second party receives the first feature value ciphertext and the second feature value ciphertext after masking for each subset.

[0081] Step S117: The second party decrypts the first feature value ciphertext and the second feature value ciphertext after masking, respectively, to obtain the first feature value and the second feature value after masking.

[0082] In some embodiments, the second party can decrypt the masked first feature value ciphertext and the masked second feature value ciphertext corresponding to each subset to obtain the masked first feature value and the masked second feature value corresponding to that subset. For example, the second party can use a private key to decrypt the masked first feature value ciphertext and the masked second feature value ciphertext.

[0083] Step S119: The second party uses the masked first feature value and the masked second feature value to calculate the segmentation gain factor, which is used to calculate the segmentation gain and is used to train the non-leaf nodes of the data processing model.

[0084] In some embodiments, for each subset, the second party can perform calculations on the masked first feature value and masked second feature value corresponding to the subset according to a preset algorithm to obtain a segmentation gain factor for the subset. The segmentation gain factor can be used to calculate the segmentation gain, which can be used to measure the degree of order among multiple specific samples, including samples corresponding to sample identifiers within the subset. The segmentation gain can include at least one of the following: information gain, information gain ratio, and Gini coefficient. Those skilled in the art should understand that the segmentation gain is not limited to the information gain, information gain ratio, and Gini coefficient listed above; in practice, the segmentation gain can vary depending on the training algorithm.

[0085] For example, the first feature value ciphertext after masking for a certain subset can be E(gr), and the second feature value ciphertext after masking for the same subset can be E((h+λ)×r). 2 By decryption, the first masked feature value corresponding to this subset can be gr, and the second masked feature value corresponding to this subset can be (h+λ)×r. 2 The second method can calculate the split gain factor.

[0086] For example, the first ciphertext of the masked feature value corresponding to a certain subset can be E(gr+s1), and the second ciphertext of the masked feature value corresponding to the subset can be E((h+λ)×r). 2 +s2). By decryption, the first feature value after masking corresponding to this subset can be gr+s1, and the second feature value after masking corresponding to this subset can be (h+λ)×r. 2 +s2. The second method can calculate the split gain factor. Since both the first noise data s1 and the second noise data s2 are small random numbers, therefore and They are approximately equal.

[0087] In some embodiments, the second party may also calculate the segmentation gain of a subset based on the segmentation gain factor of each subset. The second party may select subsets based on the segmentation gains of each subset, and then determine the splitting conditions of non-leaf nodes based on the selected subsets. For example, the second party may select the subset with the largest segmentation gain. Of course, the second party may also jointly calculate the segmentation gain of a subset with the first party based on the segmentation gain factor of each subset.

[0088] The model training methods in some embodiments of this specification can enhance data privacy protection during multi-party collaborative modeling by using random numbers to mask the ciphertext of feature values.

[0089] This specification provides another embodiment of the model training method.

[0090] The described model training method can be used to train a non-leaf node in a forest model, where the non-leaf node can be the root node or an internal node. In practical applications, using this model training method recursively, all non-leaf nodes in the forest model can be trained, thereby achieving cooperative and safe modeling. The model training method can be applied to a first party, which can hold the feature data of the samples.

[0091] Please see Figure 3 The model training method may include the following steps.

[0092] Step S21: Based on the feature data, divide the sample identifier set into multiple subsets.

[0093] Step S23: Receive the first gradient value ciphertext and the second gradient value ciphertext corresponding to each sample identifier.

[0094] Step S25: Within each subset, homomorphically add the first gradient value ciphertexts of multiple sample identifiers to obtain the first feature value ciphertext of the subset, and homomorphically add the second gradient value ciphertexts of multiple sample identifiers to obtain the second feature value ciphertext of the subset.

[0095] Step S27: Use random numbers to mask the first feature value ciphertext and the second feature value ciphertext respectively, to obtain the masked first feature value ciphertext and the masked second feature value ciphertext;

[0096] Step S29: Send the first feature value ciphertext and the second feature value ciphertext corresponding to each subset to the second party so as to train the non-leaf nodes of the data processing model.

[0097] The model training methods in some embodiments of this specification enhance data privacy protection during multi-party collaborative modeling by masking feature value ciphertext with random numbers based on homomorphic encryption algorithms.

[0098] This specification provides another embodiment of the model training method. The model training method can be used to train a non-leaf node in a forest model, where the non-leaf node can be the root node or an internal node. In practical applications, using this model training method, a recursive approach can be adopted to train each non-leaf node in the forest model, thereby achieving cooperative and safe modeling. The model training method can be applied to a second party, which can hold the labeled data of the samples.

[0099] Please see Figure 4The model training method may include the following steps.

[0100] Step S31: Receive the first feature value ciphertext and the second feature value ciphertext after masking corresponding to the subset. The subset is obtained by segmenting the sample identifier set, which includes multiple sample identifiers.

[0101] Step S33: Decrypt the first feature value ciphertext and the second feature value ciphertext after masking, respectively, to obtain the first feature value and the second feature value after masking.

[0102] Step S35: Calculate the segmentation gain factor using the masked first feature value and the masked second feature value. The segmentation gain factor is used to calculate the segmentation gain of the subset. The segmentation gain is used to train the non-leaf nodes of the data processing model.

[0103] The model training methods in some embodiments of this specification enhance data privacy protection during multi-party collaborative modeling by masking feature value ciphertext with random numbers based on homomorphic encryption algorithms.

[0104] This specification provides an embodiment of a model training apparatus applied to a first party that holds feature data of samples. Please refer to... Figure 5 The device may include the following units.

[0105] The segmentation unit 41 is used to segment the sample identifier set into multiple subsets based on feature data, wherein the sample identifier set includes the identifiers of multiple samples;

[0106] The receiving unit 43 is used to receive the first gradient value ciphertext and the second gradient value ciphertext corresponding to each sample identifier. The first gradient value ciphertext and the second gradient value ciphertext are obtained by encrypting the first gradient value and the second gradient value of the loss function by homomorphic encryption algorithm, respectively.

[0107] The addition unit 45 is used to homomorphically add the first gradient value ciphertexts of multiple sample identifiers within each subset to obtain the first feature value ciphertext of the subset, and homomorphically add the second gradient value ciphertexts of multiple sample identifiers to obtain the second feature value ciphertext of the subset.

[0108] The masking unit 47 is used to mask the first feature value ciphertext and the second feature value ciphertext respectively using random numbers to obtain the masked first feature value ciphertext and the masked second feature value ciphertext.

[0109] The sending unit 49 is used to send the first feature value ciphertext and the second feature value ciphertext corresponding to each subset to the second party so as to facilitate the training of the non-leaf nodes of the data processing model.

[0110] This specification provides an embodiment of a model training apparatus for use by a second party holding labeled sample data. Please refer to... Figure 6 The device may include the following units.

[0111] The receiving unit 51 is used to receive the first feature value ciphertext and the second feature value ciphertext after masking corresponding to the subset. The subset is obtained by segmenting the sample identifier set, which includes multiple sample identifiers.

[0112] The decryption unit 53 is used to decrypt the first feature value ciphertext and the second feature value ciphertext after masking, respectively, to obtain the first feature value and the second feature value after masking.

[0113] The calculation unit 55 is used to calculate a segmentation gain factor using the masked first feature value and the masked second feature value. The segmentation gain factor is used to calculate the segmentation gain of the subset. The segmentation gain is used to train the non-leaf nodes of the data processing model.

[0114] The following describes an embodiment of the electronic device described in this manual. Figure 7 This is a schematic diagram of the hardware structure of the electronic device in this embodiment. For example... Figure 7 As shown, the electronic device may include one or more (only one is shown in the figure) processors, memory, and transmission modules. Of course, those skilled in the art will understand that... Figure 7 The hardware structure shown is for illustrative purposes only and does not limit the hardware structure of the aforementioned electronic device. In practice, the electronic device may include more than [specific hardware structures]. Figure 7 Showing more or fewer component units; or, having the same as Figure 7 The different configurations shown.

[0115] The memory may include high-speed random access memory; or it may include non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. Of course, the memory may also include remotely configured network memory. The remotely configured network memory can be connected to the blockchain client via networks such as the Internet, corporate intranets, local area networks, or mobile communication networks. The memory can be used to store program instructions or modules of application software, such as those described in this specification. Figure 3 or Figure 4 The program instructions or modules corresponding to the embodiments.

[0116] The processor can be implemented in any suitable manner. For example, the processor can take the form of a microprocessor or processor and a computer-readable medium storing computer-readable program code (e.g., software or firmware) executable by the (micro)processor, logic gates, switches, application-specific integrated circuits (ASICs), programmable logic controllers, and embedded microcontrollers, etc. The processor can read and execute program instructions or modules in the memory.

[0117] The transmission module can be used to transmit data via a network, such as the Internet, corporate intranet, local area network, or mobile communication network.

[0118] This specification also provides an embodiment of a computer storage medium. The computer storage medium includes, but is not limited to, Random Access Memory (RAM), Read-Only Memory (ROM), cache, hard disk drive (HDD), memory card, etc. The computer storage medium stores computer program instructions. When the computer program instructions are executed, they implement: this specification. Figure 3 or Figure 4 The program instructions or modules corresponding to the embodiments.

[0119] It should be noted that the various embodiments in this specification are described in a progressive manner, and the same or similar parts between the various embodiments can be referred to mutually. Each embodiment focuses on describing the differences from other embodiments. In particular, for method embodiments implemented on one side (e.g. Figure 3 and Figure 4 The corresponding embodiments (device embodiments, electronic device embodiments, and computer storage medium embodiments) are basically similar to the method embodiments, so the descriptions are relatively simple, and relevant details can be found in the descriptions of the method embodiments. Furthermore, it is understood that those skilled in the art, after reading this specification, can conceive of any combination of some or all of the embodiments listed in this specification without creative effort, and such combinations are also within the scope of disclosure and protection of this specification.

[0120] In the 1990s, improvements to a technology could be clearly distinguished as either hardware improvements (e.g., improvements to the circuit structure of diodes, transistors, switches, etc.) or software improvements (improvements to the methodology). However, with technological advancements, many methodological improvements today can be considered direct improvements to the hardware circuit structure. Designers almost always obtain the corresponding hardware circuit structure by programming the improved methodology into the hardware circuit. Therefore, it cannot be said that a methodological improvement cannot be implemented using hardware physical modules. For example, a Programmable Logic Device (PLD) (such as a Field Programmable Gate Array (FPGA)) is such an integrated circuit whose logic function is determined by the user programming the device. Designers can program and "integrate" a digital system onto a PLD themselves, without needing chip manufacturers to design and manufacture dedicated integrated circuit chips. Furthermore, nowadays, instead of manually manufacturing integrated circuit chips, this programming is mostly implemented using "logic compiler" software. Similar to the software compiler used in program development, the original code before compilation must also be written in a specific programming language, called a Hardware Description Language (HDL). There are many HDLs, such as ABEL (Advanced Boolean Expression Language), AHDL (Altera Hardware Description Language), Confluence, CUPL (Cornell University Programming Language), HDCal, JHDL (Java Hardware Description Language), Lava, Lola, MyHDL, PALASM, and RHDL (Ruby Hardware Description Language). Currently, the most commonly used are VHDL (Very-High-Speed ​​Integrated Circuit Hardware Description Language) and Verilog2. Those skilled in the art should also understand that by simply performing some logic programming on the method flow using one of these hardware description languages ​​and programming it into an integrated circuit, the hardware circuit implementing the logical method flow can be easily obtained.

[0121] The systems, devices, modules, or units described in the above embodiments can be implemented by computer chips or entities, or by products with certain functions. A typical implementation device is a computer. Specifically, a computer can be, for example, a personal computer, laptop computer, cellular phone, camera phone, smartphone, personal digital assistant, media player, navigation device, email device, game console, tablet computer, wearable device, or any combination of these devices.

[0122] As can be seen from the above description of the embodiments, those skilled in the art can clearly understand that this specification can be implemented by means of software plus necessary general-purpose hardware platforms. Based on this understanding, the technical solutions of this specification, in essence or the parts that contribute to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in the various embodiments or some parts of the embodiments of this specification.

[0123] This manual can be used in a wide range of general-purpose or special-purpose computer system environments or configurations. Examples include: personal computers, server computers, handheld or portable devices, tablet devices, multiprocessor systems, microprocessor-based systems, set-top boxes, programmable consumer electronics, network PCs, minicomputers, mainframe computers, and distributed computing environments including any of the above systems or devices.

[0124] This specification can be described in the general context of computer-executable instructions that are executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, etc., that perform a specific task or implement a specific abstract data type. This specification can also be practiced in distributed computing environments, where tasks are performed by remote processing devices connected via a communication network. In distributed computing environments, program modules can reside in local and remote computer storage media, including storage devices.

[0125] Although this specification has been described by way of examples, those skilled in the art will recognize that many variations and modifications are possible without departing from the spirit of this specification, and it is intended that the appended claims cover such variations and modifications without departing from the spirit of this specification.

Claims

1. A model training method applied to a first party, the first party holding feature data of samples but not label data of samples, the method comprising: Based on the feature data, the sample identifier set is divided into multiple subsets, wherein the sample identifier set includes multiple sample identifiers; Receive the first gradient value ciphertext and the second gradient value ciphertext corresponding to each sample identifier. The first gradient value ciphertext and the second gradient value ciphertext are obtained by encrypting the first gradient value and the second gradient value of the loss function respectively using a homomorphic encryption algorithm. Within each subset, the first gradient value ciphertexts of multiple sample identifiers are homomorphically added to obtain the first feature value ciphertext of that subset, and the second gradient value ciphertexts of multiple sample identifiers are homomorphically added to obtain the second feature value ciphertext of that subset. The first and second feature value ciphertexts are masked using random numbers to obtain the masked first and second feature value ciphertexts. The first and second ciphertexts of the masked features corresponding to each subset are sent to the second party so that the second party can decrypt them to obtain the masked first and second features. Using the masked first and second features, a segmentation gain factor is calculated. The segmentation gain factor is used to calculate the segmentation gain of the subsets, and the segmentation gain is used to train the non-leaf nodes of the data processing model. The second party holds the label data of the samples but does not hold the feature data of the samples.

2. The method of claim 1, wherein the data processing model includes a forest model, the forest model includes at least one tree model, and the tree model includes at least two non-leaf nodes; The first gradient value is the first-order gradient value, and the second gradient value is the second-order gradient value.

3. The method as described in claim 1, wherein the feature data of the sample includes multiple sub-data, each sub-data corresponding to a dimension; the step of dividing the sample identifier set into multiple subsets includes: The sample identifier set is divided into multiple subsets based on sub-data of at least one dimension.

4. The method as described in claim 1, wherein masking the first feature value ciphertext and the second feature value ciphertext using random numbers respectively includes: Homomorphic encryption is applied to the random number to obtain the ciphertext of the random number; Perform homomorphic operations on the random number ciphertext with the first feature value ciphertext and the second feature value ciphertext respectively to obtain the masked first feature value ciphertext and the masked second feature value ciphertext. The homomorphic operations include one or any combination of the following: homomorphic addition and homomorphic multiplication.

5. The method as described in claim 1, wherein random numbers are used to mask the first feature value ciphertext and the second feature value ciphertext respectively in any of the following ways: Method 1: The first feature value ciphertext is masked using only random numbers to obtain the masked first feature value ciphertext; the second feature value ciphertext is masked using only random numbers to obtain the masked second feature value ciphertext. Method 2: The first feature value ciphertext is masked using random numbers and first noise data to obtain the masked first feature value ciphertext; the second feature value ciphertext is masked using random numbers to obtain the masked second feature value ciphertext. Method 3: The first feature value ciphertext is masked using random numbers and first noise data to obtain the masked first feature value ciphertext; the second feature value ciphertext is masked using random numbers and second noise data to obtain the masked second feature value ciphertext. Method 4: The first feature value ciphertext is masked using random numbers to obtain the masked first feature value ciphertext; the second feature value ciphertext is masked using random numbers and second noise data to obtain the masked second feature value ciphertext.

6. The method as described in claim 1, wherein the first feature value ciphertext is The first feature value ciphertext after masking is The second feature value ciphertext is The masked second feature value ciphertext is ; It is a random number. is the coefficient of the regularization term.

7. The method as described in claim 1, wherein the first feature value ciphertext is The first feature value ciphertext after masking is The second feature value ciphertext is The masked second feature value ciphertext is ; It is a random number. The coefficient of the regularization term, This is the first noise data. This is the second noise data.

8. A model training method applied to a second party, the second party holding label data of samples but not feature data of samples, the method comprising: The system receives a subset from a first party, consisting of a masked first feature value ciphertext and a masked second feature value ciphertext. The subset is obtained by segmenting a sample identifier set, which includes multiple sample identifiers. The first party holds the feature data of the samples but not their label data. The masked first feature value ciphertext corresponding to the subset is calculated based on the first gradient value ciphertext of the sample identifiers within that subset. The masked second feature value ciphertext corresponding to the subset is calculated based on the second gradient value ciphertext of the sample identifiers within that subset. The first and second gradient value ciphertexts are obtained by encrypting the first and second gradient values ​​of the loss function using a homomorphic encryption algorithm, respectively. The first and second gradient values ​​of the loss function are calculated based on the sample label data. Decrypt the first feature value ciphertext and the second feature value ciphertext after masking respectively to obtain the first feature value and the second feature value after masking. Using the masked first feature value and the masked second feature value, a segmentation gain factor is calculated. The segmentation gain factor is used to calculate the segmentation gain of a subset. The segmentation gain is used to train the non-leaf nodes of the data processing model.

9. The method of claim 8, wherein the data processing model includes a forest model, the forest model includes at least one tree model, and the tree model includes at least two non-leaf nodes.

10. The method of claim 8, wherein the segmentation gain is used to measure the degree of order of a plurality of specific samples, the specific samples including the samples corresponding to sample identifiers within a subset.

11. A model training apparatus, applied to a first party, the first party holding feature data of samples but not label data of samples, the apparatus comprising: A segmentation unit is used to segment a sample identifier set into multiple subsets based on feature data, wherein the sample identifier set includes identifiers of multiple samples; The receiving unit is used to receive the first gradient value ciphertext and the second gradient value ciphertext corresponding to each sample identifier. The first gradient value ciphertext and the second gradient value ciphertext are obtained by encrypting the first gradient value and the second gradient value of the loss function respectively by a homomorphic encryption algorithm. The addition unit is used to homomorphically add the first gradient value ciphertexts of multiple sample identifiers within each subset to obtain the first feature value ciphertext of the subset, and homomorphically add the second gradient value ciphertexts of multiple sample identifiers to obtain the second feature value ciphertext of the subset. The masking unit is used to mask the first feature value ciphertext and the second feature value ciphertext using random numbers, respectively, to obtain the masked first feature value ciphertext and the masked second feature value ciphertext. The sending unit is configured to send the masked first feature value ciphertext and the masked second feature value ciphertext corresponding to each subset to the second party, so that the second party can decrypt the masked first feature value ciphertext and the masked second feature value ciphertext to obtain the masked first feature value and the masked second feature value. Using the masked first feature value and the masked second feature value, a segmentation gain factor is calculated. The segmentation gain factor is used to calculate the segmentation gain of the subset. The segmentation gain is used to train the non-leaf nodes of the data processing model. The second party holds the label data of the samples, but does not hold the feature data of the samples.

12. A model training apparatus applied to a second party, the second party holding label data of samples but not feature data of samples, the apparatus comprising: The receiving unit is configured to receive, from the first party, a masked first feature value ciphertext and a masked second feature value ciphertext corresponding to a subset. The subset is obtained by segmenting a sample identifier set, which includes multiple sample identifiers. The first party holds the feature data of the samples but not the label data. The masked first feature value ciphertext corresponding to the subset is calculated based on the first gradient value ciphertext of the sample identifiers within the subset. The masked second feature value ciphertext corresponding to the subset is calculated based on the second gradient value ciphertext of the sample identifiers within the subset. The first gradient value ciphertext and the second gradient value ciphertext are obtained by encrypting the first gradient value and the second gradient value of the loss function using a homomorphic encryption algorithm, respectively. The first gradient value and the second gradient value of the loss function are calculated based on the label data of the samples. The decryption unit is used to decrypt the first feature value ciphertext and the second feature value ciphertext after masking, respectively, to obtain the first feature value and the second feature value after masking; The computational unit is used to calculate a segmentation gain factor using the masked first feature value and the masked second feature value. The segmentation gain factor is used to calculate the segmentation gain of a subset, and the segmentation gain is used to train the non-leaf nodes of the data processing model.

13. An electronic device, comprising: At least one processor; A memory storing program instructions configured to be executed by the at least one processor, the program instructions including instructions for performing the method as claimed in any one of claims 1-10.

Images