Method and system for generating machine learning models and method for processing user data

Abstract

The present disclosure relates to a method for generating a machine learning model, a system and a method for processing user data, and relates to the field of computer technology. According to the label ciphertext of the sample and the user classification to which each user data sample belongs, the gradient information ciphertext of different user classifications is calculated; the label provider calculates the information gain of different user classifications according to the gradient information ciphertext of different user classifications; the label provider instructs the data provider to generate a machine learning model according to the information gain of different user classifications. The technical solution disclosed in the present disclosure can ensure the privacy of the data provider and the label provider, thereby improving security.

Description

Technical Field

[0001] The present disclosure relates to the field of computer technology, and in particular to a method for generating a machine learning model, a system for generating a machine learning model, a method for processing user data, a device for processing user data, an electronic device, and a non-volatile computer-readable storage medium. Background Art

[0002] Data-driven artificial intelligence technology is playing a huge role in all walks of life and bringing extremely high value. China has listed data as a new production factor. In recent years, data privacy protection and data compliance have also received increasing attention, which has made data unable to communicate with each other and resulted in data islands. The data island phenomenon hinders the realization of data potential.

[0003] In order to solve this problem, the concept of federated learning came into being. Federated learning enables multiple participants to jointly model their data without leaving their local data, ensuring data privacy while stimulating the data potential of all parties.

[0004] In related technologies, methods such as SecureBoost can be used to create machine learning models. Summary of the invention

[0005] The inventors of the present disclosure have discovered that the above-mentioned related technologies have the following problems: the privacy of the data provider and the label provider cannot be guaranteed, resulting in poor security.

[0006] In view of this, the present disclosure proposes a technical solution for generating a machine learning model, which can ensure the privacy of data providers and label providers, thereby improving security.

[0007] According to some embodiments of the present disclosure, a method for generating a machine learning model is provided, including: a data provider, generating a label prediction value ciphertext of each user data sample according to the parameter information ciphertext of the machine learning model sent by a label provider; the data provider, calculating the gradient information ciphertext of different user categories according to the label prediction value ciphertext, the label ciphertext of each user data sample sent by the label provider, and the user category to which each user data sample belongs; the label provider, calculating the information gain of different user categories according to the gradient information ciphertext of different user categories; the label provider, instructing the data provider to generate a machine learning model according to the information gain of different user categories.

[0008] In some embodiments, the generation method includes multiple rounds of generation processes, and generating the label prediction value ciphertext of each user data sample includes: the data provider, in the Tth round of generation process, generates the label prediction value ciphertext of the T-1th round of generation process based on the parameter information ciphertext in the T-1th round of generation process and the label prediction value ciphertext in the T-2th round of generation process.

[0009] In some embodiments, generating the label prediction value ciphertext of the T-1th round generation process includes: a data provider generates the label prediction value ciphertext of the T-1th round generation process based on the result of homomorphic addition of the parameter information ciphertext of the T-1th round generation process and the label prediction value ciphertext of the T-2th round generation process.

[0010] In some embodiments, the generation method also includes: the data provider, according to multiple user feature information contained in each user data sample, divides each user data sample into different user classification sets; wherein the generation method includes multiple rounds of generation process, and calculating the gradient information ciphertext of different user classifications includes: the data provider, in the Tth round of generation process, calculates the gradient information ciphertext of each user data sample according to the label prediction value ciphertext of each user data sample in the T-1th round of generation process, and the label ciphertext of each user data sample; and respectively calculates the weighted sum of the gradient information ciphertext of each user data sample in each user classification set as the gradient information ciphertext of different user classifications.

[0011] In some embodiments, the gradient information ciphertext includes a first-order gradient ciphertext and a second-order gradient ciphertext.

[0012] In some embodiments, calculating the gradient information ciphertext of each user data sample includes: the data provider calculates the homomorphic multiplication result of the first preset value and the label ciphertext of each user data sample; the data provider calculates the homomorphic addition result of the label prediction value ciphertext of each user data sample in the T-1th round generation process and the homomorphic multiplication result to determine the first-order gradient ciphertext.

[0013] In some embodiments, calculating the gradient information ciphertext of each user data sample includes: the data provider uses the sigmod function to process the label prediction value ciphertext of each user data sample in the T-1 round generation process to determine the function processing result; the data provider calculates the homomorphic multiplication result of the first preset value and the label ciphertext of each user data sample; calculates the homomorphic addition result of the function processing result and the homomorphic multiplication result to determine the first-order gradient ciphertext.

[0014] In some embodiments, calculating the information gain of different user classifications includes: a label provider calculates the information gain of each user classification based on the weighted sum of the first-order gradient ciphertexts of each user data sample in each user classification set, the weighted sum of the second-order gradient ciphertexts of each user data sample in each user classification set, the weighted sum of the first-order gradient ciphertexts of all user data samples, and the weighted sum of the second-order gradient ciphertexts of all user data samples.

[0015] In some embodiments, calculating the information gain of each user classification includes: a label provider calculates the information gain of each user classification based on the difference between the weighted sum of the first gain, the second gain, and the third gain and a threshold, the threshold is used to instruct the data provider to generate a machine learning model, the first gain is positively correlated with the weighted sum of the first-order gradient ciphertexts of each user data sample in each user classification set, and negatively correlated with the weighted sum of the second-order gradient ciphertexts of each user data sample in each user classification set, the second gain is positively correlated with the difference between the weighted sum of the first-order gradient ciphertexts of all user data samples and the weighted sum of the first-order gradient ciphertexts of each user data sample in each user classification set, and positively correlated with the difference between the weighted sum of the second-order gradient ciphertexts of all user data samples and the weighted sum of the second-order gradient ciphertexts of each user data sample in each user classification set, the third gain is negatively correlated with the weighted first-order gradient ciphertexts of all user data samples, and positively correlated with the weighted sum of the second-order gradient ciphertexts of all user data samples.

[0016] In some embodiments, dividing each user data sample into different user classification sets includes: the data provider sets multiple classification thresholds based on multiple user feature information contained in each user data sample; and dividing each user data sample into different user classification sets based on the multiple classification thresholds.

[0017] In some embodiments, instructing a data provider to generate a machine learning model based on the information gains of different user categories includes: a label provider determining the maximum gain information among the information gains of different user categories; and instructing the data provider to generate a machine learning model based on whether the maximum gain information is greater than a threshold.

[0018] In some embodiments, the machine learning model is a tree model, and based on whether the maximum gain information is greater than a threshold, instructing the data provider to generate the machine learning model includes: the label provider, when the maximum gain information is greater than the threshold, instructing the data provider to split the current node of the tree model, and generate a child node of the current node as the current node of the next iteration; the label provider, when the maximum gain information is less than or equal to the threshold, calculates the weight parameters of the child nodes as the plain text parameter information of the tree model; determining the maximum gain information in the information gain of different user classifications includes: the label provider, based on the current node of the next iteration, re-determining the maximum gain information; repeating the above steps until the tree model is generated.

[0019] According to other embodiments of the present disclosure, a method for processing user data is provided, including: classifying users according to the user data using a machine learning model, wherein the machine learning model is generated according to a method for generating a machine learning model in any of the above embodiments.

[0020] According to some other embodiments of the present disclosure, a system for generating a machine learning model is provided, including: a data provider, for generating a label prediction value ciphertext of each user data sample based on the parameter information ciphertext of the machine learning model sent by a label provider, and calculating the gradient information ciphertext of different user classifications based on the label prediction value ciphertext, the label ciphertext of each user data sample sent by the label provider, and the user classification to which each user data sample belongs; a label provider, for calculating the information gain of different user classifications based on the gradient information ciphertexts of different user classifications, and instructing the data provider to generate a machine learning model based on the information gain of different user classifications.

[0021] In some embodiments, the generation method includes multiple rounds of generation processes. In the T-th round of generation process, the data provider generates the label prediction value ciphertext of the T-1th round of generation process based on the parameter information ciphertext of the T-1th round of generation process and the label prediction value ciphertext of the T-2th round of generation process.

[0022] In some embodiments, the data provider generates the label prediction value ciphertext of the T-1th round generation process based on the result of homomorphic addition of the parameter information ciphertext of the T-1th round generation process and the label prediction value ciphertext of the T-2th round generation process.

[0023] In some embodiments, the data provider divides each user data sample into different user classification sets based on multiple user feature information contained in each user data sample; wherein the generation method includes multiple rounds of generation processes, and the data provider, in the Tth round of generation process, calculates the gradient information ciphertext of each user data sample based on the label prediction value ciphertext of each user data sample in the T-1th round of generation process and the label ciphertext of each user data sample; and calculates the weighted sum of the gradient information ciphertext of each user data sample in each user classification set as the gradient information ciphertext of different user classifications.

[0024] In some embodiments, the gradient information ciphertext includes a first-order gradient ciphertext and a second-order gradient ciphertext.

[0025] In some embodiments, the data provider calculates the homomorphic multiplication result of the first preset value and the label ciphertext of each user data sample; the data provider calculates the homomorphic addition result of the label prediction value ciphertext of each user data sample in the T-1th round generation process and the homomorphic multiplication result to determine the first-order gradient ciphertext.

[0026] In some embodiments, the data provider uses the sigmod function to process the label prediction value ciphertext of each user data sample in the T-1 round generation process to determine the function processing result; the data provider calculates the homomorphic multiplication result of the first preset value and the label ciphertext of each user data sample; calculates the homomorphic addition result of the function processing result and the homomorphic multiplication result to determine the first-order gradient ciphertext.

[0027] In some embodiments, the label provider calculates the information gain of each user classification based on the weighted sum of the first-order gradient ciphertexts of each user data sample in each user classification set, the weighted sum of the second-order gradient ciphertexts of each user data sample in each user classification set, the weighted sum of the first-order gradient ciphertexts of all user data samples, and the weighted sum of the second-order gradient ciphertexts of all user data samples.

[0028] In some embodiments, the label provider calculates the information gain of each user classification based on the difference between the weighted sum of the first gain, the second gain, and the third gain and the threshold, where the threshold is used to instruct the data provider to generate a machine learning model, the first gain is positively correlated with the weighted sum of the first-order gradient ciphertexts of each user data sample in each user classification set, and negatively correlated with the weighted sum of the second-order gradient ciphertexts of each user data sample in each user classification set, the second gain is positively correlated with the difference between the weighted sum of the first-order gradient ciphertexts of all user data samples and the weighted sum of the first-order gradient ciphertexts of each user data sample in each user classification set, and positively correlated with the difference between the weighted sum of the second-order gradient ciphertexts of all user data samples and the weighted sum of the second-order gradient ciphertexts of each user data sample in each user classification set, the third gain is negatively correlated with the weighted first-order gradient ciphertexts of all user data samples, and positively correlated with the weighted sum of the second-order gradient ciphertexts of all user data samples.

[0029] In some embodiments, the data provider sets multiple classification thresholds according to multiple user feature information included in each user data sample; and divides each user data sample into different user classification sets according to the multiple classification thresholds.

[0030] In some embodiments, the label provider determines the maximum gain information among the information gains of different user classifications; and instructs the data provider to generate a machine learning model based on whether the maximum gain information is greater than a threshold.

[0031] In some embodiments, the machine learning model is a tree model. When the maximum gain information is greater than a threshold, the label provider instructs the data provider to split the current node of the tree model to generate a child node of the current node as the current node of the next iteration; when the maximum gain information is less than or equal to the threshold, the label provider calculates the weight parameters of the child nodes as the plain text parameter information of the tree model; the label provider redetermines the maximum gain information based on the current node of the next iteration; and the above steps are repeated until the tree model is generated.

[0032] According to some further embodiments of the present disclosure, there is provided a device for processing user data, comprising: a classification unit, for classifying users according to the user data using a machine learning model, wherein the machine learning model is generated according to the method for generating a machine learning model of any of the above-mentioned embodiments.

[0033] In some embodiments, an electronic device is provided, comprising: a memory; and a processor coupled to the memory, the processor being configured to execute the method for generating a machine learning model or the method for processing user data in any of the above-mentioned embodiments based on instructions stored in the memory device.

[0034] According to some further embodiments of the present disclosure, a non-volatile computer-readable storage medium is provided, on which a computer program is stored, and when the program is executed by a processor, the method for generating a machine learning model or the method for processing user data in any of the above embodiments is implemented.

[0035] In the above embodiment, the data provider and the label provider realize the secure calculation of information gain through the interaction of parameter information ciphertext, label prediction value ciphertext and gradient information ciphertext. In this way, the privacy of the data provider and the label provider can be guaranteed, thereby improving security. BRIEF DESCRIPTION OF THE DRAWINGS

[0036] The accompanying drawings, which constitute a part of the specification, illustrate embodiments of the present disclosure and, together with the description, serve to explain the principles of the present disclosure.

[0037] The present disclosure may be more clearly understood from the following detailed description with reference to the accompanying drawings:

[0038] Figure 1 A flowchart showing some embodiments of the method for generating a machine learning model of the present disclosure;

[0039] Figure 2 A flowchart showing some embodiments of the method for processing user data disclosed herein;

[0040] Figure 3 A schematic diagram showing some embodiments of the method for generating a machine learning model of the present disclosure;

[0041] Figure 4a A block diagram showing some embodiments of a system for generating a machine learning model of the present disclosure;

[0042] Figure 4b A block diagram showing some embodiments of a system for generating a machine learning model of the present disclosure;

[0043] Figure 5 A block diagram showing some embodiments of the electronic device of the present disclosure;

[0044] Figure 6 A block diagram showing some other embodiments of the electronic device of the present disclosure. DETAILED DESCRIPTION

[0045] Various exemplary embodiments of the present disclosure will now be described in detail with reference to the accompanying drawings. It should be noted that the relative arrangement of components and steps, numerical expressions and numerical values ​​set forth in these embodiments do not limit the scope of the present disclosure unless otherwise specifically stated.

[0046] At the same time, it should be understood that for the convenience of description, the sizes of the various parts shown in the drawings are not drawn according to the actual proportional relationship.

[0047] The following description of at least one exemplary embodiment is merely illustrative in nature and is in no way intended to limit the present disclosure, its application, or uses.

[0048] Technologies, methods, and equipment known to ordinary technicians in the relevant art may not be discussed in detail, but where appropriate, the technologies, methods, and equipment should be considered part of the specification.

[0049] In all examples shown and discussed herein, any specific values ​​should be interpreted as merely exemplary and not as limiting. Therefore, other examples of the exemplary embodiments may have different values.

[0050] It should be noted that like reference numerals and letters refer to similar items in the following figures, and therefore, once an item is defined in one figure, it need not be further discussed in subsequent figures.

[0051] As mentioned above, the present disclosure considers joint modeling between two parties. A new scenario in which one party has only data sample labels and the other party has data sample features. The AP (Active Party) is the label provider, that is, the business party, which only has data labels and hopes to use the data of other organizations for joint modeling. It is usually the active party of federated learning; the organization that provides data to the AP party is the PP (Passive Party), that is, the data provider, and is usually the passive party of federated learning. The above practical application scenario is referred to as the single-party sub-model scenario.

[0052] Compared with the existing technology application scenarios, the single-party sub-model scenario puts forward new requirements for achieving the interpretability of the jointly constructed federated XBG model.

[0053] On the one hand, in order to protect the data privacy of the PP party, the AP party cannot obtain the prediction value corresponding to the specific sample The reason is that in order to obtain interpretability, the AP needs to obtain the characteristic meaning of the model node. At this time, if the AP obtains the sample prediction value The AP will also obtain the predicted path of the sample in the model, which will expose the characteristic information of each sample and further infer the sample privacy information of the PP.

[0054] On the other hand, in order to protect the data privacy of the AP party, the PP party cannot obtain the private data of the AP party, namely the sample labels and the leaf node weight information of the model, because the leaf node weight of the model contains the sample label data information of the AP party.

[0055] However, methods such as SecureBoost cannot simultaneously meet the above two requirements in a single-party sub-model scenario. Although methods such as SecureBoost ensure that the AP retains the model weight parameters, the AP still needs to obtain a single sample prediction value to calculate the information gain. When the AP knows the split characteristics of the entire model, this behavior will cause the AP to infer the PP's sample data through a single sample prediction value, and thus the data privacy of both parties cannot be guaranteed, which limits the application scenarios of federated learning.

[0056] That is to say, when methods such as SecureBoost perform interpretable joint XGB (eXtremeGradient Boosting) modeling in a single-party sub-model scenario, one party needs to use a single sample label y and a single sample prediction value Calculate information gain and use it to further split and train the model. If the AP calculates information gain, the AP will infer the PP's sample privacy information through a single sample prediction value, which does not meet the requirements for protecting the privacy of PP's data features; if the PP calculates information gain, the PP will obtain the sample prediction value and sample label, which does not meet the requirements for protecting the privacy of AP's data labels.

[0057] In view of the fact that related technologies are unable to achieve interpretability and protect the privacy information of both parties in a joint modeling problem in a unilateral sub-model scenario, this disclosure proposes an interpretable unilateral sub-model federated learning privacy protection scheme.

[0058] In summary, the present disclosure disassembles the information gain calculation formula and designs a secure multi-party information gain calculation protocol based on a unilateral sub-model scenario. Based on the protocol, an interpretable unilateral sub-model federated learning scheme is designed. Finally, the interpretable model splitting and training in the unilateral sub-model scenario is completed without leaking the privacy of both parties. For example, the technical solution of the present disclosure can be implemented through the following embodiments.

[0059] Figure 1 A flowchart illustrating some embodiments of the method for generating a machine learning model of the present disclosure.

[0060] like Figure 1As shown, in step 110, the data provider generates a ciphertext of the label prediction value of each user data sample based on the ciphertext of the parameter information of the machine learning model sent by the label provider (ie, the business party).

[0061] For example, the user data sample set X∈R n×d , the label set of user data samples y={y1,y2,...,y n}∈R n In the T-th round of generation, the T-th tree model Tree is generated T , that is, each round of generation process generates a tree model, and the label prediction value set is The label provider encrypts y and generates a label ciphertext set [y] = {[y1], [y2], ..., [y n ]}, and send [y] to the data provider.

[0062] In some embodiments, the data provider obtains the encrypted public key pk generated by the label provider, as well as the correspondence between the sample prediction value and the model leaf node; the label provider generates an encrypted public-private key pair (sk, pk) and has the parameter information of the T-1th tree Sample label y = {y1,y2,...,y n}, It is the weight information of the leaf node of the T-1th tree.

[0063] For example, the label provider uses the public key pk to generate the sample label ciphertext [y] = Enc pk (y), the parameter information ciphertext of the T-1th tree And [y], Send to the data party.

[0064] In some embodiments, the initialized Tree T Only the root node Node-0 is included. During the i-th iteration of the T-th generation process, the point Node-i is generated. Each node Node-i needs to maintain the following information: Feature meaning k i , split strategy split i (), sample set R i Initialize the sample set of Node-0 to R i =I, I contains all user data samples, and the node set to be split L = {Node-0}.

[0065] For example, the feature meaning k i Open to label providers, split strategy i (), sample set R iThey are retained by the data provider and are not disclosed to the label provider.

[0066] In some embodiments, the generation method includes multiple rounds of generation processes. In the T-th round of generation process, the data provider generates the label prediction value ciphertext of the T-1th round of generation process based on the parameter information ciphertext of the T-1th round of generation process and the label prediction value ciphertext of the T-2th round of generation process.

[0067] For example, the data provider generates the label prediction value ciphertext of the T-1th round generation process based on the result of homomorphic addition of the parameter information ciphertext of the T-1th round generation process and the label prediction value ciphertext of the T-2th round generation process.

[0068] For example, the data provider uses the above correspondence to calculate the sample prediction value ciphertext For example, if the tree model structure is a binary tree model, the correspondence between the sample prediction value and the model leaf node is the weight parameter of the left and right child nodes of the current node and the child node to which the sample is finally classified. Based on this correspondence, the sample prediction value can be obtained. In this way, the data provider does not need to obtain the weight parameters of the model node, but can use the correspondence and the ciphertext sent by the label provider to perform calculations.

[0069] In step 120, the data provider calculates the gradient information ciphertext of different user categories based on the label prediction value ciphertext, the label ciphertext of each user data sample sent by the label provider, and the user category to which each user data sample belongs.

[0070] In some embodiments, the data provider divides each user data sample into different user classification sets according to multiple user feature information contained in each user data sample.

[0071] In some embodiments, the data provider sets multiple classification thresholds according to multiple user feature information included in each user data sample; and divides each user data sample into different user classification sets according to the multiple classification thresholds.

[0072] For example, for each user feature information dimension k=1,…d value of the data provider, the user data sample set X is divided into q-1 user classification sets (which can be called bins) according to q classification thresholds. In this way, the user classification sets I1,…,I2 corresponding to each user feature information are obtained. q-1 .

[0073] In some embodiments, the generation method includes multiple rounds of generation processes. In the Tth round of generation process, the data provider calculates the gradient information ciphertext of each user data sample based on the label prediction value ciphertext of each user data sample in the T-1th round of generation process and the label ciphertext of each user data sample; and calculates the weighted sum of the gradient information ciphertext of each user data sample in each user classification set as the gradient information ciphertext of different user classifications.

[0074] In some embodiments, the gradient information ciphertext includes a first-order gradient ciphertext and a second-order gradient ciphertext. For example, the data provider uses [y], Calculate each user data sample X i The first-order gradient ciphertext [g i ] and the second-order gradient ciphertext [h i ].

[0075] In some embodiments, the data provider calculates the homomorphic multiplication result of the first preset value (such as -1) and the label ciphertext of each user data sample; the data provider calculates the homomorphic addition result of the label prediction value ciphertext of each user data sample in the T-1th round generation process and the homomorphic multiplication result to determine the first-order gradient ciphertext.

[0076] For example, if the mean square error is used as the loss function, the first-order gradient and the second-order gradient are:

[0077] h i =1

[0078] The data provider can calculate the first-order gradient ciphertext and the second-order gradient ciphertext:

[0079] [h i ]=[1]

[0080] is homomorphic subtraction, is a homomorphic multiplication.

[0081] In some embodiments, the data provider uses the sigmod function to process the label prediction value ciphertext of each user data sample in the T-1 round generation process to determine the function processing result; the data provider calculates the homomorphic multiplication result of the first preset value and the label ciphertext of each user data sample; calculates the homomorphic addition result of the function processing result and the homomorphic multiplication result to determine the first-order gradient ciphertext.

[0082] For example, if the cross entropy is used as the loss function, the first-order gradient and second-order gradient calculated according to the calculation formula in the previous article are:

[0083]

[0084] The data provider can approximate the Sigmod function by performing Taylor expansion at 0 and then find the first-order gradient and the second-order gradient.

[0085]

[0086]

[0087]

[0088] In step 130, the label provider calculates the information gain of different user categories based on the gradient information ciphertexts of different user categories.

[0089] In some embodiments, the label provider calculates the information gain of each user classification based on the weighted sum of the first-order gradient ciphertexts of each user data sample in each user classification set, the weighted sum of the second-order gradient ciphertexts of each user data sample in each user classification set, the weighted sum of the first-order gradient ciphertexts of all user data samples, and the weighted sum of the second-order gradient ciphertexts of all user data samples.

[0090] In some embodiments, the data provider calculates the sub-first-order gradient aggregate value ciphertext within the user classification set j Sum of Second-Order Gradient Aggregate Values And the total first-order gradient aggregate value ciphertext [G] = ∑ i∈I [g i ] and the total second-order gradient aggregation value ciphertext [H] = ∑ i∈I [h i ] and send it to the label provider.

[0091] In some embodiments, the label provider calculates the information gain of each user classification based on the difference between the weighted sum of the first gain, the second gain, and the third gain and the threshold, where the threshold is used to instruct the data provider to generate a machine learning model, the first gain is positively correlated with the weighted sum of the first-order gradient ciphertexts of each user data sample in each user classification set, and negatively correlated with the weighted sum of the second-order gradient ciphertexts of each user data sample in each user classification set, the second gain is positively correlated with the difference between the weighted sum of the first-order gradient ciphertexts of all user data samples and the weighted sum of the first-order gradient ciphertexts of each user data sample in each user classification set, and positively correlated with the difference between the weighted sum of the second-order gradient ciphertexts of all user data samples and the weighted sum of the second-order gradient ciphertexts of each user data sample in each user classification set, the third gain is negatively correlated with the weighted first-order gradient ciphertexts of all user data samples, and positively correlated with the weighted sum of the second-order gradient ciphertexts of all user data samples.

[0092] For example, the label provider uses the private key sk to decrypt the sub-first-order gradient aggregation value ciphertext and obtains G j =Decsk ([G j ]); Use the private key sk to decrypt the sub-second-order gradient aggregation value ciphertext and obtain H j =Dec sk ([H ( ]); Use the private key sk to decrypt the total first-order gradient aggregation value ciphertext to obtain G; Use the private key sk to decrypt the total second-order gradient aggregation value ciphertext to obtain H; Use the above decryption results to calculate the first gain Second gain Third gain Then calculate the information gain of user classification set j:

[0093]

[0094] λ is an adjustable parameter. j The maximum gain information MaxGain is determined in.

[0095] In step 140, the label provider instructs the data provider to generate a machine learning model based on the information gain of different user classifications.

[0096] In some embodiments, the label provider determines the maximum gain information among the information gains of different user classifications; and instructs the data provider to generate a machine learning model based on whether the maximum gain information is greater than a threshold γ.

[0097] In some embodiments, the machine learning model is a tree model. When the maximum gain information is greater than a threshold, the label provider instructs the data provider to split the current node of the tree model to generate a child node of the current node as the current node of the next iteration; when the maximum gain information is less than or equal to the threshold, the label provider calculates the weight parameters of the child nodes as the plain text parameter information of the tree model; the label provider redetermines the maximum gain information based on the current node of the next iteration; and the above steps are repeated until the tree model is generated.

[0098] For example, the iterative processing in the T-th round generation process can be implemented as follows:

[0099] While L is not NULL do

[0100] Select any node Node-i∈L and use the secure multi-party information gain calculation protocol to obtain the maximum information gain:

[0101] Data provider

[0102] If MaxGain>γ then

[0103] Split node i according to the splitting rule corresponding to the maximum information gain to obtain two child nodes i L and i R and its corresponding sample set

[0104] Update the tree model Tree T , remove node i from L, and replace i L and i R Add to the set L to be split.

[0105] else

[0106] -Do not split, remove node i from L

[0107] end

[0108] end

[0109] In some embodiments, the tag provider provides j , H ( Calculate the parameter information for round T. For example, the optimal weight of each leaf node:

[0110]

[0111] In the above embodiment, the calculation process of the first-order gradient g (the first-order gradient term of the loss function) and the second-order gradient h (the second-order gradient term of the loss function) on which the information gain in XGB depends is disassembled.

[0112] By using homomorphic encryption, in the scenario where the label provider only has the data label and the data provider has the data features, the label provider provides the sample label and model weight parameter ciphertext required for calculation; the data provider uses the weight ciphertext to calculate the sample prediction value, and further calculates the g and h ciphertext aggregation values ​​of different splitting schemes; using the above method, the label provider and the data provider jointly and securely calculate the information gain (Gain); finally, the tree is split to complete the modeling process.

[0113] In this way, the label provider can obtain the split feature meaning and model weight parameters without exposing the privacy information of both parties during the model training and prediction stages, ensuring the model interpretability and data privacy, and providing a safe and reliable explainable federated learning solution for new practical application scenarios.

[0114] The following description will be made using a two-party joint scenario as an example. The technical solution disclosed herein can also be extended to a multi-party joint scenario with more than two parties for application.

[0115] Figure 2 A flowchart showing some embodiments of the method for processing user data of the present disclosure.

[0116] like Figure 2 As shown, in step 210, user data is obtained. In step 220, users are classified according to the user data using a machine learning model, where the machine learning model is generated according to the method for generating a machine learning model in any of the above embodiments.

[0117] The data provider has a sample label set y = {y1, y2, ..., y n}, and the encrypted result [y] = {[y1], [y2], ..., [y n ]} is sent to the data provider.

[0118] In some embodiments, during the tree model training process, the data provider also needs to set the leaf node weights of the model (taking a single-layer binary tree as an example) ( represents the weight of the jth leaf node of the Tth tree) and encrypts the encrypted result Sent to the data provider in sequence.

[0119] Since the data provider holds all data features in the unilateral sub-model scenario, it can know which leaf node weight the prediction value of each training sample is equal to. According to the weight ciphertext of the leaf node sent by the label provider, the label prediction value ciphertext of each training sample on each tree model can be obtained.

[0120] In some embodiments, the data provider may combine the sample label ciphertext value [y] provided by the label provider with the obtained label prediction value ciphertext The first-order gradient and second-order gradient ciphertext are obtained according to the first-order gradient and second-order gradient calculation formulas of different loss functions.

[0121] For example, for complex loss functions involving exponential operations that exceed the computing capabilities of homomorphic encryption algorithms, Taylor expansion and other techniques are used to approximate them into simple linear function calculations.

[0122] Finally, the data provider aggregates the first-order gradient ciphertext and the second-order gradient ciphertext according to different splitting schemes, and sends the ciphertext to the label provider. The label provider decrypts and calculates the information gain of different splitting schemes; selects the optimal splitting scheme based on the information gain, and notifies the data provider to split according to this scheme, completing a single tree node splitting training iteration. Repeat the leaf node splitting process to complete the construction of the tree model.

[0123] The above embodiment can calculate the encrypted first-order gradient and the encrypted second-order gradient without knowing the plain text of the leaf node weight, thereby ensuring the data privacy security of the label provider. Moreover, the label provider can only obtain the aggregate value, but cannot obtain a single predicted value, which also ensures the data privacy of the data provider.

[0124] In some embodiments, the first protocol has two participants: the tag provider only has A The label collection of the data label They hope to use data from other organizations (data providers) for joint modeling, so they are usually the active party in federated learning. The data provider has n P Data collection of data items and its d-dimensional feature X i =(x1,x2,x3,…,x d ).

[0125] In some embodiments, the technical solution of the present disclosure includes two stages.

[0126] For example, in the first stage, encrypted sample alignment is performed. Since all participants in the vertical federation scenario use different data features of common samples for federated learning, sample alignment is required first. That is, find the common sample ID owned by the label provider and the data provider. This part can be implemented using the existing more mature privacy set intersection protocol. After the privacy set intersection, both parties will obtain n data intersections.

[0127] For example, in the second phase, federated model training is performed. Federated modeling begins after the encrypted sample alignment step is achieved.

[0128] Figure 3 Schematic diagram showing some embodiments of the method for generating a machine learning model of the present disclosure.

[0129] like Figure 3 As shown in Figure 1, in step 0, the tag provider generates a homomorphic encryption (Paillier algorithm) public key pk and a private key sk, and broadcasts the public key pk. The tag provider uses the private key sk to encrypt the n data tags y in the intersection Enc pk (y) Generate ciphertext [y] and send the public key pk and the generated ciphertext to the data provider.

[0130] In step 1, after the data provider determines all sample IDs of the split node (current node), it determines q classification thresholds S based on the d-dimensional features of the samples. k ={s k1 ,…,s kq}; According to the q classification thresholds, all samples are divided into q-1 user classification sets as the screening basis for the tree splitting nodes.

[0131] Taking the T-th round generation process of building the T-th tree training as an example, the training phase is mainly divided into the following steps.

[0132] In step 2, the data provider uses the public key pk to weight the leaf nodes of the tree model (taking a single-layer binary tree as an example) trained in the previous generation process. Encrypt and send the encrypted results in order Sent to the data provider.

[0133] In step 3, the data provider can obtain the corresponding relationship between the sample prediction value and the leaf node of the tree model according to the process of building the tree model in the previous round of generation process training.

[0134] In step 4, the data provider uses the encrypted prediction value result of the previous round of generation process in round T-2 to perform homomorphic addition operation with the encrypted result received in step 1, and combines the corresponding relationship between the sample prediction value and the leaf node of the tree model to obtain the encrypted prediction value of the specific sample in this round. For example, the calculation is as follows:

[0135]

[0136] It is homomorphic addition, that is, the addition operation on the ciphertext, and its result is equal to the encrypted value of the sum of the plaintext.

[0137] In step 5, the data listener uses the encrypted sample label [y] and the encrypted sample prediction value The first-order gradient of sample i [g i ] and the second-order gradient [h i ] to perform calculations.

[0138] In some embodiments, since there are only addition, subtraction, and multiplication operations in the above calculation expression, the data provider can directly use the additive homomorphism and multiplication homomorphism of the ciphertext to calculate and obtain the corresponding ciphertext.

[0139] For example, if the mean square error is used as the loss function, the first-order gradient and the second-order gradient are:

[0140] h i =1

[0141] The data provider can calculate the first-order gradient ciphertext and the second-order gradient ciphertext:

[0142] [h i ]=[1]

[0143] is homomorphic subtraction, is a homomorphic multiplication.

[0144] In some embodiments, for computational expressions containing more complex operations such as power operations and rational number division operations, Taylor expansion can be used to perform second-order approximation on the expression and convert it into a form that can be operated on by homomorphic encryption.

[0145] For example, if the cross entropy is used as the loss function, the first-order gradient and second-order gradient calculated according to the calculation formula in the previous article are:

[0146]

[0147] The data provider can approximate the Sigmod function by performing Taylor expansion at 0 and then find the first-order gradient and the second-order gradient.

[0148]

[0149]

[0150]

[0151] In step 6, based on the encrypted first-order gradient and encrypted second-order gradient of a single sample obtained in step 1 and step 5, the encrypted aggregate value of the first-order gradient and second-order gradient of each user classification set is calculated using the additive homomorphism of the encryption algorithm, and the result is sent back to the label provider for decryption. The formula is as follows.

[0152] In step 7, the label provider obtains the first-order gradient aggregation value G of each user classification set of the data provider according to the decryption. j =Dec sk ([G j ]) and the second-order gradient aggregation value H j =Dec sk ([H j ]), calculate the information gain Gain of each feature and each split of the data provider.

[0153] The label provider sends the maximum information gain corresponding scheme number back to the data provider. The data provider determines the optimal split feature and feature threshold based on the sequence number scheme and retrograde splitting. This data is kept by the data provider during the training process and is not disclosed to the label provider to ensure the privacy of the data provider.

[0154] If the information gain of the optimal split is less than the threshold γ, the current node is not split; otherwise, the feature corresponding to the maximum gain information value is the optimal split feature, and the current node is split according to the corresponding feature and threshold; finally, the PP party needs to calculate the sample set of the corresponding two leaf nodes and split the next leaf node based on the sample set.

[0155] In step 8, for each leaf node of the Tth tree, step 7 is executed repeatedly until the condition for stopping splitting is met, that is, all leaf nodes can no longer be split or the depth of the tree reaches the set maximum depth.

[0156] For the first-order gradient aggregation value and the second-order gradient aggregation value of the AP holding the optimal split of the node, the optimal weight of each leaf node can be calculated:

[0157]

[0158] At this point, the construction of the Tth tree is completed.

[0159] In step 9, after the label provider completes the training of the Tth tree, it will Send it to the data provider and repeat the single tree model training process from step 1 to step 9 until a total of M trees are built. The resulting complete gradient boosting tree model is:

[0160]

[0161] The unilateral sub-model is a very important business scenario. Aiming at this scenario, the present disclosure proposes a secure and explainable federated XGB sub-model modeling scheme.

[0162] In the above embodiment, the label provider retains the feature meaning and model weight parameters, while ensuring that the label provider cannot infer the data provider's data; the model weight parameters are retained by the label provider and kept confidential to the data provider, ensuring that the data listener cannot infer the label provider's privacy information through the model weight parameters; the label provider cannot know the sample prediction value of the data provider, ensuring that the label provider cannot infer the data provider's privacy information through the sample prediction value. Thus, a reliable interpretable model is achieved.

[0163] In the above embodiment, a secure multi-party information gain calculation protocol is designed. By disassembling the calculation formula of the first-order gradient and the second-order gradient in the information gain, the label provider provides the sample labels and model weight parameter ciphertexts required for the calculation; the data provider uses the weight ciphertext to calculate the sample prediction value, and further calculates the g and h ciphertext aggregation values ​​of different splitting schemes, and jointly and securely calculates the information gain.

[0164] In this way, calculations can be performed for different loss functions. For functions that cannot be directly calculated using additive homomorphic encryption, they can be approximated using techniques such as Taylor expansion, and the protocol can be extended to multi-party scenarios.

[0165] Figure 4a A block diagram illustrating some embodiments of a system for generating a machine learning model of the present disclosure.

[0166] like Figure 4a As shown, the machine learning model generation system 4a includes: a data provider 41a, which is used to generate a label prediction value ciphertext for each user data sample based on the parameter information ciphertext of the machine learning model sent by the label provider, and calculate the gradient information ciphertext of different user categories based on the label prediction value ciphertext, the label ciphertext of each user data sample sent by the label provider, and the user category to which each user data sample belongs; a label provider 42a, which calculates the information gain of different user categories based on the gradient information ciphertext of different user categories, and instructs the data provider to generate a machine learning model based on the information gain of different user categories.

[0167] In some embodiments, the generation method includes multiple rounds of generation processes. In the T-th round of generation process, the data provider 41a generates the label prediction value ciphertext of the T-1th round of generation process based on the parameter information ciphertext of the T-1th round of generation process and the label prediction value ciphertext of the T-2th round of generation process.

[0168] In some embodiments, the data provider 41a generates the label prediction value ciphertext of the T-1th round generation process based on the result of homomorphic addition of the parameter information ciphertext of the T-1th round generation process and the label prediction value ciphertext of the T-2th round generation process.

[0169] In some embodiments, the data provider 41a divides each user data sample into different user classification sets according to multiple user feature information contained in each user data sample; wherein the generation method includes multiple rounds of generation processes, and the data provider 41a, in the Tth round of generation process, calculates the gradient information ciphertext of each user data sample according to the label prediction value ciphertext of each user data sample in the T-1th round of generation process and the label ciphertext of each user data sample; and calculates the weighted sum of the gradient information ciphertext of each user data sample in each user classification set as the gradient information ciphertext of different user classifications.

[0170] In some embodiments, the gradient information ciphertext includes a first-order gradient ciphertext and a second-order gradient ciphertext.

[0171] In some embodiments, the data provider 41a calculates the homomorphic multiplication result of the first preset value and the label ciphertext of each user data sample; the data provider calculates the homomorphic addition result of the label prediction value ciphertext of each user data sample in the T-1th round generation process and the homomorphic multiplication result to determine the first-order gradient ciphertext.

[0172] In some embodiments, the data provider 41a uses the sigmoid function to process the label prediction value ciphertext of each user data sample in the T-1 round generation process to determine the function processing result; the data provider calculates the homomorphic multiplication result of the first preset value and the label ciphertext of each user data sample; calculates the homomorphic addition result of the function processing result and the homomorphic multiplication result to determine the first-order gradient ciphertext.

[0173] In some embodiments, the label provider 42a calculates the information gain of each user classification based on the weighted sum of the first-order gradient ciphertexts of each user data sample in each user classification set, the weighted sum of the second-order gradient ciphertexts of each user data sample in each user classification set, the weighted sum of the first-order gradient ciphertexts of all user data samples, and the weighted sum of the second-order gradient ciphertexts of all user data samples.

[0174] In some embodiments, the label provider 42a calculates the information gain of each user classification based on the difference between the weighted sum of the first gain, the second gain, and the third gain and the threshold, where the threshold is used to instruct the data provider to generate a machine learning model, the first gain is positively correlated with the weighted sum of the first-order gradient ciphertexts of each user data sample in each user classification set, and negatively correlated with the weighted sum of the second-order gradient ciphertexts of each user data sample in each user classification set, the second gain is positively correlated with the difference between the weighted sum of the first-order gradient ciphertexts of all user data samples and the weighted sum of the first-order gradient ciphertexts of each user data sample in each user classification set, and positively correlated with the difference between the weighted sum of the second-order gradient ciphertexts of all user data samples and the weighted sum of the second-order gradient ciphertexts of each user data sample in each user classification set, the third gain is negatively correlated with the weighted first-order gradient ciphertexts of all user data samples, and positively correlated with the weighted sum of the second-order gradient ciphertexts of all user data samples.

[0175] In some embodiments, the data provider 41a sets multiple classification thresholds according to multiple user feature information included in each user data sample; and divides each user data sample into different user classification sets according to the multiple classification thresholds.

[0176] In some embodiments, the label provider 42a determines the maximum gain information among the information gains of different user classifications; and instructs the data provider to generate a machine learning model based on whether the maximum gain information is greater than a threshold.

[0177] In some embodiments, the machine learning model is a tree model. The label provider 42a, when the maximum gain information is greater than a threshold, instructs the data provider to split the current node of the tree model and generate a child node of the current node as the current node of the next iteration; the label provider, when the maximum gain information is less than or equal to the threshold, calculates the weight parameters of the child nodes as the plain text parameter information of the tree model; the label provider 42a redetermines the maximum gain information based on the current node of the next iteration; and repeats the above steps until the tree model is generated.

[0178] Figure 4b A block diagram illustrating some embodiments of a system for generating a machine learning model of the present disclosure.

[0179] like Figure 4b As shown, the user data processing device 4b includes: a classification unit 41b, which is used to classify users according to the user data using a machine learning model. In some embodiments, the machine learning model provides an electronic device, including: a memory; and a processor coupled to the memory, the processor is configured to execute the method for generating the machine learning model in any of the above embodiments, or the method for processing user data based on instructions stored in the memory device.

[0180] Figure 5 A block diagram showing some embodiments of the electronic device of the present disclosure.

[0181] like Figure 5 As shown, the electronic device 5 of this embodiment includes: a memory 51 and a processor 52 coupled to the memory 51, and the processor 52 is configured to execute the method for generating a machine learning model or the method for processing user data in any embodiment of the present disclosure based on instructions stored in the memory 51.

[0182] The memory 51 may include, for example, a system memory, a fixed non-volatile storage medium, etc. The system memory may store, for example, an operating system, an application program, a boot loader, a database, and other programs.

[0183] Figure 6 A block diagram showing some other embodiments of the electronic device of the present disclosure.

[0184] like Figure 6 As shown, the electronic device 6 of this embodiment includes: a memory 610 and a processor 620 coupled to the memory 610, and the processor 620 is configured to execute the method for generating a machine learning model or the method for processing user data in any of the aforementioned embodiments based on instructions stored in the memory 610.

[0185] The memory 610 may include, for example, a system memory, a fixed non-volatile storage medium, etc. The system memory may store, for example, an operating system, an application program, a boot loader, and other programs.

[0186] The electronic device 6 may also include an input / output interface 630, a network interface 640, a storage interface 650, etc. These interfaces 630, 640, 650, the memory 610, and the processor 620 may be connected, for example, via a bus 660. The input / output interface 630 provides a connection interface for input / output devices such as a display, a mouse, a keyboard, a touch screen, a microphone, and a speaker. The network interface 640 provides a connection interface for various networked devices. The storage interface 650 provides a connection interface for external storage devices such as an SD card and a USB flash drive.

[0187] Those skilled in the art will appreciate that the embodiments of the present disclosure may be provided as methods, systems, or computer program products. Therefore, the present disclosure may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Furthermore, the present disclosure may take the form of a computer program product implemented on one or more computer-usable non-transient storage media including but not limited to disk storage, CD-ROM, optical storage, etc., which contain computer-usable program code.

[0188] So far, the method for generating a machine learning model, the system for generating a machine learning model, the method for processing user data, the device for processing user data, the electronic device, and the non-volatile computer-readable storage medium according to the present disclosure have been described in detail. In order to avoid obscuring the concept of the present disclosure, some details known in the art are not described. Based on the above description, those skilled in the art can fully understand how to implement the technical solution disclosed herein.

[0189] The method and system of the present disclosure may be implemented in many ways. For example, the method and system of the present disclosure may be implemented by software, hardware, firmware, or any combination of software, hardware, and firmware. The above order of steps for the method is for illustration only, and the steps of the method of the present disclosure are not limited to the order specifically described above, unless otherwise specifically stated. In addition, in some embodiments, the present disclosure may also be implemented as a program recorded in a recording medium, which includes machine-readable instructions for implementing the method according to the present disclosure. Therefore, the present disclosure also covers a recording medium storing a program for executing the method according to the present disclosure.

[0190] Although some specific embodiments of the present disclosure have been described in detail by way of example, it should be understood by those skilled in the art that the above examples are for illustration only and are not intended to limit the scope of the present disclosure. It should be understood by those skilled in the art that the above embodiments may be modified without departing from the scope and spirit of the present disclosure. The scope of the present disclosure is defined by the appended claims.

Claims

1. A method for generating a machine learning model, comprising: The data provider generates a ciphertext of the label prediction value of each user data sample according to the ciphertext of the parameter information of the machine learning model sent by the label provider, wherein the ciphertext of the parameter information includes the ciphertext of the model weight parameter; The data provider calculates the gradient information ciphertexts of different user classifications according to the label prediction value ciphertext, the label ciphertexts of the user data samples sent by the label provider, and the user classifications to which the user data samples belong; The label provider calculates information gains of different user categories according to the gradient information ciphertexts of the different user categories; The label provider instructs the data provider to generate the machine learning model based on the information gain of the different user classifications.

2. The generation method according to claim 1, wherein: The generation method comprises multiple rounds of generation process. The generating of the ciphertext of the label prediction value of each user data sample includes: The data provider, during the T-th round generation process, generates the label prediction value ciphertext of the T-1th round generation process based on the parameter information ciphertext of the T-1th round generation process and the label prediction value ciphertext of the T-2th round generation process.

3. The generation method according to claim 2, wherein: The generation of the label prediction value ciphertext of the T-1th round generation process includes: The data provider generates the label prediction value ciphertext of the T-1th round generation process according to the result of homomorphic addition of the parameter information ciphertext of the T-1th round generation process and the label prediction value ciphertext of the T-2th round generation process.

4. The generation method according to claim 1, further comprising: The data provider divides each user data sample into different user classification sets according to multiple user feature information included in each user data sample; The generation method includes multiple rounds of generation process, and the calculation of gradient information ciphertexts of different user classifications includes: The data provider, in the T-th round generation process, calculates the gradient information ciphertext of each user data sample according to the label prediction value ciphertext of each user data sample in the T-1-th round generation process and the label ciphertext of each user data sample; The weighted sum of the gradient information ciphertexts of each user data sample in each user classification set is calculated respectively as the gradient information ciphertexts of the different user classifications.

5. The generation method according to claim 4, wherein: The gradient information ciphertext includes a first-order gradient ciphertext and a second-order gradient ciphertext.

6. The generation method according to claim 5, wherein: The step of calculating the gradient information ciphertext of each user data sample comprises: The data provider calculates a homomorphic multiplication result of a first preset value and a label ciphertext of each user data sample; The data provider calculates the homomorphic addition result of the label prediction value ciphertext of each user data sample in the T-1 round generation process and the homomorphic multiplication result to determine the first-order gradient ciphertext.

7. The generation method according to claim 5, wherein: The step of calculating the gradient information ciphertext of each user data sample comprises: The data provider uses the sigmod function to process the ciphertext of the label prediction value of each user data sample in the T-1 round generation process to determine the function processing result; The data provider calculates a homomorphic multiplication result of a first preset value and a label ciphertext of each user data sample; A homomorphic addition result of the function processing result and the homomorphic multiplication result is calculated to determine the first-order gradient ciphertext.

8. The generation method according to claim 5, wherein: The calculating of information gain of different user classifications includes: The label provider calculates the information gain of each user classification based on the weighted sum of the first-order gradient ciphertexts of each user data sample in each user classification set, the weighted sum of the second-order gradient ciphertexts of each user data sample in each user classification set, the weighted sum of the first-order gradient ciphertexts of all user data samples, and the weighted sum of the second-order gradient ciphertexts of all user data samples.

9. The generation method according to claim 5, wherein: The calculating the information gain of each user classification comprises: The label provider calculates the information gain of each user classification according to the difference between the weighted sum of the first gain, the second gain, and the third gain and a threshold, wherein the threshold is used to instruct the data provider to generate the machine learning model. The first gain is positively correlated with the weighted sum of the first-order gradient ciphertext of each user data sample in each user classification set, and negatively correlated with the weighted sum of the second-order gradient ciphertext of each user data sample in each user classification set. The second gain is positively correlated with the difference between the weighted sum of the first-order gradient ciphertexts of all user data samples and the weighted sum of the first-order gradient ciphertexts of each user data sample in each user classification set, and is positively correlated with the difference between the weighted sum of the second-order gradient ciphertexts of all user data samples and the weighted sum of the second-order gradient ciphertexts of each user data sample in each user classification set. The third gain is negatively correlated with the weighted sum of the first-order gradient ciphertexts of all the user data samples, and is positively correlated with the weighted sum of the second-order gradient ciphertexts of all the user data samples.

10. The generation method according to claim 4, wherein: The dividing the user data samples into different user classification sets includes: The data provider sets a plurality of classification thresholds according to a plurality of user characteristic information included in each of the user data samples; According to the multiple classification thresholds, the user data samples are divided into different user classification sets.

11. The generation method according to any one of claims 1 to 10, wherein: The instructing the data provider to generate the machine learning model according to the information gain of the different user classifications comprises: The label provider determines the maximum gain information among the information gains of the different user classifications; According to whether the maximum gain information is greater than a threshold, the data provider is instructed to generate the machine learning model.

12. The generation method according to claim 11, wherein: The machine learning model is a tree model, The instructing the data provider to generate the machine learning model according to whether the maximum gain information is greater than a threshold value comprises: The label provider, when the maximum gain information is greater than the threshold, instructs the data provider to split the current node of the tree model and generate a child node of the current node as the current node of the next iteration; The label provider, when the maximum gain information is less than or equal to the threshold, calculates the weight parameter of the child node as the parameter information plain text of the tree model; The determining of the maximum gain information among the information gains of different user classifications comprises: The label provider re-determines the maximum gain information according to the current node of the next iteration; The generating method further comprises: Repeat the above steps until the tree model is generated.

13. A method for processing user data, comprising: According to the user data, the users are classified using a machine learning model, wherein the machine learning model is generated according to the method for generating a machine learning model according to any one of claims 1 to 12.

14. A system for generating a machine learning model, comprising: The data provider is used to generate a label prediction value ciphertext of each user data sample according to the parameter information ciphertext of the machine learning model sent by the label provider, and calculate the gradient information ciphertext of different user classifications according to the label prediction value ciphertext, the label ciphertext of each user data sample sent by the label provider, and the user classification to which each user data sample belongs, wherein the parameter information ciphertext includes the model weight parameter ciphertext; The label provider calculates the information gain of different user classifications according to the gradient information ciphertexts of the different user classifications, and instructs the data provider to generate the machine learning model according to the information gain of the different user classifications.

15. A user data processing device, comprising: A classification unit is used to classify users according to user data using a machine learning model, wherein the machine learning model is generated according to the method for generating a machine learning model according to any one of claims 1 to 12.

16. An electronic device, comprising: Memory; and A processor coupled to the memory, the processor being configured to execute the method for generating a machine learning model as described in any one of claims 1 to 12, or the method for processing user data as described in claim 13, based on instructions stored in the memory.

17. A non-volatile computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the method for generating a machine learning model as described in any one of claims 1 to 12, or the method for processing user data as described in claim 13.

Images