Federal learning decision tree model training method and device

Abstract

The invention discloses a training method and device for a federal learning decision tree model, and relates to the technical field of big data. A specific embodiment of the method comprises the following steps: acquiring first credit characteristic data and a credit label of each user in a plurality of users; determining a first dissimilarity matrix according to the first credit feature data; receiving a second dissimilarity matrix sent by the data provider; determining global perception sample selection probability distribution according to the first dissimilarity matrix and the second dissimilarity matrix; and then cyclically executing the following processes: training to obtain a decision tree model: determining a user core set of the current node according to global perception sample selection probability distribution, then determining each sub-node after the current node is split according to the user core set, and taking each sub-node as the current node. According to the embodiment, the accuracy of model prediction can be improved, the performance of the model is improved, the optimization efficiency of the model and the training speed of the model are improved, and the training cost of the model is reduced.

Description

Technical Field

[0001] The present invention relates to the field of big data technology, and in particular to a training method and device for a federated learning decision tree model. Background Art

[0002] Among the various machine learning models, tree models play an important role in finance, risk control and other fields due to their strong interpretability and high reliability. Among them, the decision tree model based on information gain is a basic model with many application scenarios. However, when training a decision tree model based on information gain, the data features and data volume of a single data source are limited, and it is impossible to directly obtain data from other data sources, resulting in training; and when a node is split, it is necessary to calculate the information gain on a large amount of data, which results in high training costs. Summary of the invention

[0003] In view of this, an embodiment of the present invention provides a training method and device for a federated learning decision tree model, which can improve the accuracy of model prediction and enhance the performance of the model by expanding data features through joint learning. A core set is constructed using global perception probability, and the core set is used for model training, which can improve the optimization efficiency of the model and the training speed of the model and reduce the training cost of the model.

[0004] To achieve the above object, according to one aspect of an embodiment of the present invention, a method for training a federated learning decision tree model is provided, comprising:

[0005] Acquire first credit characteristic data and a credit label of each user among a plurality of users;

[0006] Determining a first dissimilarity matrix according to the first credit feature data;

[0007] Receiving a second dissimilarity matrix sent by a data provider and obtained based on second credit feature data of each user among the multiple users;

[0008] Determining a global perception sample selection probability distribution according to the first dissimilarity matrix and the second dissimilarity matrix;

[0009] The following process is executed repeatedly to train the decision tree model:

[0010] Determine the user core set of the current node according to the global perception sample selection probability distribution, the user sample set of the current node and the credit label of each user in the user sample set; determine the child nodes after the current node is split according to the first credit feature data, the second credit feature data and the user core set, and use each of the child nodes as the current node.

[0011] Optionally, determining a global perception sample selection probability distribution according to the first dissimilarity matrix and the second dissimilarity matrix includes:

[0012] Determining a dissimilarity measure matrix based on the first dissimilarity matrix and the second dissimilarity matrix;

[0013] Determine a representation probability matrix according to the dissimilarity measurement matrix and a preset model;

[0014] The global perception sample selection probability distribution is determined according to the characterization probability matrix.

[0015] Optionally, determining a user core set of the current node includes:

[0016] Determining a first selection probability of the current node according to the global perception sample selection probability and the user sample set of the current node;

[0017] Determine a predicted credit label for each user in the user sample set, so as to determine a second selection probability of the current node according to the predicted credit label and the credit label of each user in the user sample set;

[0018] Determine a sample selection probability according to the first selection probability and the second selection probability;

[0019] The user core set is determined from the user sample set according to the sample selection probability.

[0020] Optionally, determining a predicted credit label for each user in the user sample set includes:

[0021] Selecting a first core set from the user sample set according to the first selection probability;

[0022] The predicted credit label is determined according to the credit labels of each user in the first core set.

[0023] Optionally, determining the second selection probability of the current node includes:

[0024] Determining a residual value according to the predicted credit label and the credit label of each user in the user sample set;

[0025] The second selection probability is generated according to the residual value.

[0026] Optionally, determining each child node after the current node is split includes:

[0027] Determining a first information gain of each first credit feature in each splitting mode according to the first credit feature data and the credit label of each user in the user core set;

[0028] receiving each split user set obtained according to the second credit characteristic data and sent by the data provider, wherein the split user set includes a user identifier;

[0029] Determine a second information gain corresponding to the split user set according to the user identifiers in the split user set and the user core set;

[0030] The respective sub-nodes are determined according to the first information gain and the second information gain.

[0031] Optionally, determining each of the subnodes according to the first information gain and the second information gain includes:

[0032] Selecting a maximum information gain from the first information gain and the second information gain;

[0033] If the maximum information gain is the first information gain, obtaining a target first credit feature and a target splitting method corresponding to the maximum information gain, splitting the current node according to the target first credit feature and the target splitting method, and determining each child node;

[0034] If the maximum information gain is the second information gain, a target split user set corresponding to the maximum information gain is obtained, and each child node is determined according to the user sample set of the current node and the target split user set.

[0035] Another aspect of an embodiment of the present invention provides a training device for a federated learning decision tree model, including:

[0036] An acquisition module, which acquires first credit feature data and a credit label of each user among a plurality of users;

[0037] A first determination module determines a first dissimilarity matrix according to the first credit feature data;

[0038] A receiving module, receiving a second dissimilarity matrix obtained based on second credit feature data of each user among the multiple users and sent by a data provider;

[0039] A second determination module determines a global perception sample selection probability distribution according to the first dissimilarity matrix and the second dissimilarity matrix;

[0040] The training module cyclically executes the following process to train a decision tree model: determining the user core set of the current node according to the global perception sample selection probability distribution, the user sample set of the current node, and the credit label of each user in the user sample set; determining each child node after the current node is split according to the first credit feature data, the second credit feature data and the user core set, and taking each of the child nodes as the current node.

[0041] According to another aspect of an embodiment of the present invention, there is provided an electronic device, including:

[0042] one or more processors;

[0043] a storage device for storing one or more programs,

[0044] When the one or more programs are executed by the one or more processors, the one or more processors implement the training method of the federated learning decision tree model provided by the present invention.

[0045] According to another aspect of an embodiment of the present invention, a computer-readable medium is provided, on which a computer program is stored, and when the program is executed by a processor, the training method of the federated learning decision tree model provided by the present invention is implemented.

[0046] An embodiment of the above invention has the following advantages or beneficial effects: the training method of the federated learning decision tree model of the embodiment of the present invention determines the first dissimilarity matrix according to the first credit feature data of each user among multiple users, receives the second dissimilarity matrix obtained based on the second credit feature data sent by the data provider, obtains the global perception sample selection probability distribution based on the first dissimilarity matrix and the second dissimilarity matrix, and then constructs the user core set based on the global perception sample selection probability distribution in combination with the user sample set of the current node, uses the user core set and the first credit feature data and the second credit feature data to obtain each child node after the current node is split, and obtains the federated learning decision tree model after training. The method uses federated learning to obtain the second credit feature of the user from the data provider in compliance with regulations to achieve the purpose of expanding the features, which can make the prediction accuracy of the trained model higher and improve the model performance. The method reduces the space complexity and time complexity of the model training process by constructing the user core set and using the user core set for model training, while reducing the computational cost in the model training process and the communication burden in federated learning, reducing the training cost of the model and improving the application value of the model. This method determines the user core set by constructing a global perception sample selection probability and combining it with the user sample set of the current node. The user core set can better characterize the original user sample set, provide good data support for the establishment of each model, and is diverse and representative. It also improves the optimization efficiency of the model and the efficiency of model training.

[0047] The further effects of the above-mentioned non-conventional optional manner will be described below in conjunction with the specific implementation manner. BRIEF DESCRIPTION OF THE DRAWINGS

[0048] The accompanying drawings are used to better understand the present invention and do not constitute an improper limitation of the present invention.

[0049] Figure 1 is a schematic diagram of the main process of a training method for a federated learning decision tree model according to an embodiment of the present invention;

[0050] Figure 2 is a schematic diagram of the main process of another method for training a federated learning decision tree model according to an embodiment of the present invention;

[0051] Figure 3 is a schematic diagram of the main process of another method for training a federated learning decision tree model according to an embodiment of the present invention;

[0052] Figure 4 is a schematic diagram of the main process of another method for training a federated learning decision tree model according to an embodiment of the present invention;

[0053] Figure 5 is a schematic diagram of main modules of a training device for a federated learning decision tree model according to an embodiment of the present invention;

[0054] Figure 6 is an exemplary system architecture diagram to which embodiments of the present invention may be applied;

[0055] Figure 7 It is a schematic diagram of the structure of a computer system of a terminal device or a server suitable for implementing an embodiment of the present invention. DETAILED DESCRIPTION

[0056] The following is a description of exemplary embodiments of the present invention in conjunction with the accompanying drawings, including various details of the embodiments of the present invention to facilitate understanding, which should be considered as merely exemplary. Therefore, it should be recognized by those of ordinary skill in the art that various changes and modifications may be made to the embodiments described herein without departing from the scope and spirit of the present invention. Similarly, for clarity and conciseness, the description of well-known functions and structures is omitted in the following description.

[0057] It should be noted that the collection, collection, updating, analysis, processing, use, transmission, storage and other aspects of user personal information involved in the technical solution of this disclosure are in compliance with the provisions of relevant laws and regulations, are used for legitimate purposes, and do not violate public order and good morals. Necessary measures are taken for user personal information to prevent illegal access to user personal information data and maintain the security of user personal information, network security and national security.

[0058] Figure 1 is a schematic diagram of the main process of a training method for a federated learning decision tree model according to an embodiment of the present invention, such as Figure 1 As shown, the method comprises the following steps:

[0059] Step S101: obtaining first credit characteristic data and credit label of each user among a plurality of users;

[0060] Step S102: determining a first dissimilarity matrix according to the first credit feature data;

[0061] Step S103: receiving a second dissimilarity matrix obtained based on second credit feature data of each of the multiple users and sent by a data provider;

[0062] Step S104: determining a global perception sample selection probability distribution according to the first dissimilarity matrix and the second dissimilarity matrix;

[0063] Step S105: Determine the user core set of the current node according to the global perception sample selection probability distribution, the user sample set of the current node, and the credit label of each user in the user sample set; determine each child node after the current node is split according to the first credit feature data, the second credit feature data, and the user core set;

[0064] Step S106: Determine whether the stop condition is met; if so, execute step S107; if not, take each child node as the current node and execute step S105;

[0065] Step S107: After the training is completed, a federated learning decision tree model is obtained.

[0066] In an embodiment of the present invention, the federated learning decision tree model can be used in the fields of finance, risk control, etc. For example, it can be applied to the evaluation of user credit rating or risk assessment in loan business to determine the user's loan amount based on the user's credit rating or risk assessment results.

[0067] When training a decision tree model, first obtain the first credit feature data and credit label of each user from a plurality of users, and the plurality of users may constitute an original user sample set. Among them, the user may be an individual or an enterprise, and the credit label may be the credit rating of the user, such as a credit label may be a high credit rating, a low credit rating, or other ratings. The first credit feature data is feature data used to evaluate the credit rating of the user, and the first credit feature data includes the first credit feature and the corresponding data. The first credit feature data may be determined based on the loan application data submitted by the user. If the user is an individual, the loan application data may include one or more of the user's occupation, nature of the work unit, education level, school, income, age, marital status, etc., and the first credit feature data may be one or more of the loan application data, or the loan application data may be preprocessed to obtain the first credit feature data. For example, a numerical value may be used to represent the education level, such as 0 for college students and 1 for graduate students.

[0068] The first dissimilarity matrix is ​​determined according to the first credit feature data, that is, in the sample feature space of the guest party, the dissimilarity measure between samples is calculated, that is, the dissimilarity measure between the first credit feature data of any two users among the multiple users is calculated, so as to obtain the first dissimilarity matrix, wherein each sample corresponds to each user, and each sample may include the user and its first credit feature data. The first dissimilarity matrix may be D g As shown in formula (1),

[0069]

[0070] In formula (1), It represents the dissimilarity measure between the i-th sample and the j-th sample, where i and j are natural numbers greater than or equal to 1 and less than or equal to n, and n is the number of samples.

[0071] In an embodiment of the present invention, since the amount of the first credit feature data is limited when training the decision tree model, in order to improve the prediction accuracy of the decision tree model, a federated learning method can be used to receive data sent by other participants based on privacy protection and legal compliance requirements. In federated learning, the guest is a participant that can directly obtain the first credit feature data and credit label of each user among multiple users, and the host is a participant that can provide the second credit feature. The training method of the federated learning decision tree model can be executed on the guest's server. Since it is impossible to directly obtain the user's second credit feature data from the data provider, that is, the host, it is possible to receive the processed data sent by the data provider, and then determine the training sample based on the processed data and the guest's first credit feature data and credit label to enrich the characteristics of the training sample.

[0072] In the embodiment of the present invention, a second dissimilarity matrix sent by a data provider, i.e., a host, can be received. The second dissimilarity matrix is ​​obtained by the data provider based on the second user feature data of each user among multiple users. In the sample feature space of the data provider, a dissimilarity measure between samples is calculated, i.e., a dissimilarity measure between the second credit feature data of any two users among the multiple users is calculated, thereby obtaining a second dissimilarity matrix. The second dissimilarity matrix can be obtained by using D hAfter the data provider calculates the second dissimilarity matrix, it receives the second dissimilarity matrix sent by the data provider. The second credit feature data of the user cannot be directly obtained according to the second dissimilarity matrix, but the data provider can provide the second credit feature in the second credit feature data without providing data details. The second credit feature data can be data used to evaluate the credit rating of the user, and can be based on at least one of the user's loan records, consumption records, income records, etc. For example, if the data provider is an e-commerce platform, the second credit feature data may include the user's annual consumption amount.

[0073] In the embodiment of the present invention, the calculation of the dissimilarity measure between samples may be implemented by using a variety of methods, such as the calculation methods of Euclidean distance, weighted Euclidean distance, cosine distance, Mahalanobis distance, etc.

[0074] In an embodiment of the present invention, determining the global perception sample selection probability distribution according to the first dissimilarity matrix and the second dissimilarity matrix includes:

[0075] Determining a dissimilarity measure matrix based on the first dissimilarity matrix and the second dissimilarity matrix;

[0076] Determine a representation probability matrix according to the dissimilarity measurement matrix and a preset model;

[0077] The global perception sample selection probability distribution is determined based on the representation probability matrix.

[0078] In the embodiment of the present invention, after receiving the second dissimilarity matrix from the data provider, the first dissimilarity matrix and the second dissimilarity matrix can be fused by a fusion method to obtain a dissimilarity measure matrix between samples based on the feature spaces of both parties, which can be represented by D. The fusion method can be selected according to different methods for calculating the dissimilarity measure between samples. For example, when the Euclidean distance is used to calculate the dissimilarity measure between samples, the fused dissimilarity measure matrix can be calculated by the method shown in formula (2).

[0079]

[0080] In formula (2), D h The dissimilarity measure between the i-th sample and the j-th sample in , d ij is the dissimilarity measure between the i-th sample and the j-th sample in D.

[0081] In an embodiment of the present invention, before receiving the second dissimilarity matrix sent by the data provider, the samples of the party can be aligned with the samples of the data provider, wherein the multiple users of the party are the same as the multiple users of the data provider, and the samples of both parties are aligned, that is, the arrangement order of the multiple users is the same in the party and the data provider, and the sample alignment can be achieved by user identification, wherein the user identification can be a user number, and the accuracy of the obtained fused dissimilarity measurement matrix can be guaranteed by sample alignment.

[0082] After obtaining the dissimilarity measurement matrix, the representation probability matrix is ​​obtained according to the dissimilarity measurement matrix and the preset model. The preset model can be the optimal model constructed as shown in formula (3),

[0083]

[0084] In formula (3), z ij Indicates the probability of selecting the i-th sample to represent the j-th sample and represents the representation loss, represents the regularization term. Based on the optimal model shown in formula (3), z is randomly initialized ij Then, Lagrangian method and / or ADMM can be used to solve z ij , thus obtaining the representation probability matrix Z, as shown in formula (4),

[0085]

[0086] After obtaining the representation probability matrix, each row of the representation probability matrix can be summed and mapped to obtain the global perception sample selection probability distribution A-Prob global , as shown in formula (5),

[0087]

[0088] In formula (5), represents the probability of the i-th sample being selected under global perception, and and

[0089] In an embodiment of the present invention, after obtaining the global perception sample selection probability distribution, a user core set of the current node is constructed based on the global perception sample selection probability and the current node, and then each child node after the current node is split is further determined based on the user core set and the first credit feature data of the guest and the second credit feature of the host. The above process is executed repeatedly until the iteration ends, thereby obtaining each child node in the decision tree and obtaining a federated learning decision tree model.

[0090] In the embodiment of the present invention, Figure 2 As shown, determine the user core set of the current node, including:

[0091] Step S201: determining the first selection probability of the current node according to the global perception sample selection probability and the user sample set of the current node;

[0092] Step S202: determining the predicted credit label of each user in the user sample set, and determining the second selection probability of the current node according to the predicted credit label and the credit label of each user in the user sample set;

[0093] Step S203: determining the sample selection probability according to the first selection probability and the second selection probability;

[0094] Step S204: determining a core set of users from the user sample set according to the sample selection probability.

[0095] In the embodiment of the present invention, the user sample set may include one or more users. The user sample set of the current node is the number of all samples to be split at the current node, which is the same as the number of users corresponding to the current node. If the current node is a root node, the user sample set of the current node includes all users. The global perception sample selection probability distribution indicates the probability of each sample being selected under global perception. For each sample in the sample set of the current node, the selection probability of each sample can be determined, thereby obtaining the first selection probability of the current node. Let the user sample set of the current node be The first choice probability Prob of the current node can be obtained as shown in formula (6): glocal ,

[0096]

[0097] In an embodiment of the present invention, determining a predicted credit label for each user in a user sample set includes:

[0098] Selecting a first core set from the user sample set according to the first selection probability;

[0099] A predicted credit label is determined based on the credit labels of each user in the first core set.

[0100] In an embodiment of the present invention, after obtaining the first selection probability of the current node, a preset number of samples can be selected from the user sample set according to the first selection probability of each sample in the user sample set to form the first core set of the current node. The preset number can be custom set, and the preset number is the size of the first core set. The first k samples can be selected in descending order of the first selection probability to form the first core set. The first core set is used to predict the user credit label, which can improve the accuracy of the prediction.

[0101] In an embodiment of the present invention, for each user in the first core set, a credit label of each user is obtained, and the credit label with the largest number of users in the first core set is used as the predicted credit label of the current node, and the predicted credit label is used as the predicted credit label of each user in the user sample set of the current node.

[0102] In the embodiment of the present invention, determining the second selection probability of the current node determined by the user includes:

[0103] Determine a residual value based on the predicted credit label and credit label of each user in the user sample set;

[0104] Generate the second selection probability based on the residual value.

[0105] In the real-time exchange rate of the present invention, for each user in the user sample set of the current node, according to each user's credit label y i and predict credit labels Calculate the residual of the two Then, the second selection probability is generated according to the residual value, which is the local fitting sample selection probability Prob local , as shown in formula (7),

[0106]

[0107] In formula (7), represents the probability of the i-th sample being selected under local fitting, and and The larger the residual value of a sample is, the greater the probability that it is a local fitting sample is, and the greater the probability that it is selected as a model training sample is, and the model training sample can be a sample in the user's core set.

[0108] In one implementation of the embodiment of the present invention, generating the second selection probability according to the residual value may include: calculating the residual value of the credit label and the predicted credit label of each user, calculating the sum of the residual values ​​of each user, and taking the ratio of the residual value of each user to the sum of the residual values ​​as the second selection probability corresponding to the user.

[0109] In the embodiment of the present invention, after determining the second selection probability of the current node, the first selection probability and the second selection probability are weighted and summed to obtain the sample selection probability. For example, a hyperparameter can be used as the weight of the second selection probability, so as to suppress the problem that the existence of abnormal data such as noise and outliers leads to a large residual value, which in turn leads to a high second selection probability and reduces the optimization efficiency of the model. For example, the sample selection probability Prob can be obtained by using the formula shown in formula (8):

[0110] Prob=Prob global +α·Problocal , formula (8),

[0111] In formula (8), α is a hyperparameter, which can be customized according to actual conditions.

[0112] In the embodiment of the present invention, after obtaining the sample selection probability, a user core set is determined from the user sample set of the current node, and the user core set includes multiple samples, that is, multiple users, so that the current node can be split according to the first credit feature data and the second credit feature of the multiple users. Specifically, a preset number of samples can be selected from the user sample set of the current node in the order of sample selection probability from large to small to form the user core set of the current node, wherein the preset number can be set according to a custom setting. By constructing the user core set of the current node to determine each child node after the current node is split, that is, for model training, the scale of the training sample set can be reduced, and the space complexity and time complexity of node splitting can be reduced.

[0113] In the embodiment of the present invention, Figure 3 As shown, the child nodes after the current node is split are determined, including:

[0114] Step S301: determining the first information gain of each first credit feature in each splitting mode according to the first credit feature data and credit label of each user in the user core set;

[0115] Step S302: receiving each split user set obtained according to the second credit characteristic data and sent by the data provider, where the split user set includes a user identifier;

[0116] Step S303: determining a second information gain corresponding to the split user set according to the user identifiers and the user core set in the split user set;

[0117] Step S304: Determine each child node according to the first information gain and the second information gain.

[0118] In the embodiment of the present invention, after determining the user core set of the current node, the first credit feature data of each user in the user core set is obtained. For each first credit feature, multiple splitting methods can be set. The splitting method indicates the splitting threshold. Multiple splitting methods can be achieved by setting multiple splitting thresholds. For example, for age, the splitting threshold can be set to 30 or 40. The number of samples in the sub-nodes split by 30 or 40 is generally different; for income, the threshold can be set to 5000 or 10000. For each splitting method of each first credit feature, the first information gain corresponding to each first credit feature under each splitting method is calculated. The first information gain can be obtained by the ID3 algorithm of the decision tree, that is, the first information gain is the difference between the empirical entropy and the conditional empirical entropy. Among them, the conditional entropy is the information entropy before the current node is split, and the conditional empirical entropy is the information entropy after the current node is split according to a certain splitting method of a certain first credit feature.

[0119] In an embodiment of the present invention, the data provider can set multiple splitting methods for each second credit feature based on the second credit feature data of all users, and obtain a split user set for each second credit feature under each splitting method. The split user set can be one of the child nodes after the split, for example, it can be a left leaf node, and the split user set can include the user identifiers of each user in the left leaf node. Then, the split user set sent by the data provider is received, and the intersection of the split user set and the user core set is determined. Specifically, the user identifier in the split user set is compared with the user identifier in the user core set to obtain the same user identifier. Based on the obtained same user identifier, the credit label corresponding to each user identifier can be obtained, so that the corresponding second information gain can be calculated based on the same user identifier and its corresponding credit label. Thus, the second information gain corresponding to each splitting method of each second credit feature can be obtained.

[0120] In the embodiment of the present invention, Figure 4 As shown, each sub-node is determined according to the first information gain and the second information gain, including:

[0121] Step S401: selecting a maximum information gain from the first information gain and the second information gain;

[0122] Step S402: If the maximum information gain is the first information gain, obtain the target first credit feature and the target splitting method corresponding to the maximum information gain, split the current node according to the target first credit feature and the target splitting method, and determine each child node;

[0123] Step S403: If the maximum information gain is the second information gain, a target split user set corresponding to the maximum information gain is obtained, and each child node is determined according to the user sample set of the current node and the target split user set.

[0124] In the embodiment of the present invention, after obtaining the first information gain and the second information gain, the maximum information gain is selected from the first information gain and the second information gain, and then it is determined whether the maximum information gain is the first information gain or the second information gain. If the maximum information gain is the first information gain, the target first credit feature and the target splitting method corresponding to the maximum information gain can be directly obtained, and the target first credit feature is used as the splitting feature, and according to the target splitting method, the current node is split with the target splitting feature and the target splitting method to obtain the corresponding child nodes, and at the same time, each user in each child node can be obtained, and then the credit label of each user in each child node is obtained, and the credit label with the largest number of users in the child node is used as the credit label of the child node.

[0125] If the maximum information gain is the second information gain, then obtain the target split user set corresponding to the maximum information gain, and then determine the intersection of the user sample set of the current node and the target split user set. The intersection is one of the child nodes after the current node is split. If it can be a left leaf node, then the user sample set except the left leaf node is the right leaf node after the current node is split. Then obtain the credit label of each user in each child node, and take the credit label with the largest number of users in the child node as the credit label of the child node.

[0126] In the embodiment of the present invention, after obtaining each child node after the current node is split, step S105 is executed to determine whether the stop condition is met. If not, step S105 is executed. If it is met, the training ends and a federated learning decision tree model is obtained, wherein the stop condition may be whether the depth of the decision tree obtained by training is greater than or equal to the preset depth, or the stop condition may be whether the splitting accuracy of the decision tree obtained by training is greater than or equal to the preset splitting accuracy. The splitting accuracy can be obtained by predicting the predicted credit labels of all users of all child nodes and the real credit labels of all users according to the decision tree prediction. For example, the user core set includes 100 users, of which 70 users have credit labels of high credit rating and 30 users have credit labels of low credit rating. The decision tree is split to obtain 7 child nodes, and the number of users predicted to obtain credit labels of high credit rating is 65, and the number of users predicted to obtain credit labels of low credit rating is 35. It can be seen that the credit label prediction of 5 users is inaccurate, and the splitting accuracy of the decision tree can be obtained to be 95%.

[0127] The training method of the federated learning decision tree model of the embodiment of the present invention determines a first dissimilarity matrix according to the first credit feature data of each user among multiple users, receives a second dissimilarity matrix obtained based on the second credit feature data sent by the data provider, obtains a global perception sample selection probability distribution based on the first dissimilarity matrix and the second dissimilarity matrix, and then constructs a user core set based on the global perception sample selection probability distribution in combination with the user sample set of the current node, uses the user core set and the first credit feature data and the second credit feature data to obtain each child node after the current node is split, and obtains a federated learning decision tree model after training. The method utilizes federated learning to obtain the second credit feature of the user from the data provider in compliance with regulations to achieve the purpose of expanding the features, which can make the prediction accuracy of the trained model higher and improve the model performance. The method reduces the space complexity and time complexity of the model training process by constructing a user core set and using the user core set for model training, while reducing the computational cost in the model training process and the communication burden in federated learning, reducing the training cost of the model and improving the application value of the model. This method determines the user core set by constructing a global perception sample selection probability and combining it with the user sample set of the current node. The user core set can better characterize the original user sample set, provide good data support for the establishment of each model, and is diverse and representative. It also improves the optimization efficiency of the model and the efficiency of model training.

[0128] The embodiment of the present invention also provides a method for predicting a user's credit rating, including: obtaining a user's first credit feature data and a second credit feature data, inputting the first credit feature data and the second credit feature data into a federated learning decision tree model, and obtaining the user's credit rating. The federated learning decision tree model can be used to evaluate the user's credit rating in micro-loan business, determine the user's credit rating based on the model, and determine the user's loan eligibility and amount based on the user's credit rating.

[0129] Another aspect of the embodiment of the present invention is as follows Figure 5 As shown, a training device 500 for a federated learning decision tree model is also provided, comprising:

[0130] An acquisition module 501 acquires first credit feature data and a credit label of each user among a plurality of users;

[0131] A first determination module 502 determines a first dissimilarity matrix according to first credit feature data;

[0132] A receiving module 503 receives a second dissimilarity matrix obtained based on second credit feature data of each user among a plurality of users and sent by a data provider;

[0133] A second determination module 504 determines a global perception sample selection probability distribution according to the first dissimilarity matrix and the second dissimilarity matrix;

[0134] The training module 505 executes the following process in a loop to obtain a decision tree model through training: determining the user core set of the current node according to the global perception sample selection probability distribution, the user sample set of the current node, and the credit label of each user in the user sample set; determining the child nodes after the current node is split according to the first credit feature data, the second credit feature data, and the user core set, and taking each child node as the current node.

[0135] In an embodiment of the present invention, the second determination module 504 is further used to: determine a dissimilarity measure matrix according to the first dissimilarity matrix and the second dissimilarity matrix; determine a representation probability matrix according to the dissimilarity measure matrix and a preset model; and determine a global perception sample selection probability distribution according to the representation probability matrix.

[0136] In an embodiment of the present invention, the training module 505 is further used to: determine the first selection probability of the current node based on the global perception sample selection probability and the user sample set of the current node; determine the predicted credit label of each user in the user sample set to determine the second selection probability of the current node based on the predicted credit label and credit label of each user in the user sample set; determine the sample selection probability based on the first selection probability and the second selection probability; and determine a user core set from the user sample set based on the sample selection probability.

[0137] In the embodiment of the present invention, the training module 505 is further used to: select a first core set from the user sample set according to the first selection probability; and determine a predicted credit label according to the credit label of each user in the first core set.

[0138] In the embodiment of the present invention, the training module 505 is further used to: determine a residual value according to the predicted credit label and the credit label of each user in the user sample set; and generate a second selection probability according to the residual value.

[0139] In the embodiment of the present invention, the training module 505 is further used to: determine the first information gain of each first credit feature under each splitting mode according to the first credit feature data and credit label of each user in the user core set; receive each split user set obtained according to the second credit feature data sent by the data provider, the split user set including the user identifier; determine the second information gain corresponding to the split user set according to the user identifier and the user core set in the split user set; determine each child node according to the first information gain and the second information gain.

[0140] In an embodiment of the present invention, the training module 505 is further used to: select the maximum information gain from the first information gain and the second information gain; if the maximum information gain is the first information gain, obtain the target first credit feature and the target splitting method corresponding to the maximum information gain, split the current node according to the target first credit feature and the target splitting method, and determine each child node; if the maximum information gain is the second information gain, obtain the target split user set corresponding to the maximum information gain, and determine each child node according to the user sample set of the current node and the target split user set.

[0141] Figure 6 An exemplary system architecture 600 is shown to which a method for training a federated learning decision tree model or an apparatus for training a federated learning decision tree model according to an embodiment of the present invention can be applied.

[0142] like Figure 6 As shown, system architecture 600 may include terminal devices 601, 602, 603, network 604 and server 605. Network 604 is used to provide a medium for communication links between terminal devices 601, 602, 603 and server 605. Network 604 may include various connection types, such as wired, wireless communication links or optical fiber cables, etc.

[0143] Users can use terminal devices 601, 602, and 603 to interact with server 605 through network 604 to receive or send messages, etc. Various communication client applications can be installed on terminal devices 601, 602, and 603, such as shopping applications, web browser applications, search applications, instant messaging tools, email clients, social platform software, etc. (only examples).

[0144] The terminal devices 601 , 602 , and 603 may be various electronic devices having a display screen and supporting web browsing, including but not limited to smart phones, tablet computers, laptop computers, and desktop computers, etc.

[0145] The server 605 may be a server that provides various services, such as a backend management server (only an example) that provides support for shopping websites browsed by users using the terminal devices 601, 602, and 603. The backend management server may analyze and process the received data such as product information query requests, and feed back the processing results (such as target push information, product information - only an example) to the terminal device.

[0146] It should be noted that the training method of the federated learning decision tree model provided in the embodiment of the present invention is generally executed by the server 605 , and accordingly, the training device of the federated learning decision tree model is generally set in the server 605 .

[0147] It should be understood that Figure 6The number of terminal devices, networks and servers in the embodiment is only for illustration. Any number of terminal devices, networks and servers may be provided according to implementation requirements.

[0148] Reference below Figure 7 , which shows a schematic diagram of the structure of a computer system 700 of a terminal device suitable for implementing an embodiment of the present invention. Figure 7 The terminal device shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present invention.

[0149] like Figure 7 As shown, the computer system 700 includes a central processing unit (CPU) 701, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 702 or a program loaded from a storage part 708 into a random access memory (RAM) 703. In the RAM 703, various programs and data required for the operation of the system 700 are also stored. The CPU 701, the ROM 702, and the RAM 703 are connected to each other via a bus 704. An input / output (I / O) interface 705 is also connected to the bus 704.

[0150] The following components are connected to the I / O interface 705: an input section 706 including a keyboard, a mouse, etc.; an output section 707 including a cathode ray tube (CRT), a liquid crystal display (LCD), etc., and a speaker, etc.; a storage section 708 including a hard disk, etc.; and a communication section 709 including a network interface card such as a LAN card, a modem, etc. The communication section 709 performs communication processing via a network such as the Internet. A drive 710 is also connected to the I / O interface 705 as needed. A removable medium 711, such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc., is installed on the drive 710 as needed, so that a computer program read therefrom is installed into the storage section 708 as needed.

[0151] In particular, according to the embodiments disclosed in the present invention, the process described above with reference to the flowchart can be implemented as a computer software program. For example, the embodiments disclosed in the present invention include a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program includes a program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from the network through the communication part 709, and / or installed from the removable medium 711. When the computer program is executed by the central processing unit (CPU) 701, the above-mentioned functions defined in the system of the present invention are executed.

[0152] It should be noted that the computer-readable medium shown in the present invention may be a computer-readable signal medium or a computer-readable storage medium or any combination of the above two. The computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present invention, a computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in combination with an instruction execution system, device or device. In the present invention, a computer-readable signal medium may include a data signal propagated in a baseband or as part of a carrier wave, which carries a computer-readable program code. This propagated data signal may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. Computer-readable signal media may also be any computer-readable medium other than computer-readable storage media, which may send, propagate or transmit a program for use by or in conjunction with an instruction execution system, apparatus or device. The program code contained on the computer-readable medium may be transmitted using any appropriate medium, including but not limited to: wireless, wire, optical cable, RF, etc., or any suitable combination of the above.

[0153] The flow chart and block diagram in the accompanying drawings illustrate the possible architecture, function and operation of the system, method and computer program product according to various embodiments of the present invention. In this regard, each box in the flow chart or block diagram can represent a module, a program segment, or a part of a code, and the above-mentioned module, program segment, or a part of a code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order from the order marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram or flow chart, and the combination of the boxes in the block diagram or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0154] The modules involved in the embodiments of the present invention may be implemented by software or hardware. The modules described may also be set in a processor, for example, it may be described as: a processor includes an acquisition module, a first determination module, a second determination module and a training module. The names of these modules do not constitute a limitation on the modules themselves in some cases, for example, the acquisition module may also be described as "obtaining the first credit feature data and credit label of each user among multiple users".

[0155] As another aspect, the present invention also provides a computer-readable medium, which may be included in the device described in the above embodiment; or may exist independently without being assembled into the device. The above computer-readable medium carries one or more programs, and when the above one or more programs are executed by a device, the device includes: obtaining first credit feature data and credit label of each user among multiple users; determining a first dissimilarity matrix according to the first credit feature data; receiving a second dissimilarity matrix obtained based on the second credit feature data of each user among multiple users sent by a data provider; determining a global perception sample selection probability distribution according to the first dissimilarity matrix and the second dissimilarity matrix; looping the following process to train a decision tree model: determining a user core set of the current node according to the global perception sample selection probability distribution, the user sample set of the current node, and the credit label of each user in the user sample set; determining each child node after the current node is split according to the first credit feature data, the second credit feature data, and the user core set, and taking each child node as the current node.

[0156] According to the technical solution of the embodiment of the present invention, the training method of the federated learning decision tree model determines the first dissimilarity matrix according to the first credit feature data of each user among multiple users, receives the second dissimilarity matrix obtained based on the second credit feature data sent by the data provider, obtains the global perception sample selection probability distribution based on the first dissimilarity matrix and the second dissimilarity matrix, and then constructs the user core set based on the global perception sample selection probability distribution in combination with the user sample set of the current node, uses the user core set and the first credit feature data and the second credit feature data to obtain each child node after the current node is split, and obtains the federated learning decision tree model after training. The method uses federated learning to obtain the second credit feature of the user from the data provider in compliance with regulations to achieve the purpose of expanding the features, which can make the prediction accuracy of the trained model higher and improve the model performance. The method reduces the space complexity and time complexity of the model training process by constructing the user core set and using the user core set for model training, while reducing the computational cost in the model training process and the communication burden in federated learning, reducing the training cost of the model and improving the application value of the model. This method determines the user core set by constructing a global perception sample selection probability and combining it with the user sample set of the current node. The user core set can better characterize the original user sample set, provide good data support for the establishment of each model, and is diverse and representative. It also improves the optimization efficiency of the model and the efficiency of model training.

[0157] The above specific implementations do not constitute a limitation on the protection scope of the present invention. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions may occur depending on design requirements and other factors. Any modification, equivalent substitution and improvement made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. A method for training a federated learning decision tree model, characterized in that: include: Acquire first credit feature data and a credit label of each user among a plurality of users; Determining a first dissimilarity matrix according to the first credit feature data; Receiving a second dissimilarity matrix sent by a data provider and obtained based on second credit feature data of each user among the multiple users; Determining a global perception sample selection probability distribution according to the first dissimilarity matrix and the second dissimilarity matrix; The following process is executed repeatedly to train the decision tree model: Determine the user core set of the current node according to the global perception sample selection probability distribution, the user sample set of the current node and the credit label of each user in the user sample set; determine the child nodes after the current node is split according to the first credit feature data, the second credit feature data and the user core set, and use each of the child nodes as the current node.

2. The training method according to claim 1, characterized in that: Determining a global perception sample selection probability distribution according to the first dissimilarity matrix and the second dissimilarity matrix includes: Determining a dissimilarity measure matrix based on the first dissimilarity matrix and the second dissimilarity matrix; Determine a representation probability matrix according to the dissimilarity measurement matrix and a preset model; The global perception sample selection probability distribution is determined according to the characterization probability matrix.

3. The training method according to claim 2, characterized in that: Determine the user core set of the current node, including: Determining a first selection probability of the current node according to the global perception sample selection probability and the user sample set of the current node; Determine a predicted credit label for each user in the user sample set, so as to determine a second selection probability of the current node according to the predicted credit label and the credit label of each user in the user sample set; Determine a sample selection probability according to the first selection probability and the second selection probability; The user core set is determined from the user sample set according to the sample selection probability.

4. The training method according to claim 3, characterized in that: Determining a predicted credit label for each user in the user sample set includes: Selecting a first core set from the user sample set according to the first selection probability; The predicted credit label is determined according to the credit labels of each user in the first core set.

5. The training method according to claim 3, characterized in that: Determine the second selection probability of the current node, including: Determining a residual value according to the predicted credit label and the credit label of each user in the user sample set; The second selection probability is generated according to the residual value.

6. The training method according to claim 1, characterized in that: Determining each child node after the current node is split, including: Determining a first information gain of each first credit feature in each splitting mode according to the first credit feature data and the credit label of each user in the user core set; receiving each split user set obtained according to the second credit characteristic data and sent by the data provider, wherein the split user set includes a user identifier; Determine a second information gain corresponding to the split user set according to the user identifiers in the split user set and the user core set; The respective sub-nodes are determined according to the first information gain and the second information gain.

7. The training method according to claim 6, characterized in that: Determining each of the subnodes according to the first information gain and the second information gain includes: Selecting a maximum information gain from the first information gain and the second information gain; If the maximum information gain is the first information gain, obtaining a target first credit feature and a target splitting method corresponding to the maximum information gain, splitting the current node according to the target first credit feature and the target splitting method, and determining each child node; If the maximum information gain is the second information gain, a target split user set corresponding to the maximum information gain is obtained, and each child node is determined according to the user sample set of the current node and the target split user set.

8. A training device for a federated learning decision tree model, characterized in that: include: An acquisition module, which acquires first credit feature data and a credit label of each user among a plurality of users; A first determination module, determining a first dissimilarity matrix according to the first credit feature data; A receiving module, receiving a second dissimilarity matrix obtained based on second credit feature data of each user among the multiple users and sent by a data provider; A second determination module determines a global perception sample selection probability distribution according to the first dissimilarity matrix and the second dissimilarity matrix; The training module cyclically executes the following process to train a decision tree model: determining the user core set of the current node according to the global perception sample selection probability distribution, the user sample set of the current node, and the credit label of each user in the user sample set; determining each child node after the current node is split according to the first credit feature data, the second credit feature data and the user core set, and taking each of the child nodes as the current node.

9. An electronic device, characterized in that: include: one or more processors; a storage device for storing one or more programs, When the one or more programs are executed by the one or more processors, the one or more processors implement the method according to any one of claims 1 to 7.

10. A computer readable medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the method according to any one of claims 1 to 7 is implemented.

Images