Multi-party semi-supervised learning-based method for recommending financial products for potential users
Through multi-party semi-supervised learning and vertical federal learning, combined with the GBDT model, the data island problem of financial institutions when recommending new financial products is solved, the accuracy and batch recommendation of financial products are achieved, and the reliability and accuracy of recommendations are improved.
Patent Information
- Application Number
- PCT/CN2024/091099
- Authority / Receiving Office
- WO · WO
- Patent Type
- Applications
- Current Assignee / Owner
- Priority Date
- 2023-05-08
- Filing Date
- 2024-05-06
- Publication Date
- 2024-11-14
AI Technical Summary
Financial institutions face challenges when recommending new financial products, especially when only a small amount of user information that has been purchased, and the data of each party is isolated due to privacy and security protection requirements, making it difficult to build a high-performance machine learning model for accurate recommendation.
Using multi-party semi-supervised learning method, by establishing a data set containing purchased and unpurned user information, preprocessing and sample alignment, building a vertical federated learning model, using gradient boosting tree (GBDT) as the base learner, performing multiple iterative training and predict, calculate the predicted probability of samples, gradually increase the positive sample data set, reduce the unlabeled sample data set, and achieve accurate recommendations from potential users.
While protecting the security and privacy of multi-party data, the batch recommendation problem with only a small number of positive samples and a large number of unlabeled samples is effectively solved, which improves the reliability and accuracy of recommendations, and realizes the accuracy and batch recommendation of potential users of financial products.
Smart Images

Figure CN2024091099_14112024_PF_FP_ABST
Abstract
Description
A multi-party semi-supervised learning method for recommending potential users of financial products Technical Field
[0001] The present invention belongs to the field of big data recommendation and relates to a method for recommending potential users of financial products based on multi-party semi-supervised learning. Background Art
[0002] With the continuous development of society and the economy, more and more financial institutions are launching diverse financial products for diverse user groups. Financial products refer to the various financial services and products offered by financial institutions such as banks, securities companies, insurance companies, and provident funds, such as loans, credit cards, mutual funds, stocks, and provident fund contributions. However, due to the varying needs of different users for different financial products, quickly and accurately recommending a new financial product to potential users after its release presents a significant challenge. Machine learning and big data technologies can be used to analyze the characteristics of users who have purchased a financial product from a particular financial institution and identify potential users with similar characteristics among those who have not yet purchased the product. This allows for precise recommendations to meet the needs of both users and the institution. However, when financial institutions only have limited information on users who have purchased the product and lack information on those who have not yet, they need to obtain more data from other institutions or channels to assist with recommendations. However, due to data privacy and security requirements, data is isolated, forming data silos. This makes it difficult for different participants to aggregate data to build a high-performance machine learning model.
[0003] Summary of the Invention
[0004] In view of this, the object of the present invention is to provide a method for recommending potential users of financial products by combining multi-party semi-supervised learning.
[0005] In order to achieve the above object, the present invention provides the following technical solutions:
[0006] A method for recommending potential users of financial products based on multi-party semi-supervised learning, comprising the following steps:
[0007] S1: Establish a multi-party dataset for recommending potential users of financial products, which includes information about users who purchased financial products and other users from multiple parties. Preprocess and align the dataset to construct a positive sample dataset and an unlabeled sample dataset.
[0008] S2: Perform random sampling with replacement in the unlabeled sample dataset to establish a negative sample dataset; construct a training set using the negative sample dataset and the positive sample dataset, and construct a prediction set using the unsampled samples in the unlabeled sample dataset; construct a longitudinal federated learning model based on the base learner, train it on the training set, and predict it on the prediction set to obtain a prediction score for each sample in the prediction set;
[0009] S3: Repeat the sampling, training, and prediction process of step S2 multiple times; calculate the probability of each sample being predicted as a positive sample based on the sum of the prediction scores of each sample in the unlabeled sample dataset and the number of times it appears in the prediction set; sort all samples in the unlabeled sample dataset from highest to lowest according to the probability of being a positive sample, select the top-ranked samples as reliable positive samples based on prior knowledge, and add them to the positive sample dataset, while deleting them from the unlabeled sample dataset;
[0010] S4: Repeat steps S2-S3 until the preset maximum number of iterations is reached; thereby, all reliable positive samples selected from the unlabeled sample dataset are used as potential users and accurately recommended in batches to financial product providers.
[0011] Furthermore, the multi-party dataset for recommending potential users of financial products in step S1 includes Party A, which has information on users who have purchased financial products, and Party B and Party C, other multi-party data sources other than Party A. Step S1 specifically includes the following steps:
[0012] S11: Preprocess the data of multiple data sets recommended by potential users of financial products, including redundant data processing, missing value processing, outlier processing, data standardization processing and label data processing, to obtain the A-party data set Represents the eigenvector of the i-th sample of party A, with a vector dimension of d A , express Corresponding labels, where i = 1,…,a; Party B dataset Represents the feature vector of the i-th sample of party B, with a vector dimension of d B , express Corresponding labels, where i = 1,…,b; C-square dataset Represents the ith sample feature vector of C square, the vector dimension is d C , express corresponding labels, where i = 1,…,c;
[0013] S12: Dataset D for Party B and Party C B and D C Perform sample encryption alignment according to their sample IDs, retain the aligned sample data of Party B and Party C, discard the unaligned sample data, and obtain n samples; the aligned dataset of Party B is The C-square dataset is express The corresponding label, express corresponding labels, where i = 1,…,n;
[0014] S13: Dataset D A 、 and Perform sample encryption alignment according to their sample IDs, and use the three-way aligned samples as positive samples to form a positive sample dataset P = {(xp i ,yp i )},in Respectively represent the i-th sample feature vector after alignment of party A, party B, and party C, yp i ∈{1} is the positive sample label, |P| represents the number of positive samples, i=1,…,|P|; the three unaligned samples are regarded as unlabeled samples, forming the unlabeled sample dataset U={(xu i ,yu i )},in Respectively represent the i-th sample feature vector after alignment of party B and party C, yu i ∈{0} is the unlabeled sample label, |U| represents the number of unlabeled samples, i=1,…,|U|.
[0015] Furthermore, in step S11, the redundant data processing specifically includes: determining whether the data is duplicated based on certain fields in the data record, retaining only one of the duplicate data, deleting the other duplicate data from the data set, and retaining a backup of the original data for backtracking and comparative analysis when needed;
[0016] The missing value processing specifically includes: performing missing rate statistics on each feature in the data, treating data with a number of missing features greater than half of the total sample feature size as invalid data and eliminating them; and filling the remaining data using a specific method based on data distribution;
[0017] The outlier processing specifically includes: determining the outlier threshold based on the actual situation; performing visual analysis on the data to identify data that exceeds the threshold; processing the outlier, and then rechecking the data set after processing to ensure that the outlier has been processed and the basic characteristics of the data set have not changed significantly;
[0018] The data normalization process specifically includes: dividing the existing features into continuous features and discrete features according to data type, processing the continuous features using maximum-minimum normalization, and processing the discrete features using one-hot encoding;
[0019] The label data processing specifically includes: adding a label column to the A party data set, setting its label data to "1", representing positive sample data; adding a label column to the B party and C party data sets respectively, setting its label data to "0", representing unlabeled data.
[0020] Furthermore, step S2 specifically includes: establishing a process for predicting unlabeled sample candidate recommendations, looping through the process M times, and the prediction process for the mth round is as follows: randomly extracting |P| samples from U with replacement, where |P| represents the number of samples in P, and forming these |P| unlabeled samples into a negative sample dataset N m ; by P and N m Composition of training set The unsampled samples in U form the prediction set Construct a vertical federated model with Gradient Boosting Decision Tree (GBDT) as the base learner. Training on Input into the trained model for prediction and get The score of each sample in is used as the predicted score of the sample; specifically, the following steps are included:
[0021] S21: Use the bootstrap sampling method to randomly extract |P| samples from U with replacement, and use these |P| unlabeled samples to form the mth round of negative sample dataset in are the feature vectors of party B and party C in the i-th negative sample of round m, Represents the label of the i-th negative sample in the m-th round, i=1,…,|P|,|N m | represents the negative sample dataset N for the mth round m The number of samples; the positive sample data set P and the negative sample data set N of the mth round m Constitute the corresponding m-th round training set in They represent the feature vectors of party B and party C in the i-th sample of round m, represents the label of the i-th sample in the m-th round, i = 1, 2, ..., 2|P|; the unextracted samples in the unlabeled sample dataset U constitute the corresponding m-th round prediction set in Represents the feature vector of the jth sample in the mth round, j=1,2,...,|U|-|N m |;
[0022] S22: Use the GBDT algorithm as the base learner to build a vertical federation model through T decision trees Integration of Training, input data Each sample To predict its mth round output Where T represents the total number of decision trees in the mth round, Represents the prediction result of the t-th tree in the m-th round, Represents the feature vector of the i-th sample in the m-th round, i=1,2,...,2|P|; according to the predicted output and the true label The gradient value of the loss function between the two is used to establish the first-order derivative and the second-order derivative gradient histogram. The gradient histograms of all features of both sides are combined to find the global optimal segmentation of the current node to construct the optimal decision tree.
[0023] S23: Use the trained GBDT decision model for the mth round to test the mth round of the test set Make predictions and obtain the mth round test set Each sample The prediction score of , traverse all samples in U, and find the one with Samples with the same ID i , and put The prediction score is assigned to xu i , as a sample i The prediction score of the mth round is Indicates, where i=1,…,|U| is the sample number in the unlabeled dataset U; if the sample xu in U i Not present in round m In the case of i The corresponding prediction score for the mth round Equal to 0.
[0024] Further, in step S22, each sample of the mth round is first initialized The prediction results is a random value, and then the specific process of federated training of the t-th tree in the m-th round is as follows:
[0025] S221: Starting from party B, first for each sample xtr of party B i m,B Calculate the first-order gradient of the loss function of the m-th round model and the second-order gradient where i=1,2,...,2|P|, Represents the paired samples aggregated by the first t-1 trees in the mth round The prediction results, represents the true label of the i-th sample in the m-th round, is the loss function; using additive homomorphic encryption and Encrypt and get and Party B will and Send to Party C;
[0026] S222: Party C creates the mth round gradient histogram based on its own feature data and sends the encrypted gradient histogram to Party B.
[0027] S223: Party B decrypts the m-th round encrypted gradient histogram from Party C, enumerates each feature gradient histogram according to the splitting gain calculation formula, calculates the optimal solution, finds the global optimal splitting point, and returns the splitting information to Party C for analysis;
[0028] S224: Party C receives the feature number K sent from Party B. opt and threshold number V opt Determine the threshold of the feature and divide the current sample space; then Party C establishes a lookup table locally to record the threshold of the selected feature, forming a record [record number, feature, threshold], and the record number and the sample space on the left after division (I L ) is returned to Party B;
[0029] S225: Party B receives [record number, I L ] divides the current node and associates the current node with [participant, record number]; Party B synchronizes the division information of the current node with Party C and proceeds to split the next node;
[0030] S226: Iterate steps S222 to S225 until the training stop condition or the maximum depth of the tree is reached.
[0031] Furthermore, step S222 specifically includes the following steps:
[0032] S2221: For the current m-th round sample All C square features in the , sort all samples according to the characteristic value of each feature, and then divide the sorted samples into q categories by bucketing, and get the corresponding feature threshold of each category Where k represents the feature number, Indicates the threshold of the qth category of the feature numbered k in the mth round;
[0033] S2222: Based on the mth round obtained from Party B and Party C aggregates the encrypted gradient information and constructs the mth round encrypted gradient histogram, i.e.
[0034] where i = 1, 2, ..., 2|P|, v = 1, 2, ..., q;
[0035] S2223: Party C calculates the mth round and Send to Party B.
[0036] Furthermore, step S223 specifically includes the following steps:
[0037] S2231: Party B aggregates the first-order gradient and second-order gradient of all samples in the current node space and executes Where I represents all samples of the current node;
[0038] S2232: Party B receives the mth round from Party C and Decrypt and get the mth round decryption value and The following calculations are performed on all categories of all features of party C in turn to obtain and
[0039] Among them I L Represents the sample space of the left child node after segmentation, I R represents the sample space of the right child node after segmentation, represents the sum of the first-order gradients of the loss function of all samples in the sample space of the left child node in the mth round, represents the sum of the second-order gradients of the loss function of all samples in the sample space of the left child node in the mth round, represents the sum of the first-order gradients of the loss function of all samples in the sample space of the right child node in the mth round, Represents the sum of the second-order gradients of the loss function of all samples in the sample space of the right child node in the mth round;
[0040] S2233: Calculate the optimal split value of the current node in round m
[0041] Where λ is a hyperparameter;
[0042] S2234: All thresholds for each feature of the sample Can get one value, select the largest one Value, determine the feature threshold as the global optimal segmentation of the mth round, the global optimal segmentation is [participant, feature number (K opt ), threshold number (V opt )] and the feature number (K opt ) and threshold number (V opt ) is returned to Party C.
[0043] Furthermore, step S23 specifically includes the following steps:
[0044] S231: Party B queries the [Party, Record Number] record associated with the current node; based on this record, Party B sends the sample number and record number to be annotated to Party C, and inquires about the next tree search direction, i.e., the left child node or the right child node;
[0045] S232: After receiving the sample number and record number to be labeled, Party C compares the value of the corresponding feature in the sample to be labeled with the threshold value in the record [record number, feature, threshold] in the local lookup table to determine the next tree search direction; then, Party C sends the search decision to Party B;
[0046] S233: Party B receives the search decision from Party C and goes to the corresponding child node;
[0047] S234: Iterate steps S231 to S233 until a leaf node is reached, and the corresponding classification label and the weight of the label are obtained, thereby obtaining the weight of the sample. The corresponding sample xu in U i The prediction score of the mth round
[0048] Where I represents the sample space of leaf nodes and λ is a hyperparameter.
[0049] Further, in step S3, each sample xu in U i The sum of the M-round prediction scores and the sum of the number of occurrences in the M-round prediction set, calculate the sample xui Predict the probability of being a positive sample; sort all samples in U from large to small according to the probability of being a positive sample, select the top-ranked samples as reliable positive samples based on prior knowledge, and add them to the positive sample dataset P, while deleting them from U. The specific steps include:
[0050] S31: The sum of the prediction scores of each sample in U And the sum of its occurrence times in the M rounds of prediction sets, calculate the probability ρ of the sample prediction being a positive sample i , the calculation formula is as follows:
[0051] in is the indicator function, indicating that if the sample xu in U i The test set present in round m If , then I = 1, otherwise I = 0;
[0052] S32: Use the probability ρ for all samples in U i Sort in descending order, select the top θ samples as reliable positive samples, add them to the positive sample dataset P, and delete them from U, where the value of θ is set according to prior knowledge.
[0053] Furthermore, the step S4 is specifically as follows: repeating steps S2-S3 until the preset maximum number of iterations is reached. In each iteration, for the reliability and accuracy of the recommendation, only a small number of reliable positive samples can be selected in each iteration, so in order to select a certain amount of reliable positive samples for batch recommendation, multiple iterations are required. In each iteration, step S3 will select some reliable positive samples from the unlabeled dataset U and add them to the positive sample dataset P, while deleting them from the unlabeled dataset U. In this case, the sample size of |P| will increase with the increase in the number of iterations, and the sample size of |U| will decrease with the increase in the number of iterations until the preset maximum number of iterations is reached. Therefore, all reliable positive samples selected from U during multiple iterations can be used as potential purchasers of financial products and accurately recommended in batches to the owners of financial products.
[0054] The beneficial effects of the present invention are as follows: in order to solve the problem that financial product providers only have positive sample data and are unable to make recommendations, the present invention combines vertical federated learning and semi-supervised learning methods, and jointly trains and predicts recommendation models for potential users of financial products with multiple parties while protecting the data security and privacy of multiple parties. This effectively solves the problem of batch recommendations with only a small number of positive samples and a large number of unlabeled samples, improves the reliability of recommendations, and realizes accurate and batch recommendations for potential users of financial products.
[0055] Other advantages, objects, and features of the present invention will be described in part in the following description and, in part, will be apparent to those skilled in the art upon examination of the following description or may be learned from practice of the present invention. The objects and other advantages of the present invention may be realized and obtained through the following description. BRIEF DESCRIPTION OF THE DRAWINGS
[0056] In order to make the purpose, technical solutions and advantages of the present invention more clear, the present invention will be described in detail below with reference to the accompanying drawings, in which:
[0057] Figure 1 is a flowchart of a method for recommending potential users of financial products using multi-party semi-supervised learning;
[0058] Figure 2 is a schematic diagram of multi-party data sample alignment in vertical federated learning;
[0059] Figure 3 is a schematic diagram of random sampling to form training sets and test sets;
[0060] FIG4 is a schematic diagram of constructing a gradient histogram;
[0061] Figure 5 is a schematic diagram of the training process of the vertical federated GBDT model;
[0062] Figure 6 is a schematic diagram of the prediction process of the vertical federated GBDT model. DETAILED DESCRIPTION
[0063] The following describes the embodiments of the present invention by means of specific examples, and those skilled in the art can easily understand other advantages and effects of the present invention from the contents disclosed in this specification. The present invention can also be implemented or applied through other different specific embodiments, and the details in this specification can also be modified or changed in various ways based on different viewpoints and applications without departing from the spirit of the present invention. It should be noted that the illustrations provided in the following embodiments are only schematic illustrations of the basic concept of the present invention, and the following embodiments and features in the embodiments can be combined with each other without conflict.
[0064] Among them, the accompanying drawings are only for illustrative purposes and represent only schematic diagrams rather than actual pictures, and should not be understood as limiting the present invention. In order to better illustrate the embodiments of the present invention, some parts of the accompanying drawings may be omitted, enlarged or reduced, and do not represent the dimensions of actual products. For those skilled in the art, it is understandable that some well-known structures and their descriptions may be omitted in the accompanying drawings.
[0065] The same or similar numbers in the drawings of the embodiments of the present invention correspond to the same or similar parts; in the description of the present invention, it should be understood that if there are terms such as "upper", "lower", "left", "right", "front", "back", etc. indicating directions or positional relationships, they are based on the directions or positional relationships shown in the drawings. They are only for the convenience of describing the present invention and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific direction, be constructed and operate in a specific direction. Therefore, the terms describing the positional relationship in the drawings are only used for illustrative purposes and cannot be understood as limiting the present invention. For ordinary technicians in this field, the specific meanings of the above terms can be understood according to specific circumstances.
[0066] Taking the example of a scenario where flexible employment workers are recommended for provident fund contributions, information on potential users who haven't yet made contributions exists across multiple parties, all of which have data security and privacy requirements. After vertical federated learning and sample alignment of this multi-party data, Party A will only have information on flexible employment workers who have already made contributions, while Parties B and C will have information on both those who have already made contributions and those who are potential flexible employment workers who haven't yet.
[0067] In this embodiment, Party A is the provident fund provider, possessing provident fund contribution data for flexible employment personnel, including basic personal information, provident fund account number, provident fund balance, number of provident fund contribution periods, provident fund contribution ratio, provident fund contribution amount, and other information. Optionally, Party B can be the tax data provider, possessing basic personal information, personal income tax, housing management tax, and other information. Optionally, Party C can be the social security data provider, possessing basic personal information, medical insurance, pension insurance, unemployment insurance, and other information.
[0068] Please refer to Figures 1 to 6 for a multi-party semi-supervised learning method for recommending potential users of financial products. The method includes the following steps:
[0069] S1: Create a multi-party dataset of flexible employment personnel contribution recommendations, including Party A, Party B, and Party C, whose provident fund contribution information is available. Preprocess the multi-party dataset and perform sample alignment to form a positive sample dataset P and an unlabeled sample dataset U. The specific steps are as follows:
[0070] S11: Data preprocessing operations are performed on the flexible employment personnel contribution recommendation multi-party dataset, including redundant data processing, missing value processing, outlier processing, data standardization processing, and label data processing, to obtain the A-party dataset Represents the eigenvector of the i-th sample of party A, with a vector dimension of d A , express Corresponding labels, where i = 1,…,a; Party B dataset Represents the feature vector of the i-th sample of party B, with a vector dimension of d B , express Corresponding labels, where i = 1,…,b; C-square dataset Represents the ith sample feature vector of C square, the vector dimension is d C , express The corresponding labels, where i = 1,…,c. The specific operations of data preprocessing are as follows:
[0071] The redundant data processing specifically includes: judging whether the data is repeated by certain fields in the data record, retaining only one of the duplicate data, deleting the other duplicate data from the data set, and retaining a backup of the original data for backtracking and comparative analysis when needed.
[0072] The missing value processing specifically includes: calculating the missing rate of each feature in the data, deeming data with more than half the number of missing features as invalid data and eliminating them; and filling the remaining data using a specific method based on the data distribution. Optionally, the median filling method can be used to fill missing values.
[0073] The outlier processing specifically includes: optionally, using the standard deviation method to determine the outlier threshold based on the actual situation; optionally, using a box plot to visualize the data and identify data that exceeds the threshold; optionally, using the median to replace the outlier; and after processing, rechecking the dataset to ensure that the outlier has been addressed and the basic characteristics of the dataset have not changed significantly.
[0074] The data normalization process specifically includes: dividing the existing features into continuous features and discrete features according to data types, and optionally, processing the continuous features using maximum-minimum normalization and processing the discrete features using one-hot encoding.
[0075] The label data processing specifically includes: adding a label column to the A party data set, setting its label data to "1", representing positive sample data; adding a label column to the B party and C party data sets respectively, setting its label data to "0", representing unlabeled data.
[0076] S12: Dataset D for Party B and Party C B and D C Perform sample encryption alignment according to their sample IDs, retain the aligned sample data of Party B and Party C, discard the unaligned sample data, and obtain n samples. The aligned dataset of Party B is The C-square dataset is express The corresponding label, express Corresponding labels, where i = 1,…,n.
[0077] Optionally, an RSA algorithm and a hash function may be used to perform cryptographic alignment of samples.
[0078] S13: Dataset D A 、 and According to the sample ID, the three-way aligned samples are used as positive samples to form the positive sample data set P = {(xp i ,yp i )},in Respectively represent the i-th sample feature vector after alignment of party A, party B, and party C, yp i ∈{1} is the positive sample label, |P| represents the number of positive samples, i=1,…,|P|; the three unaligned samples are regarded as unlabeled samples, forming the unlabeled sample dataset U={(xu i ,yu i )},in Respectively represent the i-th sample feature vector after alignment of party B and party C, yu i ∈{0} is the unlabeled sample label, |U| represents the number of unlabeled samples, i=1,…,|U|.
[0079] Optionally, an RSA algorithm and a hash function may be used to perform cryptographic alignment of samples.
[0080] S2: Establish a process for predicting candidate recommendations of unlabeled samples, and execute this process in a loop for M rounds. Optionally, the total number of loop rounds M is set to 10. The prediction process for the mth round is as follows: randomly extract |P| samples from U with replacement, where |P| represents the number of samples in P, and form a negative sample dataset N with these |P| unlabeled samples m . By P and N m Composition of training set The unsampled samples in U form the prediction set Construct a vertical federated model with Gradient Boosting Decision Tree (GBDT) as the base learner. Training will be conducted on Input into the trained model for prediction and get The score of each sample in is used as the predicted score of the sample.
[0081] S21: Use the bootstrap sampling method to randomly extract |P| samples from U with replacement, and use these |P| unlabeled samples to form the mth round of negative sample dataset in are the feature vectors of party B and party C in the i-th negative sample of round m, Represents the label of the i-th negative sample in the m-th round, i=1,…,|P|,|N m | represents the negative sample dataset N for the mth round m The number of samples. The positive sample dataset P and the negative sample dataset N of the mth round m Constitute the corresponding m-th round training set in They represent the feature vectors of party B and party C in the i-th sample of round m, represents the label of the i-th sample in the m-th round, i = 1, 2, ..., 2|P|; the unextracted samples in the unlabeled sample dataset U constitute the corresponding m-th round prediction set in Represents the feature vector of the jth sample in the mth round, j=1,2,...,|U|-|N m |.
[0082] S22: Use the GBDT algorithm as the base learner to build a vertical federation model through T decision trees Integration of Training, input data Each sample To predict its mth round output Where T represents the total number of decision trees in the mth round. Optionally, the value of the total number of decision trees T is set to 40. Represents the prediction result of the t-th tree in the m-th round, Represents the feature vector of the i-th sample in the m-th round, i=1,2,...,2|P|. According to the prediction output and the true label The gradient value of the loss function between them is used to establish the first-order derivative and second-order derivative gradient histograms, and the gradient histograms of all features of both sides are combined to find the global optimal segmentation of the current node to construct the optimal decision tree.
[0083] First, initialize each sample in the mth round The prediction results A random value, optional. The random value of is -1 or 1, and then the specific process of federated training of the t-th tree in the m-th round is as follows:
[0084] S221: Starting from party B, first for each sample of party B Calculate the first-order gradient of the loss function of the m-th round model and the second-order gradient where i=1,2,...,2|P|, Represents the paired samples aggregated by the first t-1 trees in the mth round The prediction results, represents the true label of the i-th sample in the m-th round, is the loss function. and Encrypt and get and Party B will and Send to Party C.
[0085] Optionally, the loss function here uses the mean square error loss function, and the homomorphic encryption scheme used is Paillier homomorphic encryption. Indicates that data a is homomorphically encrypted.
[0086] S222: For Party C, build the mth round gradient histogram based on its own feature data and send the encrypted gradient histogram to Party B. The specific process is as follows:
[0087] S2221: For the current m-th round sample All C square features in the , sort all samples according to the characteristic value of each feature, and then divide the sorted samples into q categories by bucketing, and get the corresponding feature threshold of each category Where k represents the feature number, Indicates the threshold for the qth category of feature number k in round m. Optionally, category q is set to 10 here.
[0088] S2222: Based on the mth round obtained from Party B and Party C aggregates the encrypted gradient information and constructs the mth round encrypted gradient histogram, i.e.
[0089] Where i = 1, 2, ..., 2|P|, v = 1, 2, ..., q.
[0090] S2223: Party C calculates the mth round and Send to Party B.
[0091] S223: Party B decrypts the mth round of encrypted gradient histogram from Party C, and according to the splitting gain calculation formula, enumerates each feature gradient histogram to calculate the optimal solution, finds the global optimal split point, and returns the split information to Party C for analysis. The specific process is as follows:
[0092] S2231: Party B aggregates the first-order gradient and second-order gradient of all samples in the current node space and executes Where I represents all samples of the current node.
[0093] S2232: Party B receives the mth round from Party C and Decrypt and get the mth round decryption value and The following calculations are performed on all categories of all features of party C in turn to obtain and
[0094] Among them I L Represents the sample space of the left child node after segmentation, I R represents the sample space of the right child node after segmentation, represents the sum of the first-order gradients of the loss function of all samples in the sample space of the left child node in the mth round, represents the sum of the second-order gradients of the loss function of all samples in the sample space of the left child node in the mth round, represents the sum of the first-order gradients of the loss function of all samples in the sample space of the right child node in the mth round, Represents the sum of the second-order gradients of the loss function of all samples in the sample space of the right child node in the mth round.
[0095] S2233: Calculate the optimal split value of the current node in round m
[0096] Where λ is a hyperparameter. Optionally, the hyperparameter λ is set to 0.5.
[0097] S2234: All thresholds for each feature of the sample You can get one value, select the largest one value, determine the feature threshold as the global optimal segmentation of the mth round, the global optimal segmentation can be used [participant, feature number (K opt ), threshold number (V opt )] and the feature number (K opt) and threshold number (V opt ) is returned to Party C.
[0098] S224: Party C receives the feature number K sent from Party B. opt and threshold number V opt Determine the threshold of the feature and divide the current sample space. Then, Party C establishes a lookup table locally to record the threshold of the selected feature, forming a record [record number, feature, threshold], and stores the record number and the sample space on the left after division (I L ) is returned to Party B.
[0099] S225: Party B will receive [record number, I L ] divides the current node and associates the current node with [participant, record number]. Party B synchronizes the current node division information with Party C and proceeds to split the next node.
[0100] S226: Iterate steps S222 to S225 until the training stop condition or the maximum depth of the tree is reached.
[0101] S23: Use the trained GBDT decision model for the mth round to test the mth round of the test set Make predictions and obtain the mth round test set Each sample The prediction score of , traverse all samples in U, and find the one with Sample xu with the same ID i , and put The prediction score is assigned to xu i , as a sample i The prediction score of the mth round is Indicates, where i=1,…,|U| is the sample number in the unlabeled dataset U. If the sample xu in U i Not present in round m In the case of i The corresponding prediction score for the mth round Equal to 0. The specific process is as follows:
[0102] S231: Party B queries the [Party, Record Number] record associated with the current node. Based on this record, Party B sends the sample number and record number to Party C and inquires about the next tree search direction, i.e., the left child node or the right child node.
[0103] S232: After receiving the sample ID and record ID, Party C compares the value of the corresponding feature in the sample to be labeled with the threshold value in the record [record ID, feature, threshold] in the local lookup table to determine the next tree search direction. Party C then sends the search decision to Party B.
[0104] S233: Party B receives the search decision from Party C and goes to the corresponding child node.
[0105] S234: Iterate steps S231 to S233 until a leaf node is reached, and the corresponding classification label and the weight of the label are obtained, and the weight of the sample is obtained. The corresponding sample xu in U i The prediction score of the mth round
[0106] Where I represents the sample space of leaf nodes and λ is a hyperparameter. Optionally, the hyperparameter λ is set to 0.5.
[0107] S3: Each sample xu in U i The sum of the M-round prediction scores and the sum of the number of occurrences in the M-round prediction set, calculate the sample xu i The probability of predicting a positive sample. Sort all samples in U by their probability of being positive, select the top-ranked samples as reliable positive samples based on prior knowledge, and add them to the positive sample dataset P while removing them from U. Optionally, the total number of rounds M is set to 10.
[0108] S31: The sum of the prediction scores of each sample in U And the sum of its occurrence times in the M rounds of prediction sets, calculate the probability ρ of the sample prediction being a positive sample i , the calculation formula is as follows:
[0109] in is the indicator function, indicating that if the sample xu in U i The test set present in round m If , then I = 1, otherwise I = 0.
[0110] S32: Use the probability ρ for all samples in U i Sort in descending order, select the top θ samples as reliable positive samples, add them to the positive sample dataset P, and delete them from U, where the value of θ is set based on prior knowledge. Optionally, θ is set to 0.1.
[0111] S4: Repeat steps S2-S3 until the preset maximum number of iterations is reached. Optionally, the maximum number of iterations is set to 5. In each iteration, for the reliability and accuracy of the recommendation, only a small number of reliable positive samples can be selected in each iteration, so in order to select a certain amount of reliable positive samples for batch recommendation, multiple iterations are required. In each iteration, step S3 will select some reliable positive samples from the unlabeled dataset U and add them to the positive sample dataset P, while deleting them from the unlabeled dataset U. In this case, the sample size of |P| will increase with the increase in the number of iterations, and the sample size of |U| will decrease with the increase in the number of iterations until the preset maximum number of iterations is reached. Therefore, all reliable positive samples selected from U during multiple iterations can be used as potential provident fund depositors and recommended to the provident fund in batches.
[0112] For example, the personal basic information, personal income tax, property management tax and other application scenario-related information of a group of customers in Party B's tax data; the personal basic information, medical insurance, pension insurance, unemployment insurance and other application scenario information of this group of customers in Party C's social security data are pre-processed and put into the trained longitudinal federal GBDT model respectively. Each party determines whether there are potential users of provident fund contributions for flexible employment personnel among this group of customers based on the characteristics of this group of customers, and then recommends the customers who are judged as potential users to the provident fund party in batches.
[0113] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not limiting. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical solutions of the present invention can be modified or replaced by equivalents without departing from the purpose and scope of the technical solutions, which should all be included in the scope of the claims of the present invention.
Claims
1. A method for recommending potential users of financial products based on multi-party semi-supervised learning, characterized by: The following steps are involved: S1: Establish a multi-party dataset for recommending potential users of financial products, which contains information of users who purchase financial products and other multi-party user information, and perform preprocessing and sample alignment on the multi-party dataset for recommending potential users of financial products, to construct a positive sample dataset and an unlabeled sample dataset; S2: Perform random sampling with replacement in the unlabeled sample data set to establish a negative sample data set; construct a training set using the negative sample data set and the positive sample data set, and construct a prediction set using the samples that have not been sampled in the unlabeled sample data set; construct a longitudinal federated learning model based on the base learner, perform training on the training set, and perform prediction on the prediction set to obtain a prediction score for each sample in the prediction set; S3: Repeat the sampling, training and prediction process of step S2 multiple times; calculate the probability of the sample being predicted as a positive sample according to the sum of the prediction scores of each sample in the unlabeled sample data set and the number of times it appears in the prediction set; sort all samples in the unlabeled sample data set from large to small according to the probability of positive samples, select the top-ranked samples as reliable positive samples according to prior knowledge, and add them to the positive sample data set, and delete them from the unlabeled sample data set; S4: Repeat steps S2-S3 until the preset maximum number of iterations is reached; thereby, all reliable positive samples selected from the unlabeled sample data set are used as potential users and accurately recommended in batches to financial product providers.
2. The method for recommending potential users of financial products based on multi-party semi-supervised learning according to claim 1, characterized in that: The multi-party data set for recommending potential users of financial products in step S1 includes party A who has information about users who have purchased financial products, and party B and party C, other multi-party data sources other than party A. Step S1 specifically includes the following steps: S11: Perform data preprocessing on the multi-party datasets recommended by potential users of financial products, including redundant data processing, missing value processing, outlier processing, data standardization processing, and label data processing to obtain the A-party dataset Represents the ith sample feature vector of party A, with a vector dimension of d A , express Corresponding labels, where i = 1,…,a; Party B dataset Represents the i-th sample feature vector of party B, with a vector dimension of d B , express Corresponding labels, where i = 1,…,b; C-square data set Represents the ith sample feature vector of C square, the vector dimension is d C , express The corresponding labels, where i = 1,…,c; S12: Dataset D for Party B and Party C B and D C Perform sample encryption alignment according to their sample IDs, retain the aligned sample data of party B and party C, discard the unaligned sample data, and obtain n samples; the aligned data set of party B is The C-square dataset is express The corresponding label, express The corresponding labels, where i = 1,…,n; S13: For dataset D A , and The samples are encrypted and aligned according to their sample IDs, and the samples aligned by the three parties are taken as positive samples to form the positive sample data set P = {(xp i ,yp i )},in Respectively represent the ith sample feature vector after alignment of party A, party B, and party C, yp i ∈{1} is the positive sample label, |P| represents the number of positive samples, i=1,…,|P|; the three unaligned samples are regarded as unlabeled samples, forming an unlabeled sample dataset U={(xu i ,yu i )},in Respectively represent the i-th sample feature vector after alignment of party B and party C, yu i ∈{0} is the unlabeled sample label, |U| represents the number of unlabeled samples, i=1,…,|U|.
3. The method for recommending potential users of financial products based on multi-party semi-supervised learning according to claim 2, characterized in that: In step S11, the redundant data processing specifically includes: judging whether the data is repeated through certain fields in the data record, retaining only one of the repeated data, deleting the other repeated data from the data set, and retaining a backup of the original data for backtracking and comparative analysis when necessary; The missing value processing specifically includes: performing missing rate statistics on each feature in the data, treating data with a number of missing features greater than half of the overall sample feature size as invalid data and eliminating them; and filling the remaining data using a specific method based on data distribution; The outlier processing specifically includes: confirming the threshold of the outlier according to the actual situation; visually analyzing the data to find data that exceeds the threshold; processing the outlier, and after processing, checking the data set again to ensure that the outlier has been processed and the basic characteristics of the data set have not changed significantly; The data standardization processing specifically includes: dividing the existing features into continuous features and discrete features according to data types, processing the continuous features by using maximum-minimum standardization, and processing the discrete features by using one-hot encoding; The label data processing specifically includes: adding a label column to the A data set, setting its label data to "1", representing positive sample data; adding a label column to the B and C data sets respectively, setting its label data to "0", representing unlabeled data.
4. The method for recommending potential users of financial products based on multi-party semi-supervised learning according to claim 3 is characterized by: The step S2 specifically includes: establishing a process for predicting unlabeled sample candidate recommendations, and executing the process in a loop for M rounds. The prediction process for the mth round is as follows: randomly extracting |P| samples from U with replacement, where |P| represents the number of samples in P, and combining these |P| unlabeled samples into a negative sample data set N m ; by P and N m Composition of training set The unsampled samples in U form the prediction set Construct a vertical federated model with Gradient Boosting Decision Tree (GBDT) as the base learner. training on Input into the trained model for prediction and get The score of each sample in is used as the predicted score of the sample; specifically, the following steps are included: S21: Use bootstrap sampling method to randomly extract |P| samples from U with replacement, and use these |P| unlabeled samples to form the mth round of negative sample data set in are the feature vectors of party B and party C in the i-th negative sample of the m-th round, represents the label of the i-th negative sample in the m-th round, i=1,…,|P|,|N m | represents the mth round of negative sample dataset N m The number of samples; from the positive sample data set P and The negative sample dataset N of the mth round m Constitute the corresponding m-th round training set in They represent the feature vectors of party B and party C in the i-th sample of the m-th round, represents the label of the i-th sample in the m-th round, i=1,2,...,2|P|; the unextracted samples in the unlabeled sample dataset U constitute the corresponding m-th round prediction set in Represents the feature vector of the jth sample in the mth round, j = 1, 2, ..., |U|-|N m |; S22: Use the GBDT algorithm as the base learner to build a vertical federation model through T decision trees Integrated pair For training, input data Each sample To predict its mth round output Where T represents the total number of decision trees in the mth round, represents the prediction result of the t-th tree in the m-th round, represents the feature vector of the i-th sample in the m-th round, i=1,2,...,2|P|; according to the predicted output and the true label The gradient value of the loss function between is used to establish the first-order derivative and second-order derivative gradient histograms, and the gradient histograms of all features of both sides are combined to find the global optimal segmentation of the current node to construct the optimal decision tree; S23: Use the trained GBDT decision model for the mth round to test the mth round test set Make predictions and obtain the mth round test set Each sample The prediction score of , traverse all samples in U, and find the corresponding Samples with the same ID i , and put The prediction score is assigned to xu i , as a sample i The prediction score of the mth round is Indicates, where i = 1, ..., |U| is the sample number in the unlabeled dataset U; if the sample xu in U i Not present in round m In the case of i The corresponding prediction score for the mth round is equal to 0.
5. The method for recommending potential users of financial products based on multi-party semi-supervised learning according to claim 4, characterized in that: In step S22, each sample of the mth round is first initialized The prediction results is a random value, and then the specific process of federated training of the t-th tree in the m-th round is as follows: S221: Starting from party B, first for each sample of party B Calculate the first-order gradient of the loss function of the m-th round model and the second-order gradient where i = 1, 2, ..., 2|P|, Represents the pair samples aggregated from the first t-1 trees in the mth round The prediction results, represents the true label of the i-th sample in the m-th round, is the loss function; using additive homomorphic encryption and Encrypt and get and Party B will and Send to Party C; S222: For Party C, the mth round gradient histogram is established according to its own feature data, and the encrypted gradient histogram is sent to Party B. S223: Party B decrypts the m-th round encrypted gradient histogram from Party C, enumerates each feature gradient histogram to calculate the optimal solution according to the splitting gain calculation formula, finds the global optimal splitting point, and returns the splitting information to Party C for analysis; S224: Party C receives the feature number K sent from Party B opt and threshold number V opt Determine the threshold of the feature and divide the current sample space; then Party C establishes a lookup table locally to record the threshold of the selected feature, forming a record [record number, feature, threshold], and the record number and the sample space on the left after the division (I L ) is returned to Party B; S225: Party B receives [record number, I L ] divides the current node and associates the current node with [participant, record number]; Party B synchronizes the division information of the current node with Party C and proceeds to the division of the next node; S226: Iterate steps S222 to S225 until the training stop condition or the maximum depth of the tree is reached.
6. The method for recommending potential users of financial products based on multi-party semi-supervised learning according to claim 5, characterized in that: Step S222 specifically includes the following steps: S2221: For the current mth round of samples All C square features in the , sort all samples according to the feature value of each feature, and then divide the sorted samples into q categories by buckets, and get the corresponding feature thresholds of each category Where k represents the feature number, Indicates the threshold of the qth category of the feature numbered k in the mth round; S2222: Based on the mth round obtained from Party B and Party C aggregates the encrypted gradient information and constructs the mth round of encrypted gradient histogram, namely Where i = 1, 2, ..., 2|P|, v = 1, 2, ..., q; S2223: Party C calculates the mth round and Send to Party B.
7. The method for recommending potential users of financial products based on multi-party semi-supervised learning according to claim 5, characterized in that: Step S223 specifically includes the following steps: S2231: Party B aggregates the first-order gradient and second-order gradient of all samples in the current node space and executes Where I represents all samples of the current node; S2232: Party B receives the mth round from Party C and Decrypt and get the mth round decryption value and The following calculations are performed on all categories of all features of party C in turn to obtain and Among them I L Represents the sample space of the left child node after segmentation, I R represents the sample space of the right child node after segmentation, represents the sum of the first-order gradients of the loss function of all samples in the left child node sample space in the mth round, represents the sum of the second-order gradients of the loss function of all samples in the left child node sample space in the mth round, represents the sum of the first-order gradients of the loss function of all samples in the right child node sample space in the mth round, Represents the sum of the second-order gradients of the loss function of all samples in the right child node sample space in the mth round; S2233: Calculate the optimal split value of the current node in round m Where λ is a hyperparameter; S2234: For all thresholds of each feature of the sample Can get one value, select the largest one value, determine the feature threshold as the global optimal segmentation of the mth round, the global optimal segmentation is [participant, feature number (K opt ), threshold number (V opt )], and the feature number (K opt ) and threshold number (V opt ) is returned to Party C.
8. The method for recommending potential users of financial products based on multi-party semi-supervised learning according to claim 4, characterized in that: Step S23 specifically includes the following steps: S231: Party B queries the [participant, record number] record associated with the current node; based on the record, Party B sends the sample number to be labeled and the record number to Party C, and inquires about the next tree search direction, i.e., the left child node or the right child node; S232: After receiving the sample number and record number to be labeled, Party C compares the value of the corresponding feature in the sample to be labeled with the threshold in the record [record number, feature, threshold] in the local lookup table to obtain the next tree search direction; then, Party C sends the search decision to Party B; S233: Party B receives the search decision from Party C and goes to the corresponding child node; S234: Iterate steps S231 to S233 until a leaf node is reached, and the corresponding classification label and the weight of the label are obtained, thereby obtaining the weight of the sample. The corresponding sample xu in U i The prediction score of the mth round Where I represents the sample space of leaf nodes and λ is a hyperparameter.
9. The method for recommending potential users of financial products based on multi-party semi-supervised learning according to claim 4, characterized in that: In step S3, each sample xu in U i The sum of the M-round prediction scores and the sum of the number of occurrences in the M-round prediction set is used to calculate the sample xu i Predict the probability of being a positive sample; sort all samples in U from large to small according to the probability of positive samples, select the top-ranked samples as reliable positive samples based on prior knowledge, and add them to the positive sample data set P, and delete them from U at the same time, which specifically includes the following steps: S31: The sum of the predicted scores for each sample in U And the sum of its occurrence times in the M rounds of prediction sets, calculate the probability ρ of the sample prediction being a positive sample i , the calculation formula is as follows: in is the indicator function, indicating that if the sample xu in U i The test set present in round m If , I = 1, otherwise I = 0; S32: Use the probability ρ for all samples in U i Sort in descending order, select the top θ samples as reliable positive samples, add them to the positive sample dataset P, and delete them from U, where the value of θ is set according to prior knowledge.
Citation Information
Patent Citations
Method and device for constructing GBDT model and prediction method and device
CN110348580A
Potential customer identification method and system based on longitudinal federated learning, and medium
CN113240461A
Data protection method based on WOE mask in multi-party longitudinal federated learning LightGBM training
CN113779608A
Business information recommendation method and device based on multi-party high-dimensional data longitudinal federal learning
CN114677200A
Financial product potential user recommendation method based on multi-party semi-supervised learning
CN116523602A
Cited By
Financial product recommendation method, system and equipment based on semi-Gaussian sampling and medium
CN119205281A
Intelligent factory lean manufacturing fault detection method based on federal incremental learning
CN119723157A
