Improved data processing and CTR federal recommendation method based on federal learning

Abstract

The invention relates to an improved data processing and CTR federated recommendation method based on federated learning, and the method comprises the following steps: 1, generating a federated recommendation data set with a weight based on a processing strategy of a recommendation data set of a data set sample size and a positive and negative label ratio coefficient lambda; 2, based on the federal recommendation data set with the weight obtained in the step 1, two types of federal baseline recommendation models of LR and GBDT and a federal CTR recommendation model are established respectively, and the federal CTR recommendation model comprises a federal FM model and a federal DeepFM model; and step 3, obtaining a recommendation result based on the data set processing method and the federal recommendation model established in the steps 1 and 2. According to the method, the technical problem of how to segment the original recommendation data set to simulate the real federal recommendation data set can be solved, and model performance similar to centralized training and a traditional federal recommendation model can be obtained on the premise of not leaking data privacy and compression model performance.

Description

Technical Field

[0001] The present invention belongs to the field of artificial intelligence technology and relates to an improved data processing and CTR federated recommendation method, in particular to an improved data processing and CTR federated recommendation method based on federated learning. Background Art

[0002] With the continuous development of technologies such as machine learning and artificial intelligence, Internet finance has also entered a new era: in the current increasingly fierce competition for traffic, platforms hope to predict users' interests and hobbies through every browse and click of the user, so as to recommend items that users are more interested in to obtain traffic and revenue.

[0003] However, due to the increasingly improved laws and regulations on data privacy protection and the "data island" problem between different institutions, the click-through rate (CTR) recommendation model has long been often unable to obtain enough high-quality data due to the scale of the enterprise itself, resulting in poor performance of the recommendation model, or because of the high-dimensional and sparse characteristics of the recommended data, the training requires huge data communication costs. Therefore, as a distributed learning framework that emphasizes data privacy protection, federated learning and related technologies have gradually entered the CTR recommendation research field.

[0004] Federated Learning (FL) has gradually gained people's attention for its mode of not sharing real data with data privacy protection technology. Its novel data privacy protection + distributed training method not only attracts research from academic circles, but also its characteristics of protecting data owners from data leakage in the recommendation field and feeding more data to the model to improve model performance make it more in line with the needs of industry and commerce, such as intelligent search engines, Internet e-commerce advertising recommendations, bank financial institutions' lending and credit assessment in the commercial field. The application of federated learning in these fields can provide developers with better models while protecting user privacy, thereby obtaining better user feedback and potential commercial benefits. And as a novel framework, federated learning can be regarded as a treasure vein that has just begun to be developed, with very considerable research prospects.

[0005] The difference between the recommendation dataset and other datasets is that the number of samples in the dataset is much higher than that of other datasets, and the features in a sample often have many missing values ​​or zero values. For category features, since there may be multiple products in the same category, the traditional one-hot method may result in a very large vector. Such high-dimensional and sparse data samples will cause problems such as low learning rate of model training, and the experimental conditions cannot refer to other federated datasets. For example, the FeMnist dataset is generated by 13 server clusters collaborating to generate a new recommendation dataset, which requires a huge amount of resources and the participation of multiple institutions in collaboration. In addition, this article hopes to simulate the recommendation dataset of multiple clients in reality as much as possible, that is, the dataset of each client should have a certain feature distribution tendency. For example, the dataset of clients whose main business scenario is movies should have a certain preference for movie items, and the click rate of cars in the dataset of automobile companies should account for a high proportion.

[0006] Therefore, how to divide a complete recommendation dataset into local datasets simulating different clients is a very important issue.

[0007] The present invention proposes an improved data processing and CTR federated recommendation method based on federated learning.

[0008] After searching, no published technical documents identical or similar to the present invention were found. Summary of the invention

[0009] The purpose of the present invention is to overcome the shortcomings of the prior art and propose an improved data processing based on federated learning and a corresponding CTR federated recommendation method, which can solve the technical problem of how to split the original recommendation data set to simulate the real federated recommendation data set, and can obtain model performance similar to that of centralized training and traditional federated recommendation models without leaking data privacy and compressing model performance.

[0010] The present invention solves the practical problem by adopting the following technical solutions:

[0011] An improved data processing and CTR federated recommendation method based on federated learning includes the following steps:

[0012] Step 1: A weighted federated recommendation dataset is generated based on the processing strategy of the recommendation dataset based on the dataset sample size and the positive-negative label ratio coefficient λ;

[0013] Step 2: Based on the weighted federated recommendation dataset obtained in step 1, two types of federated baseline recommendation models, LR and GBDT, and a federated CTR recommendation model are established respectively. The federated CTR recommendation model includes a federated FM model and a federated DeepFM model.

[0014] Step 3: Based on the data set processing method and federated recommendation model established in steps 1 and 2, obtain the recommendation results.

[0015] Advantages and beneficial effects of the present invention:

[0016] 1. The present invention proposes an improved data processing and CTR federated recommendation method based on federated learning, which can solve the problem of how to split the original recommendation data set to simulate the real federated recommendation data set, and proposes a processing strategy for the recommendation data set based on the data set sample size and the positive-negative label ratio coefficient λ.

[0017] 2. Using the federated dataset generated by the processing strategy of the recommended dataset based on the dataset sample size and the positive-negative label ratio coefficient λ, improved horizontal and vertical federated recommendation models such as LR, GBDT, FM, and DeepFM were designed respectively, and the feasibility and innovation of the present invention were verified through training. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] Figure 1 It is a schematic diagram of the horizontal federated logistic regression model of the present invention;

[0019] Figure 2 It is a diagram of the training process of the longitudinal federated LR model of the present invention;

[0020] Figure 3 It is a schematic diagram of the horizontal federated GBDT model of the present invention;

[0021] Figure 4 It is a schematic diagram of clustering results in a calculation example of the present invention;

[0022] Figure 5 This is a comparison chart of the positive and negative sample ratios of the criteo original and sub-datasets in the example of the present invention. DETAILED DESCRIPTION

[0023] The embodiments of the present invention are further described in detail below with reference to the accompanying drawings:

[0024] An improved data processing and CTR federated recommendation method based on federated learning includes the following steps:

[0025] Step 1: A weighted federated recommendation dataset is generated based on the processing strategy of the recommendation dataset based on the dataset sample size and the positive-negative label ratio coefficient λ;

[0026] The specific steps of step 1 include:

[0027] (1) Define federated learning clients and data: The set of k clients is:

[0028] N={n1,n2,n3,……,n k}#(1)

[0029] where n i represents the i-th client, the original recommendation dataset is D, and according to the definition of federated learning, the dataset D is divided into:

[0030] D = {D1, D2, D3, ……, D k} #(2)

[0031] where D i is the dataset of the i-th client, and the value range of i is 0 < i ≤ k. The j-th sample data in the dataset D i is:

[0032]

[0033] where, x i is the i-th feature of this sample, and the set of all m samples

[0034]

[0035] constitutes the feature set Xi of client i i , and the generation and processing of the federated recommendation dataset are the process of formula (2).

[0036] (2) Given the number k of clients participating in federated training, find the sample size M of the original dataset D, then the maximum sample size of the dataset Di of a single client is M / k, and the dataset D is divided into D = {D i , D + , D -} two sets of positive and negative samples, let

[0037] (3) Use the clustering algorithm to cluster the data features, and then according to the need, take the average to extract data to form clients or generate a separate dataset for each cluster; adopt K-means clustering, through the K-means function based on sklearn, and use the Euclidean distance as the data sample similarity measurement method, and its calculation formula is:

[0038]

[0039] where, x and y are the sample and the selected cluster center vector respectively, and d(x, y) is the distance from this sample point to this cluster center.

[0040] (4) Assume the situation of "there is a client a with the minimum data volume P and a client b with the most unbalanced positive and negative sample ratio Q", so it is improved to:

[0041] Assume that among all k clients, a client i with a sample size of P has a positive-negative sample ratio of q, and the client with the positive-negative sample ratio closest to 1 in the client set has a positive-negative sample ratio of g. Then the model training and federation information of client i, such as model parameters, needs to be accompanied by a client weight coefficient λ, which is calculated as follows:

[0042]

[0043] Through the above steps, a weighted federated recommendation dataset can be obtained.

[0044] The working principle of step 1 is:

[0045] The difference between the recommendation dataset and other datasets is that the number of samples in the dataset is much higher than that of other datasets, and the features in a sample often have many missing values ​​or zero values. For category features, since there may be multiple commodities in the same category, the traditional one-hot method may result in a very large vector. Such high-dimensional sparse data samples will cause problems such as low learning rate of model training, and the experimental conditions cannot refer to other federated datasets. For example, the FeMnist dataset is generated by 13 server clusters to generate a new recommendation dataset, which requires a lot of resources and the participation of multiple institutions in the collaboration. In addition, the present invention hopes to simulate the recommendation dataset of multiple clients in reality as much as possible, that is, the dataset of each client should have a certain characteristic distribution tendency. For example, the dataset of a client whose main business scenario is movies should have a certain bias towards movie items, and the click rate of cars in the dataset of car companies should account for a high proportion. Therefore, how to divide a complete recommendation dataset into local datasets that simulate different clients is a very important issue.

[0046] For the recommended dataset in federated learning, the following aspects need to be considered for reasonable division:

[0047] 1. Source of the data set: The data in federated learning usually comes from different institutions or individuals, and it is necessary to ensure the diversity and fairness of the data sources.

[0048] 2. Dataset size: The size of the dataset usually determines the training effect of the model. It is necessary to ensure that the size of the dataset provided by each participant is relatively balanced.

[0049] 3. Category distribution of the data set: For the data set of classification problems, it is necessary to consider the distribution of different categories in the data set to ensure that the data set provided by each participant contains a similar number of different categories of data.

[0050] Timeliness of the dataset: Some datasets may be time-sensitive, and it is necessary to ensure that the timeliness of the datasets provided by the participants is similar, such as the Amazon dataset.

[0051] Privacy of the dataset: The dataset may contain sensitive information, and it is necessary to ensure that the privacy of the datasets provided by each participant is protected. Based on the above considerations, the following partitioning methods can be selected for design:

[0052] A. Random partitioning: Randomly allocate the dataset to different clients. This method is simple and easy to implement, but may lead to an unbalanced data distribution, thus affecting the performance of the model.

[0053] B. Clustering partitioning: Cluster the dataset and partition it according to the characteristics of the data, and allocate the data with similar characteristics to the same client. This method can ensure a relatively high data similarity among clients, but may require preprocessing of the features.

[0054] C. Partitioning by label: Partition according to the data labels and allocate the data with the same label to the same client. This method can ensure that each client has a sufficient number of samples, but if the label distribution is unbalanced, it may lead to insufficient data for some clients.

[0055] Since the recommended dataset has consistent labels, an improved method combining A and B can be used to partition the dataset.

[0056] Define the federated learning clients and data: The set of k clients is:

[0057] N = {n1, n2, n3, ……, n k} #(1)

[0058] where n i represents the i-th client, the original recommended dataset is D, and according to the definition of federated learning, the dataset D is partitioned into:

[0059] D = {D1, D2, D3, ……, D k} #(2)

[0060] where D i is the dataset of the i-th client, and the value range of i is 0 < i ≤ k. The j-th sample data in the dataset D i is:

[0061]

[0062] where x i is the i-th feature of this sample, and the set of all m samples

[0063]

[0064] The feature set X that constitutes client i i ,The generation and processing of the federated recommendation dataset is the process of formula (2).

[0065] The present invention improves the design idea based on methods A and B:

[0066] 1. Given the number of clients k participating in federated training, find the sample size M of the original dataset D, then the single client dataset D i The maximum sample size is M / k, but this approach may lead to imbalanced sample labels. Therefore, the dataset D is divided into D = {D + ,D -}Two sets of positive and negative samples, let

[0067] 2. Although step 1 ensures the balance of data samples, the sample features are not reasonably divided and are still very random. You can consider using a clustering algorithm to cluster the data features, and then take average extraction of data to form a client or generate a separate data set for each cluster as needed. K-means clustering can be used, through the K-means function based on sklearn, using the Euclidean distance as the data sample similarity measurement method, and the calculation formula is:

[0068]

[0069] Among them, x, y are the sample and the selected cluster center vector respectively, and d(x, y) is the distance from the sample point to the cluster center.

[0070] 3. Although the methods in steps 1 and 2 generate independent and identically distributed datasets that are very suitable for training and are conducive to model convergence, they do not conform to the actual federated learning scenario in reality, because the data volume and positive and negative sample ratios of multiple institutions participating in the training are often different. This is particularly obvious in the recommendation dataset, and there is often a big gap between the sample labels, that is, whether the user clicks on them. Therefore, it is necessary to assume that "there is a client a with the smallest data volume P and a client b with the most unbalanced positive and negative sample ratio Q", so it can be improved to:

[0071] Assume that among all k clients, a client i with a sample size of P has a positive-negative sample ratio of q (in the click-through rate recommendation dataset, the number of positive samples is much smaller than the number of negative samples), and the client with the positive-negative sample ratio closest to 1 in the client set has a positive-negative sample ratio of g. Then, the model training and federation information of client i, such as model parameters, needs to be accompanied by a client weight coefficient λ, which is calculated as follows:

[0072]

[0073] Through the above steps, a weighted federated recommendation dataset can be obtained.

[0074] Step 2: Based on the weighted federated recommendation dataset obtained in step 1, two types of federated baseline recommendation models, LR and GBDT, and a federated CTR recommendation model are established respectively. The federated CTR recommendation model includes a federated FM model and a federated DeepFM model.

[0075] The specific steps of step 2 include:

[0076] (1) Establishing a logistic regression model

[0077] The value y=f(x) output by the linear regression expression has a range of (-∞,∞), while the predicted value of the binary classification problem for click-through rate recommendation is [0,1]. Therefore, the sigmoid function is selected to map the linear regression output result to the interval [0,1]:

[0078] y=f(x)=w T x+b#(7)

[0079]

[0080] The output y of the linear regression model is used as the input of the sigmoid function to form the logistic regression model LR, which is used as a benchmark model for many experiments. The model expression is shown in Formula 9:

[0081]

[0082] (2) Establishing a horizontal federated logistic regression model

[0083] In each round of iteration, clients A and B train the LR model on their own data, and then encrypt the gradients of their respective trained models and upload them to a third-party assistant C. Assistant C only has the keys of clients A and B. Its task is to aggregate the model gradients uploaded by A and B. Since it is homomorphically encrypted, some mathematical calculations can be performed directly to form a federated model gradient, which is then distributed to clients A and B to update the local model. During this period, strategies such as early termination are set like traditional machine learning models. The training is repeated until the model converges, and finally the LR model effects obtained by clients A and B are consistent.

[0084] (3) Establishing a vertical federated LR model

[0085] Training the longitudinal federated logistic regression model requires splitting the LR loss function. The LR loss function F(w) is shown in Formula 11.

[0086] Among them, p is the probability of label 1 as follows, where w T is the transposed vector of vector w.

[0087]

[0088] The transformed longitudinal federated LR loss function formula 12 is obtained by polynomial approximation expansion:

[0089]

[0090] Assuming that client A holds the labels and client B only holds the feature dataset, the vertical federated LR model is trained;

[0091] At the beginning, clients A and B use a piece of data to perform sample alignment to establish a federated dataset, which is used to identify overlapping samples stored in the databases of the two parties. The federated training model is built based on these overlapping samples. The entire sample comparison process is performed in an encrypted form. Subsequently, clients AB initialize their own model parameters respectively.

[0092] The labeled A is calculated and encrypted on the same platform to obtain:

[0093]

[0094] The unlabeled B is calculated and homomorphically encrypted to obtain:

[0095]

[0096] Afterwards, A will [[d A ]] is sent to B, and B sends [[d B ]] is sent to A, then both A and B can calculate formula (12) to obtain the encrypted gradient [[g A ]] and [[g B ]], then A and B send the gradient to the assistant C, who decrypts it to get g A and g B Then it is returned to A and B, and then A and B use the decrypted gradient to update the model parameters.

[0097] (4) Establishing a horizontal federated GBDT model

[0098] For the GBDT model, each sub-client sends an encrypted local histogram to the central server. The histogram contains the sum of the gradient and hessian matrices. Finally, the server obtains the global histogram without directly obtaining any local histograms, finds the best tree segmentation solution and distributes it to each client.

[0099] (5) Establishing a vertical federated GBDT model

[0100] For the vertical federated GBDT model, the common user samples of each client participating in the training are found through the data privacy protection protocol and aligned under encryption; then the only client with the label uses the XGBoost tree learning algorithm to build a decision tree in each iteration; the client encrypts them before sending them to other clients. After receiving, each client uses the encrypted gradient and hessian matrix to calculate the encrypted feature histogram, and then encodes the features and split point values ​​and builds a lookup table; then the encoded features and split point values ​​and feature histograms are sent to the client with the label, and the client receives the feature histograms, decrypts them and finds the best gain; if the best gain feature belongs to a client, the encoded features and split point values ​​are sent back to the client.

[0101] (6) Establish a federal FM model;

[0102] According to the optimized FM model formula, the gradient vector v of each parameter of the FM model is derived i is feature x i Corresponding to the hidden vector, the length is k (k< <n),<·,·> It is the dot product operator of the vector, w0 is the bias term, which represents the basic prediction value when all features are zero, and wi is the first-order weight, which corresponds to the linear effect of feature xi.

[0103]

[0104] The clients can complete the constant term and linear term independently, but the cross term requires the clients to confirm the size of the overlapping feature space, and the assistant uses this as one of the model initialization parameters to distribute to each client to initialize the local model;

[0105] (7) Building a Federated DeepFM Model

[0106] The DeepFM model consists of the FM component on the left and the DNN component on the right. The high-dimensional sparse feature space serves as the common input of the FM and Deep layers.

[0107] The low-order features of the input data are completed by FM. The FM part is responsible for extracting only the second order; the DNN part is used to extract high-order features. In addition, the FM and Deep parts use the input and feature embedding together. The DNN part is a simple multi-layer fully connected neural network.

[0108] a (l+1) is the activation value (or output) of layer l+1l+1. In a neural network, the activation value is the result obtained by applying an activation function to the input.

[0109] σ is the activation function, which is often used to introduce nonlinearity, allowing the network to learn more complex patterns.

[0110] W(l) is the weight matrix of layer l. In a fully connected layer, it is a two-dimensional matrix whose dimension depends on the number of neurons in the previous layer and the number of neurons in the current layer.

[0111] a (l) is the activation value of the lth layer, which serves as the input of the current layer (the l+1th layer).

[0112] b (l) is the bias vector for layer l. Together with the weight matrix, it helps the model fit the bias in the data.

[0113] The output of each layer is

[0114] a (l+1) =σ(W (l) a (l) +b (l) )#(17)#

[0115] The output of the FM part is as follows:

[0116] (1) an AdditionUnit;

[0117] (2) Multiple inner product units.

[0118]

[0119] The prediction formula of DeepFM is as follows

[0120]

[0121] Similarly, by inputting FM, DeepFM and other models into the Model part of the homoNN component, the horizontal federation model construction of the CTR estimation model can be completed.

[0122] The working principle of step 2 is:

[0123] 1. Improved federated recommendation model

[0124] (1) Federated Logistic Regression Model

[0125] Logistic regression model (LR) is a statistical model widely used in classification problems. Because the value y=f(x) output by linear regression (Linear Regression, see formula 7) has a value range of (-∞,∞), and the predicted value of the binary classification problem for click-through rate recommendation is [0,1], the sigmoid function (see formula 8) is selected to map the linear regression output result to the interval [0,1].

[0126] y=f(x)=w T x+b#(7)

[0127]

[0128] The output y of the linear regression model is used as the input of the sigmoid function to form the logistic regression model LR, which is used as a benchmark model for many experiments. The model expression is shown in Formula 9.

[0129]

[0130] (2) Horizontal federated LR model

[0131] The horizontal federated logistic regression model (HF_LR) is as follows Figure 1 shown.

[0132] In this horizontal federated training process, the feature set X of the client data sets is the same, that is, the model structures of the two clients A and B in the figure are the same. In each round of iteration, both A and B train the LR model on their own data, and then encrypt the model gradients of their respective training and upload them to the third-party assistant C. Assistant C only has the keys of clients A and B. Its task is to aggregate the model gradients uploaded by A and B. Because it is homomorphically encrypted, some mathematical calculations can be performed directly to form a federated model gradient, which is then distributed to clients A and B to update the local model. During this period, early termination and other strategies can be set like traditional machine learning models. The training is cyclic until the model converges. Finally, the LR model effects obtained by clients A and B are consistent. Therefore, it can be guaranteed that more model gradients obtained by data training can be obtained without leaking data privacy, thereby improving model accuracy and accelerating convergence.

[0133] (3) Vertical federated LR model

[0134] The Vertical Fed-Logistics Regression (VF_LR) model requires splitting the LR loss function. The LR loss function F(w) is shown in Formula 11.

[0135] Where p is the probability of label 1 as follows, where wT is the transposed vector of vector w.

[0136]

[0137] In order to enable homomorphic encryption calculation, referring to the ideas of articles by Stephen Hardy and Professor Yang Qiang, the converted vertical federated LR loss function formula 12 is obtained by polynomial approximation expansion:

[0138]

[0139] Assuming that client A holds the labels and client B only holds the feature dataset, the training process of the vertical federated LR model is as follows: Figure 2 shown.

[0140] At the beginning, A and B can use a piece of data to perform sample alignment to establish a federated dataset. This sample alignment process is to identify overlapping samples stored in the databases of the two parties. The federated training model is built based on these overlapping samples. The entire sample comparison process is performed in an encrypted form, so the confidential information of both parties will not be leaked. Subsequently, clients AB initialize their own model parameters respectively.

[0141] The labeled A is calculated and encrypted on the same platform to obtain

[0142]

[0143] The unlabeled B is homomorphically encrypted to obtain

[0144]

[0145] Afterwards, A will [[d A ]] is sent to B, and B sends [[d B ]] is sent to A, then both A and B can calculate formula 4.6 to obtain the encrypted gradient [[g A ]] and [[g B ]], then A and B send the gradient to the assistant C, who decrypts it to get g A and g B Then it is returned to A and B, and then A and B use the decrypted gradient to update the model parameters.

[0146] (4) Horizontal federated GBDT model

[0147] For the GBDT model, each sub-client sends an encrypted local histogram to the central server. The histogram contains the sum of the gradient and hessian matrix. Finally, the server obtains the global histogram without directly obtaining any local histograms, finds the best tree segmentation solution and distributes it to each client. The horizontal federated GBDT model training process is as follows: Figure 3 shown.

[0148] (5) Vertical federated GBDT model

[0149] For the vertical federated GBDT model, the common user samples of each client participating in the training are found through the data privacy protection protocol and aligned under encryption; then the only client with the label uses the XGBoost tree learning algorithm to build a decision tree in each iteration. In order to protect the privacy of the gradient and hessian matrix, the client encrypts them before sending them to other clients. After receiving, each client uses the encrypted gradient and hessian matrix to calculate the encrypted feature histogram, and then encodes the features and split point values ​​(split_bin_val) and builds a lookup table; then the encoded features and split point values ​​and feature histograms are sent to the client with the label, and the client receives the feature histogram, decrypts them and finds the best gain. If the best gain feature belongs to a client, the encoded features and split point values ​​are sent back to the client.

[0150] 2. Improved federated CTR recommendation model

[0151] (1) HomoNN

[0152] The Homo NN process is similar to the horizontal LR model, and the models of each client have the same network structure. In this iteration, each client trains its own model based on its own data, and then uploads their own encrypted model parameters to the facilitator with a random mask attached. The facilitator aggregates these parameters to form the parameters of the federated model network, which are then distributed to each party to update their respective local models. The random masks used by the clients can be offset by adding a zero matrix.

[0153] (2) Homo Fed FM Model

[0154] According to the optimized FM model formula (see formula 15), the gradients of each parameter of the FM model can be derived (formula 16). i is feature x i Corresponding to the hidden vector, the length is k (k< <n),<·,·> It is the dot product operator of the vector, w0 is the bias term, which represents the basic prediction value when all features are zero, and wi is the first-order weight, which corresponds to the linear effect of feature xi.

[0155]

[0156]

[0158] Clients can complete constant terms and linear terms independently, but cross terms require clients to confirm the size of the overlapping feature space. The assistant uses this as one of the model initialization parameters and distributes it to each client to initialize the local model. It should be noted that the traditional FM model performs one-hot processing on dense features and sparse features and embeds them into one-dimensional tensors to take the inner product. In this process, high-dimensional data is compressed into a vector of fixed length, which destroys the influence of the importance of features on the accuracy of the model. This is also a content of optimization of subsequent recommendation models such as DeepFM.

[0159] (3) Homo Fed DeepFM Model

[0160] DeepFM consists of the FM component on the left and the DNN component on the right. The high-dimensional sparse feature space serves as the common input for the FM and Deep layers. Compared with its predecessor Wide&Deep, DeepFM no longer requires feature engineering and can simultaneously learn first-order, second-order, and deeper feature combinations. The latent variables v generated in the FM part will be used as the weights of the Embedding layer. It can be understood that the FM model has become a component of the entire DeepFM, and together with the DNN component, it constitutes a complete DeepFM model.

[0161] Specifically, FM is responsible for the low-order features of the input data, and usually only extracts the second order. The DNN part is used to extract high-order features. In addition, FM and Deep use the input and feature embedding together, which can greatly speed up the training speed and accuracy of the model.

[0162] The DNN part is a simple multi-layer fully connected neural network.

[0163] a (l+1) is the activation value (or output) of layer l+1l+1. In a neural network, the activation value is the result obtained by applying an activation function to the input.

[0164] σ is the activation function, which is often used to introduce nonlinearity, allowing the network to learn more complex patterns.

[0165] W(l) is the weight matrix of layer l. In a fully connected layer, it is a two-dimensional matrix whose dimension depends on the number of neurons in the previous layer and the number of neurons in the current layer.

[0166] a (l) is the activation value of the lth layer, which serves as the input of the current layer (the l+1th layer).

[0167] b (l)is the bias vector for layer l. Together with the weight matrix, it helps the model fit the bias in the data.

[0168] The output of each layer is

[0169] a (l+1) =σ(W (l) a (l) +b (l) )#(17)#

[0170] The output of the FM part is as follows:

[0171] (1) an AdditionUnit;

[0172] (2) Multiple inner product units.

[0173]

[0174] The prediction formula of DeepFM is as follows

[0175]

[0176] Similarly, by inputting FM, DeepFM and other models into the Model part of the homoNN component, the horizontal federation model construction of the CTR estimation model can be completed.

[0177] Step 3: Based on the data set processing method and federated recommendation model established in steps 1 and 2, obtain the recommendation results;

[0178] The specific method of step 3 is:

[0179] Based on the data set processing method and federated recommendation model established in steps 1 and 2, we verify the following through specific experiments:

[0180] (1) The federated dataset processing method based on the eigenvalue λ proposed in the present invention can obtain a federated dataset with obvious characteristic distribution, which better simulates the real federated training dataset environment;

[0181] (2) Based on the federated dataset generated by this method, the federated CTR prediction model of the present invention is used to implement the CTR federated recommendation model of the present invention in a small federated recommendation system through two baseline models such as LR and GBDT and two comparison paths of centralized training. The model performance is similar to that of centralized training and the recommendation results are obtained under the premise of protecting data privacy:

[0182] The present invention is further described below through specific examples:

[0183] By using the processing method based on the sample size of the dataset and the positive-negative label ratio coefficient λ proposed in the present invention, the present invention uses the Criteo recommended dataset as the experimental dataset.

[0184] The Criteo dataset is an online advertising dataset used by Criteo Labs for the Display Advertising Challenge. It contains the descriptive data feature values ​​of millions of online ads and user click information. The dataset has 40 attributes, including dense features from I1 to 13 columns, sparse features from C1 to 26 columns, and a label column, as shown in the following table. The value of 1 in the label column indicates that 11,745,438 ads have been clicked, and the value of 0 indicates that 34,095,179 ads have not been clicked. The sparse feature value is an anonymized 32-bit representation.

[0185]

[0186] The present invention adopts the labeled train.txt sample set in the Criteo dataset. The commonly used dataset preprocessing steps include: data cleaning, collection, bundling and transformation.

[0187] (1) Data cleaning: First, fill the missing values ​​for the nan values ​​in the data set. According to the table below, the proportion of missing values ​​is small, so dense features are filled with 0 and sparse features are filled with '0'.

[0188]

[0189] (2) Data clustering: Since the recommendation dataset Criteo is high-dimensional and sparse, it is meaningless to cluster the dataset directly. Therefore, we consider reducing the dimensionality of the dataset first. We perform TSNE dimensionality reduction and K-means clustering on the data, and divide the extracted 500,000 Criteo data into 10 sub-datasets according to clustering. The clustering results are as follows: Figure 4 shown.

[0190] Next, analyze the positive and negative sample ratios of the newly generated dataset and the original dataset, such as Figure 5 As shown in the figure (the ratio of positive and negative samples of the original dataset is 371711:128289, which is reduced for the convenience of display). It can be seen that the negative samples in the recommended dataset are much higher than the positive samples, and the sample size of each sub-dataset also varies according to the clustering characteristics.

[0191] (3) Data transformation: After statistical analysis, the detailed comparison of each sub-data set is shown in the following table. In order to make each indicator at the same order of magnitude, the dense features are normalized and uniformly shrunk to between 0 and 1. This can reduce the impact of some eigenvalues ​​being too large and causing small eigenvalues ​​to be ignored by the model; the sparse features are encoded by Label Encoder in order to map the sparse features to categorical values ​​and reduce the size of the category space.

[0192]

[0193] After the above preprocessing steps, the steps of simulating different clients are basically completed. Finally, the training set, validation set and test set are formed according to the ratio of 9:0.5:0.5 from each sub-dataset.

[0194] 1. Based on the dataset generated above, experiments on the improved horizontal federated recommendation model are conducted.

[0195] The first part is the centralized training of the full dataset (Total Centralized Learning, Total-CL), which simulates the performance baseline of the CTR estimation model of the dataset in this article under normal circumstances (without data privacy protection technology); the second part is the centralized training of the single client dataset (Part Centralized Learning, Part-CL), which simulates the centralized training of a single sub-dataset to compare the improvement rate of model performance after using the federated learning paradigm. The experimental results of the first and second parts are shown in the following table.

[0196]

[0197] From the above table, it can be clearly seen that the auc values ​​of the LR and GBDT models as baseline models, as well as the CTR estimation models such as FM, in the Total_CL mode are higher than those in the Part_CL mode. This is in line with the general situation, because the sample size of the former is larger than that of the latter. In the Total_CL training mode, the best model in the Criteo dataset is FFM, with an auc value of 77.03%.

[0198] The last part is federated training (FL), and the experimental results are shown in the following table.

[0199]

[0200] 3. Based on the dataset generated in 1, conduct experiments on the improved vertical federated recommendation model.

[0201] For the vertically federated CTR recommendation model, the performance of FM and DeepFM improves on the accuracy of the model over the similarly vertically federated LR and GBDT.

[0202] The auc values ​​of the vertical federated CTR recommendation models HVF_FM and HVF_DeepFM are higher than those of partial client training (Part_CL) and slightly lower than those of centralized training of all client datasets (Total_CL), which is consistent with the general characteristics of the federated learning model.

[0203]

[0204] Combined with the above chart analysis, it can be seen that the processing method based on the sample size of the data set and the positive-negative label ratio coefficient λ proposed in the present invention has achieved better model performance effects on the four horizontal federated recommendation models of HF_LR, HF_GBDT, HF_FM and HF_DeepFM and the two vertical federated recommendation models of HVF_FM and HVF_DeepFM, compared with the full (Total_CL) and partial (Part_CL) data sets of centralized training. This proves that the data processing method based on the sample size of the data set and the positive-negative label ratio coefficient λ proposed in the present invention and the improved federated CTR recommendation algorithm based on the federated learning framework can better improve the model training performance.

[0205] The present invention proposes an improved data processing and CTR federated recommendation system based on federated learning

[0206] The present invention implements a system based on the processing method of the recommended data set based on the sample size of the data set and the positive-negative label ratio coefficient λ and the improved federated learning CTR recommendation model mentioned above. According to the general system function requirements, it is divided into three roles: user User, database worker Databse Worker and platform worker Platform Worker; the business process is data collection and collation, user operation, global model training, local model update, advertising recommendation and feedback and update; the database form of the design recommendation system is user table, advertising table and user preference table; the entire federated recommendation system can be designed as a "Client-Sever" architecture, that is, "1 central server + N client servers". The central server includes the platform of the platform worker and the database of the database worker, etc., which is responsible for pre-training and distributing models, storing data, maintaining pages, etc. The client server group is composed of edge devices of many users. The client server is responsible for storing the user's own privacy information, receiving the global model and training the local model, etc. The entire system is implemented using the SSM framework based on MVC.

[0207] The backend of this system is implemented in Java and uses MySQL as the database. The encryption and aggregation of federated learning are implemented based on the FATE framework of Python. The sparse data used by the system mainly comes from three categories such as songs, cars, and movies, and the dense features are composed of some virtual generated user information. This is used as the original data set for initial training of the model and model distribution. After the user logs in to the system on the local client and enters the password and passes the verification, he can log in to the local recommendation page. The user's account and password and other information are stored in the user's personal local client server, and only the encrypted user ID is exposed to the central server. With the user's permission, the central server obtains the user's related behavior based on this ID and generates a recommendation list through the CTR recommendation model. The top 4 items are displayed on the homepage after the user logs in. In addition, the navigation bar on the homepage also provides personal center, viewing behavior history, marking records and message notifications, as well as search functions.

[0208] Users can click on and comment on a recommended ad. After the user clicks, the platform receives the user's click feedback and sends it to the database for model prediction. After the user clicks on a song ad, more song-related ads will be displayed on the homepage.

[0209] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment in combination with software and hardware. Moreover, the present application may adopt the form of a computer program product implemented in one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that include computer-usable program code.

[0210] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0211] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0212] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

Claims

1. An improved data processing and CTR federated recommendation method based on federated learning, characterized by: The following steps are involved: Step 1: A weighted federated recommendation dataset is generated based on the processing strategy of the recommendation dataset based on the dataset sample size and the positive-negative label ratio coefficient λ; Step 2: Based on the weighted federated recommendation dataset obtained in step 1, two types of federated baseline recommendation models, LR and GBDT, and a federated CTR recommendation model are established respectively. The federated CTR recommendation model includes a federated FM model and a federated DeepFM model. Step 3: Based on the data set processing method and federated recommendation model established in steps 1 and 2, obtain the recommendation results.

2. The improved data processing and CTR federated recommendation method based on federated learning according to claim 1, characterized in that: The specific steps of step 1 include: (1) Define federated learning clients and data: The set of k clients is: <h2 style=";text-align:left;direction:ltr">N = {n1,n2,n3,……,n<h2 style=";text-align:left;direction:ltr"> k <h2 style=";text-align:left;direction:ltr">} (1) Where n i represents the i-th client, the original recommendation dataset is D, and according to the definition of federated learning, the dataset D is divided into: <h2 style=";text-align:left;direction:ltr">D = {D1,D2,D3,……,D<h2 style=";text-align:left;direction:ltr"> k <h2 style=";text-align:left;direction:ltr">} (2) Among them, D i is the dataset of the i-th client, where the value range of i is 0 < i ≤ k; the j-th sample data in the dataset D i is as follows: Among them, x i is the i-th feature of the sample, the set of all m samples The feature set X that constitutes client i i ,The generation and processing of the federated recommendation dataset is the process of formula (2); (2) Given the number of clients k participating in federated training, find the sample size M of the original dataset D, then the single client dataset D i The maximum sample size is M / k, and the dataset D is divided into D = {D + ,D - }Two sets of positive and negative samples, let (3) Use a clustering algorithm to cluster the data features, and then take the average extraction of data to form a client or generate a separate data set for each cluster as needed; use K-means clustering, through the K-means function based on sklearn, and use Euclidean distance as the data sample similarity measurement method, and its calculation formula is: Among them, x, y are the sample and the selected cluster center vector respectively, and d(x, y) is the distance from the sample point to the cluster center; (4) Assuming that there is a client a with the smallest data volume P and a client b with the most unbalanced positive and negative sample ratio Q, the improvement is: Assume that among all k clients, a client i with a sample size of P has a positive-negative sample ratio of q, and the client with the positive-negative sample ratio closest to 1 in the client set has a positive-negative sample ratio of g. Then the model training and federation information of client i, such as model parameters, needs to be accompanied by a client weight coefficient λ, which is calculated as follows: Through the above steps, a weighted federated recommendation dataset is obtained.

3. The improved data processing and CTR federated recommendation method based on federated learning according to claim 1, characterized in that: The specific method of step 2 is: (1) Establish a logistic regression model; (2) Establish a horizontal federated logistic regression model; (3) Establish a vertical federated LR model; (4) Establish a horizontal federated GBDT model; (5) Establish a vertical federated GBDT model; (6) Establish a federal FM model; (7) Establish a federal FM model.

4. The improved data processing and CTR federated recommendation method based on federated learning according to claim 3, characterized in that: The specific method of establishing the logistic regression model in step (1) of step 2 is: The value y=f(x) output by the linear regression expression has a range of (-∞,∞), while the predicted value of the binary classification problem for click-through rate recommendation is [0,1]. Therefore, the sigmoid function is selected to map the linear regression output result to the interval [0,1]: y =f(x)=w T x+b (7) The output y of the linear regression model is used as the input of the sigmoid function to form the logistic regression model LR, which is used as a benchmark model for many experiments. The model expression is shown in Formula 9:

5. The improved data processing and CTR federated recommendation method based on federated learning according to claim 3, characterized in that: The specific method of establishing the horizontal federated logistic regression model in step (2) of step 2 is: In each round of iteration, clients A and B train the LR model on their own data, and then encrypt the gradients of their respective trained models and upload them to a third-party assistant C. Assistant C only has the keys of clients A and B. Its task is to aggregate the model gradients uploaded by A and B. Since it is homomorphically encrypted, some mathematical calculations can be performed directly to form a federated model gradient, which is then distributed to clients A and B to update the local model. During this period, strategies such as early termination are set like traditional machine learning models. The training is repeated until the model converges, and finally the LR model effects obtained by clients A and B are consistent.

6. The improved data processing and CTR federated recommendation method based on federated learning according to claim 3, characterized in that: The specific method of establishing the vertical federated LR model in step (3) of step 2 is: Training the longitudinal federated logistic regression model requires splitting the LR loss function. The LR loss function F(w) is shown in Formula 11. Among them, p is the probability of label 1 as follows, where w T is the transposed vector of vector w; The transformed longitudinal federated LR loss function formula 12 is obtained by polynomial approximation expansion: Assuming that client A holds the labels and client B only holds the feature dataset, the vertical federated LR model is trained; At the beginning, clients A and B use a piece of data to perform sample alignment to establish a federated dataset, which is used to identify overlapping samples stored in the databases of the two parties. The federated training model is built based on these overlapping samples. The entire sample comparison process is performed in an encrypted form. Subsequently, clients AB initialize their own model parameters respectively. The labeled A is calculated and encrypted on the same platform to obtain: The unlabeled B is calculated and homomorphically encrypted to obtain: Afterwards, A will [[d A ]] is sent to B, and B sends [[d B ]] is sent to A, then both A and B can calculate formula (12) to obtain the encrypted gradient [[g A ]] and [[g B ]], then A and B send the gradient to the assistant C, who decrypts it to get g A and g B Then it is returned to A and B, and then A and B use the decrypted gradient to update the model parameters.

7. The improved data processing and CTR federated recommendation method based on federated learning according to claim 3, characterized in that: The specific method of establishing the horizontal federated GBDT model in step (4) of step 2 is: For the GBDT model, each sub-client sends an encrypted local histogram to the central server. The histogram contains the sum of the gradient and hessian matrices. Finally, the server obtains the global histogram without directly obtaining any local histograms, finds the best tree segmentation solution and distributes it to each client.

8. The improved data processing and CTR federated recommendation method based on federated learning according to claim 3, characterized in that: The specific method of establishing the vertical federated GBDT model in step (5) of step 2 is: For the vertical federated GBDT model, the common user samples of all clients participating in the training are found through the data privacy protection protocol and aligned under encryption; then the only client with the label uses the XGBoost tree learning algorithm to build a decision tree in each iteration; The client encrypts them before sending them to other clients; after receiving each client, it uses the encrypted gradient and hessian matrix to calculate the encrypted feature histogram, and then encodes the features and splitting point values ​​and builds a lookup table; the encoded features and splitting point values ​​and feature histograms are then sent to the labeled client, which decrypts them after receiving the feature histograms and finds the best gain; if the best gain feature belongs to a client, the encoded features and splitting point values ​​are sent back to the client.

9. The improved data processing and CTR federated recommendation method based on federated learning according to claim 3, characterized in that: The specific method of establishing the federal FM model in step (6) of step 2 is: According to the optimized FM model formula, the gradient vector v of each parameter of the FM model is derived i is feature x i Corresponding to the hidden vector, the length is k (k< <n),<·,·> is the dot product operator of the vector, w0 is the bias term, which represents the basic prediction value when all features are zero, and w i is the first-order weight, corresponding to feature x i The linear effect of The clients can complete the constant term and linear term independently, but the cross term requires the clients to confirm the size of the overlapping feature space. The assistant uses this as one of the model initialization parameters and distributes it to each client to initialize the local model.

10. The improved data processing and CTR federated recommendation method based on federated learning according to claim 3, characterized in that: The specific method of establishing the federated DeepFM model in step (7) of step 2 is as follows: The DeepFM model consists of the FM component on the left and the DNN component on the right. The high-dimensional sparse feature space serves as the common input of the FM and Deep layers. The low-order features of the input data are completed by FM. The FM part is responsible for extracting only the second order; the DNN part is used to extract high-order features. In addition, the FM and Deep parts use input and feature embedding together, and the DNN part is a simple multi-layer fully connected neural network; a (l+1) is the activation value (or output) of the l+1l+1th layer; in a neural network, the activation value is the result of applying an activation function to the input; σ is the activation function, which is usually used to introduce nonlinearity to enable the network to learn more complex patterns; W(l) is the weight matrix of layer l; in a fully connected layer, it is a two-dimensional matrix whose dimension depends on the number of neurons in the previous layer and the number of neurons in the current layer; a (l) is the activation value of the lth layer, which serves as the input of the current layer (the l+1th layer); b (l) is the bias vector of layer l; together with the weight matrix, it helps the model fit the bias in the data; The output of each layer is a (l+1) =σ(W (l) a (l) +b (l) ) (17) The output of the FM part is as follows: (1) an AdditionUnit; (2) Multiple inner product units; The prediction formula of DeepFM is as follows Similarly, by inputting FM, DeepFM and other models into the Model part of the homoNN component, the horizontal federation model construction of the CTR estimation model can be completed.

Images