A neural network collaborative filtering recommendation method based on federated learning

Abstract

A neural network collaborative filtering recommendation method based on federated learning does not need to centrally collect users' personal data and upload it to a trusted central server for storage and model training. It only performs data training locally, and then optimizes multiple local models in coordination with the central server, and then aggregates the intermediate parameters of multiple local models to obtain a globally optimal model on the server side, and sends it to each terminal. In the local participant training, this method is different from traditional methods based on clustering, nearest neighbor, and matrix decomposition. It converts the data set into an implicit feedback data set at a certain ratio, constructs a neural collaborative filtering general framework NCF, inputs one-hot encoded user and item vectors, outputs prediction results, and performs differential privacy noise on the uploaded updated weights. This method effectively improves the recommendation effect while ensuring the privacy security of the data model.

Description

Technical Field

[0001] The present invention relates to the technical field of federated learning, and in particular to a neural network collaborative filtering recommendation method based on federated learning. Background Art

[0002] Recommender systems have been widely used to create personalized prediction models to help individuals identify content of interest. Such systems collect various characteristics of individuals, such as demographic characteristics, item-related rating data (explicit feedback), or user interaction data with specific items (implicit feedback). Their goal is to provide users with future preference recommendations based on past interaction data. Recommender systems are currently widely used in various fields such as e-commerce and online streaming services. In traditional recommendation systems, user preferences and learned feature vectors can reveal sensitive information. In addition, complex techniques can lead to the de-anonymization of datasets. Therefore, methods are needed to build powerful recommendation systems without leaking users' private data.

[0003] With the increasing number of computing devices in the cloud, such as mobile phones and wearable devices, the use of machine learning is becoming more and more widespread. However, traditional machine learning methods require data transmission from devices that store a lot of private information due to sensors and rich user interactions. In order to overcome the challenges of storing such information in central servers and enhance user privacy, Google proposed a new technology in 2016, named Federated Learning (FL), in which the federated setting allows machine learning to be facilitated without the need to transmit user data, thus having great advantages for both privacy and fast distributed computing.

[0004] Federated learning (FL) is a machine learning setting that allows models to be trained in a distributed environment. The main idea behind this learning technique is that different entities can jointly train models under the coordination of a central server without sharing their data. Unlike traditional machine learning, which requires users to transfer their data, FL supports a higher level of privacy because the model is trained locally on each device, and after the client operates, only the parameters are sent back to the central server without uploading its training data. Summary of the invention

[0005] The present invention aims to solve the privacy leakage problem in the recommendation system without affecting the recommendation effect, and provides a neural network collaborative filtering recommendation method based on federated learning.

[0006] A neural network collaborative filtering recommendation method based on federated learning includes the following steps:

[0007] Step 1: The central server initializes the training configuration of the neural network model and starts the training plan, and sends it to the participants P1, P2, ..., PN ; The participant uses the global parameter w0 in the received training configuration as the 0th round model parameter of the local neural network collaborative filtering model;

[0008] Step 2: The central server randomly selects C participants as training participants to participate in the tth round of training and sends them the current global model;

[0009] Step 3: Each participant in the tth round of training uses the local training dataset D c And local steps localepochs to update the t-1th round global model parameters of the existing local neural network collaborative filtering model Get the model parameters of each participant in the tth round of the local neural network collaborative filtering model c∈C, C is the number of training participants, t is the number of training rounds;

[0010] Step 4: Each participant in the tth round of training updates the model parameters after the local tth round of training. Upload to the server and perform differential privacy processing by adding Gaussian noise;

[0011] Step 5: The server uses the FedAvg algorithm to calculate the model parameters uploaded by each participant in step 4. Aggregate and get the new global parameter w t ;

[0012] Step 6: The server will generate a new global parameter w t , determine whether the local neural network collaborative filtering models of all participants have converged or reached the predetermined number of training times: if yes, go to step 7; otherwise, increase the number of training rounds t by 1 and return to step 2;

[0013] In step 7, each participant uses the locally trained neural network collaborative filtering model and the hit rate @10 evaluation protocol to predict the local test data and generate a list of the top 10 recommended items, thereby evaluating the application of the trained model in the recommendation system and outputting the recommendation results.

[0014] Furthermore, the specific content of the training configuration sent by the server in step 1 includes: a description of the task, notifying the participants of the hyperparameters of the model, the hyperparameters containing relevant information related to the training process; the global parameter w0 of the current model; and the constant C, which represents the number of local participants before aggregation.

[0015] Furthermore, in step 3, the data set is first converted into an implicit feedback data set, and the neural collaborative filtering (NCF) framework is used for training. For the convenience of representation, in the defined data set, A and B represent the number of users u and items i, respectively, and K represents the dimension of the latent space; the framework includes a data input layer, an embedding layer, a neural collaborative filtering layer, and an output layer; the input data is converted into a binary sparse vector as an input feature using one-hot encoding, and then the sparse representation of the input layer is mapped into a dense vector through the embedding layer, and then sent to the neural collaborative filtering layer to map the latent vector into a predicted score; the final output layer is the predicted score Training is performed by minimizing cross entropy; the prediction formula is:

[0016]

[0017] where P∈R A×K , Q∈R B×K , respectively represent the latent factor matrices of users and items; the eigenvector and They are used to describe user u and item i, Θ f represents the model parameters of the interaction function f, which is defined as a multi-layer neural network.

[0018] Furthermore, in step 4, each participant in the tth round of training updates the model parameters after the local tth round of training Before uploading to the server, differential privacy processing is performed on it and Gaussian noise is added to protect the privacy security of the model. The Gaussian mechanism satisfies (∈, δ)-differential privacy. According to the Gaussian mechanism, for the function f(x) that returns a number, the following F(x) definition satisfies (∈, δ)-differential privacy:

[0019] F(x)=f(x)+N(σ 2 )

[0020]

[0021] where s is the sensitivity of f, and N(σ 2 ) means that the center is 0 and the variance is σ 2 Sampling from a Gaussian distribution.

[0022] Furthermore, in step 5, the specific formula of the FedAvg algorithm is as follows:

[0023]

[0024] Where c∈C, C is the total number of participants selected in this round of training, and n c is the number of local training instances of the participant, is the total number of instances in this round of training, are the model parameters updated by the participants.

[0025] Furthermore, in step 6, the convergence of the local neural network collaborative model of the participant means that the loss function value of the local neural network collaborative filtering model no longer changes or the change amount is less than the set change threshold.

[0026] Furthermore, the calculation formula of hit rate @10 in step 7 is:

[0027]

[0028] The denominator GT is the total number of test sets, and the numerator represents the sum of the number of test sets in the top ten recommendation lists of each participant. The higher the HR@10 value, the better the recommendation system is.

[0029] Compared with the prior art, the present invention has the following beneficial effects:

[0030] 1. This method is based on the recommendation system of deep learning and can achieve better recommendation results;

[0031] 2. There are data privacy and security issues in traditional recommendation systems. Federated learning can protect data from interacting with the central server and only train it locally. This method also performs differential privacy noise processing on the updated model parameters, further protecting the data security of the recommendation system. BRIEF DESCRIPTION OF THE DRAWINGS

[0032] Figure 1 The present invention provides a system principle and a neural network collaborative filtering model framework for a federated learning-based neural network collaborative filtering recommendation method. DETAILED DESCRIPTION

[0033] The technical solution of the present invention is further described in detail below in conjunction with the accompanying drawings.

[0034] In the preparation of the federated learning recommendation system, it is necessary to coordinate the central server and several participating clients to build the entire federated learning training environment. The central server needs to prepare the neural network training model for the first time, and the participating clients need to prepare the corresponding data set.

[0035] Taking the movie recommendation of the dataset MovieLens as an example, the participants can be IoT devices or servers with x86 or arm architecture. This dataset provides information between users, movies, and ratings, which is explicit feedback. However, explicit feedback has fewer features, and implicit feedback is generally constructed to enrich data features and improve the overall training effect. To this end, the dataset is first divided into training sets and test sets in proportion, the ratings in the dataset are binarized, the ratings of users and movies participating in the ratings are set to "1", and negative samples (assuming that the user has no interaction with the movie) are constructed in a certain proportion. The dataset is the implicit feedback dataset. By constructing the neural collaborative filtering general framework NCF, inputting one-hot encoded user and item vectors, the original binary sparse vector is input into the embedding layer to obtain a smaller and denser vector (dense vector), which is connected through a series of fully connected layers. These layers map the connected embeddings to a prediction vector as output. The neural collaborative filtering NCF general framework can simulate the interaction between users and items in different linear modeling methods such as GMF, nonlinear modeling methods such as MLP, and fusion enhancement modeling such as NeuMF. And for the final output layer, the strategy of minimizing the difference between the predicted value and the true value during training is point-by-point loss instead of pair-wise loss. In the output layer, a Sigmoid function is applied to obtain the most likely class, and finally the weight of the uploaded update is noised with differential privacy. In the process of updating the model by federated learning, we use the classic FedAvg algorithm to optimize the model parameters on average. The effect of the recommendation is evaluated by the hit rate @10 (HitRatio@10).

[0036] See also Figure 1 , a neural network collaborative filtering recommendation system and method based on federated learning, comprising the following steps:

[0037] Step 1: The central server initializes the training configuration of the neural network model and starts the training plan, and sends it to the participants P1, P2, ..., P N ; The participants will receive the global parameters w0 (including model weights) in the training configuration The project configuration weights I0, etc.) are used as the 0th round model parameters of the local neural network collaborative filtering model.

[0038] Step 2: The central server randomly selects C participants as training participants to participate in the tth round of training and sends them the current global model.

[0039] In the movie recommendation example of the MovieLens dataset, the original model parameter w0 of round 0 is broadcast to all C IoT devices (servers) participating in the federated training.

[0040] Step 3: Each participant in the tth round of training uses the local training dataset Dc And some local steps, generally called local epochs, to update the t-1th round global model parameters of the existing local neural network collaborative filtering model Get the model parameters of each participant in the tth round of the local neural network collaborative filtering model (including model weights Project configuration weight wait).

[0041] Step 4: Each participant in the tth round of training updates the model parameters after the local tth round of training Upload to the server.

[0042] Step 5: The server uses the FedAvg algorithm to calculate the model parameters uploaded by each participant in step 4. Aggregate and get the new global parameter w t .

[0043] Step 6: The server will generate a new global parameter w t , determine whether the local neural network collaborative filtering models of all participants have converged or reached the predetermined number of training times: if so, go to step 7; otherwise, add 1 to the number of training rounds t and return to step 2.

[0044] Step 7: Each participant uses the locally trained neural network collaborative filtering model and the HitRatio@10 evaluation protocol to predict the local test data and generate a list of the top 10 recommended items, thereby evaluating the application of the trained model in the recommendation system.

[0045] In the above, c∈C, C is the number of training participants; t is the number of training rounds.

[0046] The specific contents of the training configuration sent by the server in step 1 above are (1) a description of the task, notifying the participants of the hyperparameters of the model, which contain relevant information related to the training process, such as the local mini-batch size B; (2) the global parameter w0 of the current model; and (3) the constant C, which represents the number of local participants before aggregation.

[0047] In step 3 above, the data set is first converted into an implicit feedback data set, and the neural collaborative filtering NCF framework is used for training. For the convenience of representation, in the data set, A and B represent the number of users u and items i, respectively, and K represents the dimension of the latent space. The NCF framework includes a data input layer, an embedding layer, a neural collaborative filtering layer, and an output layer connected in sequence. In the data input layer, user u and item i are converted into binary sparse vectors as input features using one-hot encoding, and then the sparse representation of the input layer is mapped to a dense vector through the embedding layer (EmbeddingLayer). The user embedding and item embedding are then sent to a multi-layer neural network structure called the neural collaborative filtering layer to map the latent vector to a prediction score. The final output layer is the prediction score. The training is performed by minimizing the cross entropy. The prediction formula is:

[0048]

[0049] where P∈R A×K , Q∈R B×K , respectively represent the latent factor matrices of users and items; the eigenvector and They are used to describe user u and item i, Θ f represents the model parameters of the interaction function f, which is defined as a multi-layer neural network.

[0050] Each participant in the t-th round of training in step 4 above updates the model parameters after the t-th round of training locally Before uploading to the server, differential privacy processing is performed on it and Gaussian noise is added to protect the privacy security of the model. The Gaussian mechanism satisfies (∈,δ)-differential privacy. According to the Gaussian mechanism, for the function f(x) that returns a number, the following F(x) definition satisfies (∈,δ)-differential privacy:

[0051] F(x)=f(x)+N(σ 2 )

[0052]

[0053] where s is the sensitivity of f, and N(σ 2 ) means that the center is 0 and the variance is σ 2 Gaussian (normal) distribution sampling.

[0054] The specific formula of the FedAvg algorithm in step 5 above is as follows:

[0055]

[0056] Where c∈C, C is the total number of participants selected in this round of training, and n c is the number of local training instances of the participant, is the total number of instances in this round of training, are the model parameters updated by the participants.

[0057] The convergence of the local neural network collaborative model of the participants in the above step 6 means that the loss function value of the local neural network collaborative filtering model no longer changes or the change amount is less than the set change threshold.

[0058] The formula for calculating the hit rate @10 (HitRatio@10) in step 7 above is:

[0059]

[0060] The denominator GT is the total number of test sets, and the numerator represents the total number of test sets in the top ten recommendation lists of each participant. The higher the HR@10 value, the better the recommendation system is.

[0061] In the movie recommendation example of the MovieLens dataset, each training participant obtains a trained movie recommendation neural network model, takes the user and item vectors as the input values ​​of the movie recommendation neural network model, and the output value of the movie recommendation neural network model is to predict whether the effect of the movie recommended to the user participates in the interaction.

[0062] The above description is only a preferred embodiment of the present invention, and the protection scope of the present invention is not limited to the above embodiment. Any equivalent modifications or changes made by ordinary technicians in this field based on the contents disclosed by the present invention should be included in the protection scope recorded in the claims.

Claims

1. A neural network collaborative filtering recommendation method based on federated learning, characterized by: The following steps are involved: Step 1: The central server initializes the training configuration of the neural network model and starts the training plan, and sends it to the participants P1, P2, ..., P N ; The participant uses the global parameter w0 in the received training configuration as the 0th round model parameter of the local neural network collaborative filtering model; Step 2: The central server randomly selects C participants as training participants to participate in the tth round of training and sends them the current global model; Step 3: Each participant in the tth round of training uses the local training dataset D c And local steps local epochs to update the t-1th round global model parameters of the existing local neural network collaborative filtering model Get the model parameters of each participant in the tth round of the local neural network collaborative filtering model c∈C, C is the number of training participants, t is the number of training rounds; Step 4: Each participant in the tth round of training updates the model parameters after the local tth round of training. Upload to the server and perform differential privacy processing by adding Gaussian noise; Step 5: The server uses the FedAvg algorithm to calculate the model parameters uploaded by each participant in step 4. Aggregate and get the new global parameter w t ; In step 5, the specific formula of the FedAvg algorithm is as follows: Where c∈C, C is the total number of participants selected in this round of training, and n c is the number of local training instances of the participant, is the total number of instances in this round of training, are the model parameters updated by the participants; Step 6: The server will generate a new global parameter w t , determine whether the local neural network collaborative filtering models of all participants have converged or reached the predetermined number of training times: if yes, go to step 7; otherwise, increase the number of training rounds t by 1 and return to step 2; In step 7, each participant uses the locally trained neural network collaborative filtering model and the hit rate @10 evaluation protocol to predict the local test data and generate a list of the top 10 recommended items, thereby evaluating the application of the trained model in the recommendation system and outputting the recommendation results.

2. The neural network collaborative filtering recommendation method based on federated learning according to claim 1, characterized in that: The specific content of the training configuration sent by the server in step 1 includes: a description of the task, notifying the participants of the model's hyperparameters, which contain relevant information related to the training process; the global parameter w0 of the current model; and the constant C, which represents the number of local participants before aggregation.

3. The neural network collaborative filtering recommendation method based on federated learning according to claim 1, characterized in that: In step 3, the data set is first converted into an implicit feedback data set, and the neural collaborative filtering (NCF) framework is used for training. For the convenience of representation, in the defined data set, A and B represent the number of users u and items i, respectively, and K represents the dimension of the latent space. The framework includes a data input layer, an embedding layer, a neural collaborative filtering layer, and an output layer. The input data is converted into a binary sparse vector as an input feature using one-hot encoding, and then the sparse representation of the input layer is mapped into a dense vector through the embedding layer, and then sent to the neural collaborative filtering layer to map the latent vector into a predicted score. The final output layer is the predicted score. Training is performed by minimizing the cross entropy; the prediction formula is: where P∈R A×K Q∈R B×K Represents the latent factor matrix of users and items respectively; the eigenvector and They are used to describe user u and item i, Θ f represents the model parameters of the interaction function f, which is defined as a multi-layer neural network.

4. The neural network collaborative filtering recommendation method based on federated learning according to claim 1, characterized in that: In step 4, each participant in the tth round of training updates the model parameters after the local tth round of training Before uploading to the server, differential privacy processing is performed on it and Gaussian noise is added to protect the privacy security of the model. The Gaussian mechanism satisfies (∈, δ)-differential privacy. According to the Gaussian mechanism, for the function f(x) that returns a number, the following F(x) definition satisfies (∈, δ)-differential privacy: F(x)=f(x)+N(σ 2 ) where s is the sensitivity of f, and N(σ 2 ) means that the center is 0 and the variance is σ 2 Sampling from a Gaussian distribution.

5. The neural network collaborative filtering recommendation method based on federated learning according to claim 1, characterized in that: In step 6, the convergence of the local neural network collaborative model of the participant means that the loss function value of the local neural network collaborative filtering model no longer changes or the change amount is less than the set change threshold.

6. The neural network collaborative filtering recommendation method based on federated learning according to claim 1, characterized in that: The calculation formula for hit rate @10 in step 7 is: The denominator GT is the total number of test sets, and the numerator represents the sum of the number of test sets in the top ten recommendation lists of each participant. The higher the HR@10 value, the better the recommendation system is.

Images