Federal collaborative filtering recommendation method fusing explicit and implicit feedback
By introducing Paillier semi-homomorphic encryption and non-sampling matrix decomposition models into the federal collaborative filtering recommendation system, combined with the stochastic gradient descent algorithm, the problems of privacy leakage and low recommendation accuracy are solved, and efficient privacy protection and recommendation performance improvement are achieved.
Patent Information
- Application Number
- CN202411158306.X
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-08-22
- Publication Date
- 2024-12-10
AI Technical Summary
The existing recommendation systems have the risk of privacy leakage in federated collaborative filtering that integrates explicit and implicit feedback, and the recommendation accuracy of a single feedback form is not high enough to effectively utilize the user's explicit and implicit feedback data.
Paillier semi-homomorphic encryption technology combined with non-sampling matrix decomposition model and stochastic gradient descent algorithm to construct a federal collaborative filtering recommendation method that integrates explicit and implicit feedback, separates user privacy weight parameters, optimizes implicit and explicit feedback model parameters, and in Encoding gradients in a federated learning framework.
It improves the accuracy and privacy protection capabilities of the recommendation system, ensures the security of user privacy, and avoids gradient leakage and improves recommendation performance.
Smart Images
Figure CN119106201A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of federated learning technology, and in particular to a federated collaborative filtering recommendation method integrating explicit and implicit feedback. Background Art
[0002] Recommendation systems mainly make recommendations by analyzing users' historical behavior data, where users' historical behavior data is divided into explicit feedback data and implicit feedback data. Common recommendation systems only use explicit feedback data or implicit feedback data to complete recommendations. However, this recommendation method that only uses a single feedback form is often not accurate enough. In recent years, some researchers have gradually realized that there is a certain complementary relationship between explicit feedback data and implicit feedback data, so they try to combine these two data types for in-depth research. In the federated collaborative filtering recommendation method that integrates semi-homomorphic encryption, the FedNSMF method only uses implicit feedback data to implement federated collaborative filtering recommendations. Moreover, most of the existing recommendation algorithms that integrate explicit and implicit feedback have the risk of privacy leakage. Cross-user federated recommendation stores the original user data on the local device and only trains the model by uploading some parameters, which alleviates the problem of privacy protection to a certain extent. However, with the in-depth study of federated learning, researchers have found that while federated learning trains the model by uploading gradients, the gradients will also leak the user's privacy.
[0003] Therefore, based on the above technical defects, the present invention proposes a federated collaborative filtering recommendation method that integrates explicit and implicit feedback, aiming to establish a recommendation system with explicit and implicit feedback in a cross-user federated scenario, and adopts Paillier semi-homomorphic encryption technology to enhance the privacy protection capability. Summary of the invention
[0004] The purpose of the present invention is to provide a federated collaborative filtering recommendation method that integrates explicit and implicit feedback, which utilizes the advantage that explicit feedback data can clearly reflect the user's preference degree, and at the same time introduces implicit feedback data to make up for the disadvantage that the amount of user explicit feedback data is low due to users' unwillingness to rate, thereby improving the recommendation accuracy and enhancing privacy protection.
[0005] To achieve the above object, the present invention provides a federated collaborative filtering recommendation method integrating explicit and implicit feedback, comprising the following steps:
[0006] S1. Construct a non-sampling matrix factorization model to decompose the implicit feedback matrix and separate the weight parameters involving user privacy;
[0007] S2. On the basis of building an implicit feedback model, ALS is used to solve the implicit user vector and implicit item vector to optimize the parameters of the implicit feedback model;
[0008] S3, introduce baseline evaluation, build a matrix decomposition model based on explicit feedback data, and generate an explicit matrix decomposition model that integrates implicit feedback data;
[0009] S4, using SGD to perform matrix decomposition, using explicit feedback data and implicit feedback data for Top-N recommendation, and optimizing explicit and implicit feedback model parameters;
[0010] S5. Build a general federated learning framework to train models using explicit feedback data and implicit feedback data;
[0011] S6. Integrate Paillier semi-homomorphic encryption into the federated learning framework to encode gradients.
[0012] S7. Analyze the safety and recommended performance of the FedEIFCF method.
[0013] Preferably, in step S1, a non-sampling matrix decomposition model is constructed to decompose the implicit feedback matrix and separate the weight parameters related to user privacy, including the following steps:
[0014] S11. Use matrix decomposition method to decompose the user-item interaction matrix, and each entry z ui The estimate of ∈Z is as follows:
[0015]
[0016] in, represents the implicit user vector of user u, The implicit item vector representing item i;
[0017] S12. Adopt the NSMF general loss function with implicit feedback to separate the weight parameters related to user privacy, as shown below:
[0018]
[0019] Among them, u represents the user, i represents the project, represents the observed (u,i) pair; U represents the entire user set, I represents the entire item set, ω ui represents the weight of the observed (u,i) pair; λ is a regularization parameter, α u is the user-oriented weight of user u in the missing data, β i is the item-oriented weight for item i in the missing data.
[0020] Preferably, in step S2, on the basis of constructing the implicit feedback model, ALS is used to solve the implicit user vector and the implicit item vector, and the implicit feedback model parameters are optimized. The specific process is as follows:
[0021] S21. For each user u, there is a gradient X u· , as shown below:
[0022]
[0023] Among them, I u represents the item interacted by user u;
[0024] S22, using cache strategy to optimize computational complexity, let The update rule for each user u based on ALS is as follows:
[0025]
[0026] in, D is a d×d identity matrix; cache S Y It is independent of u and can speed up the calculation of all subsequent user potential factors by pre-calculating them;
[0027] S23, using the cache strategy, for each item i, there is a gradient Y i· , as shown below:
[0028]
[0029] Among them, cache
[0030] S24, Order Based on ALS Y i· Update the rules as follows:
[0031]
[0032] Among them, cache S X Regardless of i, by pre-calculating S X To speed up the calculation of potential factors for all subsequent projects.
[0033] Preferably, in step S3, a baseline evaluation is introduced, a matrix decomposition model based on explicit feedback data is constructed, and an explicit matrix decomposition model integrating implicit feedback data is generated. The specific process is as follows:
[0034] S31, using matrix decomposition method, the actual user-item rating matrix R mn Breaking it down, it looks like this:
[0035]
[0036] For all elements r in R ui ≠0, its objective function is as follows:
[0037]
[0038] S32: Introduce the rating baseline estimation. The rating baseline of user u for item i is estimated using b ui Represented as follows:
[0039] b ui =μ+b u +b i (9)
[0040] Among them, μ represents the mean score of all items, b u represents the bias of user u's rating, b i represents the bias of the score of item i;
[0041] S33. Prediction of user preference is as follows:
[0042]
[0043] S34, embed the implicit user potential feature vector X and the implicit item potential feature vector Y trained by the implicit feedback model into the explicit feedback model, and supplement the explicit user vector U and the explicit item vector V, as shown below:
[0044]
[0045] S35. Substitute formula (11) into formula (8), and the final objective function form is as follows:
[0046]
[0047] The explicit user vector U and the explicit item vector V are obtained by training the explicit feedback model, and Top-N recommendations are performed using explicit feedback data and implicit feedback data.
[0048] Preferably, in step S4, on the basis of constructing an explicit matrix decomposition model integrating implicit feedback data, SGD is used to perform matrix decomposition, explicit feedback data and implicit feedback data are used to perform Top-N recommendation, and the parameters of the explicit and implicit feedback models are optimized. The specific process is as follows:
[0049] S41, using the cache strategy, for each user u, there is a gradient U u· , as shown below:
[0050]
[0051] in, To predict deviation, cache
[0052] S42, SGD-based U u· Update the rules as follows:
[0053]
[0054] S43, using the adaptive moment estimation method, the gradient descent process is carried out by two independent parts, recording the past gradient m u and squared gradient v u The exponentially decaying average of is as follows:
[0055]
[0056] Among them, 0<μ1,μ2<1,m u and v u Initialized to 0;
[0057] S44, in order to offset the deviation, m u and v u The corrected version is as follows:
[0058]
[0059] S45, U u· Update the rules as follows:
[0060]
[0061] Among them, 0<γ u <1.0 is a constant learning rate, 0<ε u <<1;
[0062] S46, using the cache strategy, for each item i, there is a gradient V i· , as shown below:
[0063]
[0064] Among them, cache
[0065] S47, using the adaptive moment estimation method, obtain V i· The update rules are as follows:
[0066]
[0067] in, is the exponentially decaying average of the corrected past gradients, is the exponentially decaying average of the modified square gradient, 0<γ i <1.0 is a constant learning rate, 0<ε i <<1;
[0068] S48, bias b for user u's ratingu , with gradient b u , as shown below:
[0069]
[0070] Adopting the adaptive moment estimation method, we can get b u The update rules are as follows:
[0071]
[0072] in, is the exponentially decaying average of the corrected past gradients, is the exponentially decaying average of the modified square gradient, 0<γ bu <1.0 is a constant learning rate, 0<ε bu <<1;
[0073] S49, bias b for scoring item i i , with gradient b i , as shown below:
[0074]
[0075] Adopting the adaptive moment estimation method, we can get b i The update rules are as follows:
[0076]
[0077] in, is the exponentially decaying average of the corrected past gradients, is the exponentially decaying average of the modified square gradient, 0<γ bi <1.0 is a constant learning rate, 0<ε bi <<1.
[0078] Preferably, in step S5, a federated collaborative filtering recommendation method is used to decompose the iterative update process of model parameters into two parts, which are executed on the client and server respectively. The server provides the latest project profile matrix V for all users to download. Each user u has the latest downloaded V, its own implicit factors, explicit factors and weight parameters, and then performs local updates and calculates the Gradient i , and then Gradient i Sent to the server to update the project configuration file.
[0079] Preferably, in step S5, Paillier semi-homomorphic encryption is integrated into the federated learning framework so that it encodes the gradient. The specific process is as follows:
[0080] First, in the key generation phase, semi-homomorphic encryption operations are used to generate public and private keys. The key generation process is only performed on one of the users, and all participants, including the server, know the public key. The private key is only shared between users and is not protected by the server. After the key is generated, different TLS / SSL secure channels are established to send the public and private keys to the corresponding participants.
[0081] Then, before starting the matrix decomposition, the parameters are initialized. The explicit project profile matrix is initialized on the server, and the explicit user profile matrix, implicit project profile matrix, and explicit user project profile matrix parameters are initialized by each user on the local client;
[0082] Finally, matrix decomposition is performed.
[0083] Preferably, the specific process of matrix decomposition is as follows:
[0084] (1) The server uses the public key to encrypt the explicit project configuration file V and obtains the ciphertext C v From this moment on, the server is ready for the latest C v Available for download by all users;
[0085] (2) Each user downloads the latest C v , and decrypt it with the private key to obtain the plaintext V of the explicit project configuration file; calculate and update the user configuration file U and gradient G on the local client to complete the following operations;
[0086] First, calculate and update the implicit user vector X u· and the implicit item vector Y i· , the implicit user vector X u· As the explicit user vector U u· , the implicit item vector Y i· As an explicit item vector V i· Then, the user's rating bias b is calculated and updated locally u and the rating bias b of the item i , used to correct the rating deviation of users and items; finally, perform local update with V, locally calculate and update the explicit user profile U and gradient G; then encrypt G with the public key to obtain the ciphertext C G ; After getting the ciphertext C G After that, a TLS / SSL secure channel is established, and C G Send to the server;
[0087] (3) After receiving the encrypted gradient C from the user, the server G Finally, update the project configuration file with this ciphertext: Server update latest C v Downloadable for all users;
[0088] (4) Iterate steps (2) and (3) until a certain number of iterations is reached.
[0089] Therefore, the present invention adopts the above-mentioned federated collaborative filtering recommendation method that integrates explicit and implicit feedback, and uses the advantage that explicit feedback data can clearly reflect the user's preference level. At the same time, implicit feedback data is introduced to make up for the disadvantage that the amount of user explicit feedback data is low due to users' unwillingness to rate, thereby improving the recommendation accuracy and enhancing privacy protection.
[0090] The technical solution of the present invention is further described in detail below through the accompanying drawings and embodiments. BRIEF DESCRIPTION OF THE DRAWINGS
[0091] Figure 1 It is a flow chart of a federated collaborative filtering recommendation method integrating explicit and implicit feedback of the present invention;
[0092] Figure 2 The present invention is a federated framework based on Paillier semi-homomorphic encryption;
[0093] Figure 3 is the influence of the potential dimension in the embodiment of the present invention; wherein (a) is the influence of the potential dimension of the accuracy assessment; (b) is the influence of the potential dimension of the normalized discounted cumulative return assessment;
[0094] Figure 4 2 is a comparison of the convergence performance in the embodiments of the present invention; wherein, (a) is the convergence performance of the accuracy evaluation algorithm; and (b) is the convergence performance of the normalized discounted cumulative return evaluation algorithm. DETAILED DESCRIPTION
[0095] The technical solution of the present invention is further described below through the accompanying drawings and embodiments.
[0096] like Figure 1 As shown, a federated collaborative filtering recommendation method integrating explicit and implicit feedback includes the following steps:
[0097] S1. Construct a non-sampling matrix factorization model to decompose the implicit feedback matrix and separate the weight parameters involving user privacy.
[0098] S11. Use matrix decomposition method to decompose the user-item interaction matrix, and each entry z ui The estimate of ∈Z is as follows:
[0099]
[0100] in, represents the implicit user vector of user u, Implicit item vector representing item i.
[0101] S12. Adopt the NSMF general loss function with implicit feedback to separate the weight parameters related to user privacy, as shown below:
[0102]
[0103] Among them, u represents the user, i represents the project, represents the observed (u,i) pair; U represents the entire user set, I represents the entire item set, ω ui represents the weight of the observed (u,i) pair; λ is a regularization parameter, α u is the user-oriented weight of user u in the missing data, β i is the item-oriented weight for item i in the missing data.
[0104] In implicit feedback modeling, z ui = 1 for observed entries, z ui = 0 indicates missing items. In the NSMF model, the strategy of assigning uniform weights to missing items is adopted, and all missing items are assigned the same weight, that is, setting ω for the model ui =ω0 and α u =β i = 1. In the recommendation task, this strategy has better performance than applying a uniform weight strategy to all items.
[0105] S2. Based on the construction of the implicit feedback model, ALS is used to solve the implicit user vector and implicit item vector to optimize the parameters of the implicit feedback model.
[0106] S21. For each user u, there is a gradient X u· , as shown below:
[0107]
[0108] Among them, I u Represents the item interacted by user u.
[0109] S22, using cache strategy to optimize computational complexity, let The update rule for each user u based on ALS is as follows:
[0110]
[0111] in, D is a d×d identity matrix; cache S Y It is independent of u and can be used to speed up the calculation of all subsequent user latent factors by pre-calculating them.
[0112] S23, using the cache strategy, for each item i, there is a gradient Y i· , as shown below:
[0113]
[0114] Among them, cache
[0115] S24, Order Based on ALS Y i· Update the rules as follows:
[0116]
[0117] Among them, cache S X Regardless of i, the cache strategy is to pre-compute S X To speed up the calculation of potential factors for all subsequent projects.
[0118] S3. Introduce baseline evaluation, build a matrix decomposition model based on explicit feedback data, and embed the implicit user and item vectors trained by the implicit feedback model into the explicit feedback model to supplement the explicit user and item vectors and generate an explicit matrix decomposition model that integrates implicit feedback data.
[0119] S31, using matrix decomposition method, the actual user-item rating matrix R mn Breaking it down, it looks like this:
[0120]
[0121] For all elements r in R ui ≠0, its objective function is as follows:
[0122]
[0123] S32, introduce the scoring baseline estimation to eliminate the differences in users' scoring of items. The scoring baseline of user u for item i is estimated by b ui Represented as follows:
[0124] b ui =μ+b u +b i (9)
[0125] Among them, μ represents the mean score of all items, b u represents the bias of user u's rating, b i represents the bias of the score of item i.
[0126] S33. Prediction of user preference is as follows:
[0127]
[0128] S34, embed the implicit user potential feature vector X and the implicit item potential feature vector Y trained by the implicit feedback model into the explicit feedback model, and supplement the explicit user vector U and the explicit item vector V, as shown below:
[0129]
[0130] S35. Substitute formula (4.11) into formula (4.8), and the final objective function form is as follows:
[0131]
[0132] The explicit user vector U and the explicit item vector V are obtained by training the explicit feedback model, and Top-N recommendations are performed using explicit feedback data and implicit feedback data. The new explicit feedback model not only takes advantage of the explicit feedback data that can clearly reflect the user's preference, but also effectively compensates for the lack of explicit feedback data due to users' unwillingness to rate by introducing implicit feedback data.
[0133] S4. On the basis of constructing an explicit matrix decomposition model that integrates implicit feedback data, SGD is used for matrix decomposition, and Top-N recommendation is performed using explicit feedback data and implicit feedback data to optimize the parameters of the explicit and implicit feedback models.
[0134] S41, using the cache strategy, for each user u, there is a gradient U u· , as shown below:
[0135]
[0136] in, To predict deviation, cache
[0137] S42, SGD-based U u· Update the rules as follows:
[0138]
[0139] S43, update U using gradient descent method u· When U u Therefore, the adaptive moment estimation (Adam) method is used, and the gradient descent process is carried out in two independent parts, recording the past gradient m u and squared gradient v u The exponentially decaying average of is as follows:
[0140]
[0141]
[0142] Among them, 0<μ1,μ2<1,m u and v u Initialized to 0, so both values are biased towards 0.
[0143] S44, in order to offset the deviation, m u and v u The corrected version is as follows:
[0144]
[0145] S45, U u Update the rules as follows:
[0146]
[0147] Among them, 0<γ u <1.0 is a constant learning rate, 0<ε u <<1, e.g. 10 -8 , to avoid division by 0.
[0148] S46, using the cache strategy, for each item i, there is a gradient V i· , as shown below:
[0149]
[0150] Among them, cache
[0151] S47, using the adaptive moment estimation (Adam) method, obtain V i· The update rules are as follows:
[0152]
[0153] in, is the exponentially decaying average of the corrected past gradients, is the exponentially decaying average of the modified square gradient, 0<γ i <1.0 is a constant learning rate, 0<ε i <<1, e.g. 10 -8 , to avoid division by 0.
[0154] S48, bias b for user u's rating u , with gradient b u , as shown below:
[0155]
[0156] Adopting the adaptive moment estimation (Adam) method, we get b u The update rules are as follows:
[0157]
[0158] in, is the exponentially decaying average of the corrected past gradients, is the exponentially decaying average of the modified square gradient, 0<γ bu <1.0 is a constant learning rate, 0<ε bu <<1, to avoid division by 0.
[0159] S49, bias b for scoring item i i , with gradient b i , as shown below:
[0160]
[0161] Adopting the adaptive moment estimation (Adam) method, we get b i The update rules are as follows:
[0162]
[0163] in, is the exponentially decaying average of the corrected past gradients, is the exponentially decaying average of the modified square gradient, 0<γ bi <1.0 is a constant learning rate, 0<ε bi <<1, e.g. 10 -8 , to avoid division by 0.
[0164] S5. Build a general federated learning framework. In the matrix decomposition scenario of federated learning that integrates explicit feedback and implicit feedback, use explicit feedback data and implicit feedback data to train the model. Users save implicit factors, weight parameters, and explicit factors locally, and only participate in the training of the global model by uploading some parameters.
[0165] To achieve this goal, a federated collaborative filtering recommendation method is adopted to decompose the iterative update process of model parameters into two parts, which are performed on the client and server respectively. User local updates refer to updates performed on the local client of user u; server updates refer to updates performed on the server, that is, updates of explicit project profiles. This decomposition prevents the server from directly understanding the user's original preference data or learned profiles. As shown in Table 1, a federated matrix decomposition method that integrates explicit and implicit feedback is presented. The server continuously provides the latest project profile matrix V for all users to download. Each user u has the latest downloaded V, its own implicit factors, explicit factors, and weight parameters, and then performs local updates and calculates the Gradient i , and then Gradient i Sent to the server to update the project configuration file.
[0166] Table 1 Explicit and implicit matrix factorization algorithms based on federated learning
[0167]
[0168]
[0169] S6. Integrate Paillier semi-homomorphic encryption into the federated learning framework to encode gradients.
[0170] In order to solve the problem of gradient leakage of user privacy and further enhance the privacy protection ability of the federated learning framework, Paillier semi-homomorphic encryption is used to encode the gradient, so that the server cannot reverse the encoding process. The encoded data will not leak any information, and the server can still perform updates based on the encoded gradient.
[0171] like Figure 2 As shown in the figure, a federated framework with Paillier semi-homomorphic encryption that integrates explicit and implicit feedback, namely FedEIFCF, is presented. There are two types of participants involved in this framework, namely servers and users. Assuming that the server is honest but curious and the user is honest, the user's privacy cannot be protected on the server.
[0172] First, in the key generation phase, semi-homomorphic encryption operations are used to generate public and private keys, and the key generation process is only performed on one of the users. At the same time, all participants, including the server, know the public key. The private key is only shared between users and is not protected by the server. After the keys are generated, different TLS / SSL secure channels are established to send the public and private keys to the corresponding participants.
[0173] Then, before starting the matrix factorization, the parameters are initialized. The explicit project profile matrix is initialized on the server, while the parameters such as the explicit user profile matrix, the implicit project profile matrix, and the explicit user project profile matrix are initialized by each user on the local client.
[0174] Finally, matrix decomposition is performed. The specific process is as follows:
[0175] (1) The server uses the public key to encrypt the explicit project configuration file V and obtains the ciphertext C v From this moment on, the server is ready with the latest C v Available for download to all users.
[0176] (2) Each user downloads the latest C v , and decrypt it with the private key to obtain the plaintext V of the explicit project configuration file. Next, the user configuration file U and gradient G are calculated and updated on the local client to complete the following operations.
[0177] First, calculate and update the implicit user vector X u· and the implicit item vector Y i· , the implicit user vector X u· As the explicit user vector U u· , the implicit item vector Y i· As an explicit item vector V i· Then, the user's rating bias b is calculated and updated locally u and the rating bias b of the item i , which is used to correct the rating deviation of users and items. Finally, V is used to perform local updates, and the explicit user profile U and gradient G are calculated and updated locally. After that, G is encrypted with the public key to obtain the ciphertext C G After getting the ciphertext C G After that, a TLS / SSL secure channel is established, and C G Send to the server.
[0178] (3) After receiving the encrypted gradient C from the user, the server G Finally, update the project configuration file with this ciphertext: After that, the server updates the latest C v Available for download to all users.
[0179] (4) Iterate steps (2) and (3) until a certain number of iterations is reached.
[0180] S7. Analyze the safety and recommended performance of the FedEIFCF method.
[0181] 1) Security analysis. In this framework, users only send ciphertext to the server, and the data involved in training in the server are all ciphertext, so that the project configuration files downloaded by users from the server are also encrypted. Therefore, the data transmitted in the entire framework are encrypted data, and the calculations involving plaintext are only performed on the user's local client. All data except the initialized project configuration file is also completely confidential to the server, preventing gradients from leaking user privacy, thereby ensuring the privacy security of users. Therefore, as long as the homomorphic encryption system can ensure that the ciphertext is not recognized by the selected plaintext attack, no information will be leaked to the server.
[0182] 2) Recommendation performance analysis. The FedEIFCF method embeds the parameters obtained by training the implicit feedback matrix decomposition model into the explicit feedback matrix decomposition model, and uses the implicit factors as a supplement to the explicit factors. Both explicit data and implicit data participate in the training of the model, effectively making up for the shortcomings of explicit data and implicit data, and further improving the accuracy of recommendations. If the semi-homomorphic encryption part is removed, the parameter update process of the FedEIFCF method is the same as the federated matrix decomposition process. Since the semi-homomorphic encryption method does not confuse the data, it is lossless, thus ensuring the accuracy of the recommendation.
[0183] Example
[0184] 1. Experimental dataset and evaluation indicators.
[0185] This example uses the real dataset MovieLens 1M (ML1M) from http: / / files.grouplens.org / datasets / movielens / , and constructs three training sets and test sets based on it. The experimental results of recommendation accuracy are the average performance of the three test data. In addition, experiments on potential dimension, convergence and time cost are conducted on the first training set and test set.
[0186] The experimental environment is: 13th Gen Intel(R)Core(TM)i7-13700H 2.40GHz processor, 32.00GB running memory, Windows11 operating system, and Python language is used for simulation experiments.
[0187] The FedEIFCF method will provide each user with a ranked list of items as a recommendation list. In order to evaluate the recommendation list of each user's preferred item set in the test data, precision (Prec) and normalized discounted cumulative gain (NDCG) are used as evaluation indicators, and Prec@5 and NDCG@5 are used to evaluate the top 5 items in the ranked list.
[0188] 2. Baseline and parameter settings.
[0189] In order to evaluate the performance and efficiency of the proposed FedEIFCF method, the proposed method was compared with the following methods:
[0190] 1. UserCF: A classic neighborhood-based recommendation algorithm that uses user similarity to recommend items that similar users like to the target user. It is often used as a baseline for recommendation systems.
[0191] 2. P-NSMF: A federated collaborative filtering recommendation algorithm based on implicit feedback. Similar to FedEIFCF, it also uses a non-sampling matrix factorization algorithm to solve the implicit vector and also adopts a federated framework. Here, the P-NSMF method based on BGD optimization is used for comparative experiments. It optimizes user-specific latent feature vectors and item-specific latent feature vectors through BGD.
[0192] 3. Bias LFM: A classic recommendation algorithm based on explicit feedback. FedEIFCF adopts this model as the explicit feedback model.
[0193] 4. EIFCF: A collaborative filtering recommendation algorithm that integrates explicit feedback and implicit feedback. It uses the GALS algorithm to solve the implicit feedback vector, and then uses the gradient descent method to solve the fused explicit feedback vector.
[0194] 5. SVD++: A classic recommendation algorithm based on explicit feedback and implicit feedback. It uses the behavior data that hides personal preferences as implicit behavior data, and then combines it with the explicit behavior data, and uses SGD to solve it.
[0195] 6. FedEIFCF(NP): FedEIFCF does not add federated collaborative filtering recommendation with semi-homomorphic encryption.
[0196] The detailed configuration of the parameters are as follows:
[0197] (1) The number of iterations is fixed at T = 500 to ensure that all models converge. In addition, the latent dimension of all implicit feedback models is set to d = 20.
[0198] (2) P-NSMF adopts a uniform weighting strategy for missing data, so ω is set for both P-NSMF and FedEIFCF. ui =ω0 and α u =β i=1. Two common hyperparameters are selected for the model, namely, the weight value ω0 of the observed data and the regularization parameter λ. Specifically, following P-NSMF, the optimal weight value of the data is obtained from ω0∈{2,3,4,5,6}, i.e., ω0=4. In addition, the optimal value of the regularization parameter is found from λ∈{0,0.1,0.01,0.001}, i.e., λ=0.01, and the regularization parameter λ of all models is set to λ=0.01.
[0199] (3) For SVD++, Bias LFM and EIFCF, the relevant weight value δ of the missing value is set, that is, δ = 0.9.
[0200] (4) For FedEIFCF, find the optimal value of the learning rate γ from {2.6, 2.8, ···, 3.6}, i.e., γ = 3.0, and set the learning rate γ of all models to γ = 3.0.
[0201] In the implementation, multiprocessing programming in Python is used to simulate the federated learning system, where each client and server is implemented as a process.
[0202] 3. Recommended performance experimental results and analysis.
[0203] In order to verify the recommendation results of the FedEIFCF method proposed in this invention, FedEIFCF is compared with UserCF, P-NSMF, Bias LFM, EIFCF, SVD++ and FedEIFCF(NP) on three data sets, and the average values of Prec@5 and NDCG@5 are obtained, as shown in Table 2.
[0204] Table 2 Experimental results of recommended performance
[0205] method Prec@5 NDCG@5 UserCF 0.1708±0.0028 0.1278±0.0025 P-NSMF 0.4717±0.0033 0.4893±0.0037 BiasLFM 0.1664±0.0017 0.1241±0.0022 EIFCF 0.1832±0.0041 0.1416±0.0036 SVD++ 0.1687±0.0012 0.1272±0.0018 FedEIFCF 0.4872±0.0029 0.5065±0.0034 FedEIFCF(NP) 0.4874±0.0016 0.5066±0.0041
[0206] As can be seen from Table 2, FedEIFCF can achieve the same recommendation performance as FedEIFCF (NP) without privacy protection on the dataset. This shows that FedEIFCF can protect user privacy without sacrificing any performance. Compared with the recommendation algorithms based on implicit feedback data, FedEIFCF is better than P-NSMF (optimized by BGD); compared with the recommendation algorithms based on explicit feedback data, FedEIFCF is significantly better than UserCF and Bias LFM; this shows that the recommendation algorithm that integrates explicit and implicit feedback data has better recommendation performance. In addition, compared with EIFCF and SVD++ methods with explicit and implicit feedback, FedEIFCF uses a non-sampling matrix decomposition model to model the complete implicit feedback data, covering a wider range of data and having better recommendation performance.
[0207] In order to verify the impact of different latent dimensions d (i.e., d∈{20,40,60,80,100}) on the recommendation performance, comparative experiments are conducted on P-NSMF, EIFCF, SVD++, FedEIFCF, and FedEIFCF(NP). Figure 3 As shown in the figure, it can be seen that when the value of the latent dimension d is larger, FedNSMF can still achieve better performance than EIFCF and SVD++, and the same performance as FedNSMF (NP). At the same time, as the value of the latent dimension d increases, the recommendation performance may be affected, which may be due to the fact that the value of the latent dimension d is too large, causing the overfitting problem.
[0208] 4. Convergence performance experimental results and analysis.
[0209] In order to verify the convergence speed and stability of the FedEIFCF algorithm during training, comparative experiments were conducted on FedEIFCF, P-NSMF, EIFCF and SVD++ on the first data set. Figure 4 As shown in the results, it can be seen that the convergence curve of FedEIFCF is similar to that of EIFCF, which shows that the proposed federation framework has no significant effect on the convergence of the basic model. At the same time, it can be seen that although FedEIFCF with explicit and implicit feedback does not converge as fast as P-NSMF with only implicit feedback, the recommendation accuracy of FedNSMF is better than that of P-NSMF, and the difference in convergence speed is not large.
[0210] 5. Time expenditure experimental results and analysis.
[0211] In order to verify the feasibility of the FedEIFCF algorithm, the time spent by FedEIFCF, P-NSMF, EIFCF and SVD++ is compared. As shown in Table 3, when the iterations are 500, the time spent by FedEIFCF and EIFCF is relatively small.
[0212] Table 3 Comparison of time expenditure
[0213] method P-NSMF EIFCF SVD++ FedEIFCF Time(s) 25426.76 48461.59 78255.68 50467.93
[0214] Therefore, the present invention adopts the above-mentioned federated collaborative filtering recommendation method integrating explicit and implicit feedback, and aims at the problem that the recommendation accuracy of a single feedback form is not high. The parameters obtained by training the implicit feedback matrix decomposition model are embedded in the explicit feedback matrix decomposition model, and Top-N recommendations are performed using explicit feedback data and implicit feedback data. The FedEIFCF method takes advantage of the fact that explicit feedback data can clearly reflect the user's preference, and at the same time introduces implicit feedback data to make up for the shortcoming of low amount of user explicit feedback data caused by users' unwillingness to rate, so as to achieve the purpose of improving recommendation accuracy.
[0215] There is a risk of privacy leakage in the recommendation process. The federated learning framework is introduced to save the user's original rating matrix, interaction matrix, user parameters and other information locally. The user only needs to upload the intermediate parameters to participate in the training of the global recommendation model to obtain the recommendation results. However, the attacker can still infer the user's privacy information based on the continuously uploaded parameters. To address this problem, Paillier semi-homomorphic encryption is introduced to encrypt the intermediate parameters so that all data transmitted in the federated learning framework are encrypted to enhance privacy protection. Because Paillier semi-homomorphic encryption technology does not require data obfuscation, the accuracy of the recommendation is guaranteed. Finally, the recommendation performance, convergence performance and time cost of the FedEIFCF method are verified through experiments. The experimental results show that the FedEIFCF method uses explicit feedback data and implicit feedback data for recommendation, which improves the accuracy of the recommendation and enhances privacy protection.
[0216] Finally, it should be noted that the above embodiments are only used to illustrate the technical solution of the present invention rather than to limit it. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that they can still modify or replace the technical solution of the present invention with equivalents, and these modifications or equivalent replacements cannot cause the modified technical solution to deviate from the spirit and scope of the technical solution of the present invention.
Claims
1. A federated collaborative filtering recommendation method integrating explicit and implicit feedback, characterized in that: The following steps are involved: S1. Construct a non-sampling matrix factorization model to decompose the implicit feedback matrix and separate the weight parameters involving user privacy; S2. On the basis of building an implicit feedback model, ALS is used to solve the implicit user vector and implicit item vector to optimize the parameters of the implicit feedback model; S3, introduce baseline evaluation, build a matrix decomposition model based on explicit feedback data, and generate an explicit matrix decomposition model that integrates implicit feedback data; S4, using SGD to perform matrix decomposition, using explicit feedback data and implicit feedback data for Top-N recommendation, and optimizing explicit and implicit feedback model parameters; S5. Build a general federated learning framework to train models using explicit feedback data and implicit feedback data; S6. Integrate Paillier semi-homomorphic encryption into the federated learning framework to encode gradients. S7. Analyze the safety and recommended performance of the FedEIFCF method.
2. According to claim 1, a federated collaborative filtering recommendation method integrating explicit and implicit feedback is characterized in that: In step S1, a non-sampling matrix decomposition model is constructed to decompose the implicit feedback matrix and separate the weight parameters related to user privacy, including the following steps: S11. Use matrix decomposition method to decompose the user-item interaction matrix, and each entry z ui The estimate of ∈Z is as follows: in, represents the implicit user vector of user u, The implicit item vector representing item i; S12. Adopt the NSMF general loss function with implicit feedback to separate the weight parameters related to user privacy, as shown below: Among them, u represents the user, i represents the project, represents the observed (u,i) pair; U represents the entire user set, I represents the entire item set, ω ui represents the weight of the observed (u,i) pair; λ is a regularization parameter, α u is the user-oriented weight of user u in the missing data, β i is the item-oriented weight for item i in the missing data.
3. The method for federated collaborative filtering recommendation integrating explicit and implicit feedback according to claim 1, characterized in that: In step S2, based on the construction of the implicit feedback model, ALS is used to solve the implicit user vector and the implicit item vector, and the parameters of the implicit feedback model are optimized. The specific process is as follows: S21. For each user u, there is a gradient X u· , as shown below: Among them, I u represents the item interacted by user u, and m represents the number of users; S22, using cache strategy to optimize computational complexity, let The update rule for each user u based on ALS is as follows: in, D is a d×d identity matrix; cache S Y It is independent of u and can speed up the calculation of all subsequent user potential factors by pre-calculating them; S23, using the cache strategy, for each item i, there is a gradient Y i· , as shown below: Among them, cache n represents the number of items; S24, Order Based on ALS Y i· Update the rules as follows: Among them, cache S X Regardless of i, by pre-calculating S X To speed up the calculation of potential factors for all subsequent projects.
4. The method for federated collaborative filtering recommendation integrating explicit and implicit feedback according to claim 1, characterized in that: In step S3, a baseline evaluation is introduced to construct a matrix decomposition model based on explicit feedback data, and an explicit matrix decomposition model integrating implicit feedback data is generated. The specific process is as follows: S31, using matrix decomposition method, the actual user-item rating matrix R mn Breaking it down, it looks like this: For all elements r in R ui ≠0, its objective function is as follows: S32: Introduce the rating baseline estimation. The rating baseline of user u for item i is estimated using b ui Represented as follows: b ui =μ+b u +b i (9) Among them, μ represents the mean score of all items, b u represents the bias of user u's rating, b i represents the bias of the score of item i; S33. Prediction of user preference is as follows: S34, embed the implicit user potential feature vector X and the implicit item potential feature vector Y trained by the implicit feedback model into the explicit feedback model, and supplement the explicit user vector U and the explicit item vector V, as shown below: S35. Substitute formula (11) into formula (8), and the final objective function form is as follows: The explicit user vector U and the explicit item vector V are obtained by training the explicit feedback model, and Top-N recommendations are performed using explicit feedback data and implicit feedback data.
5. The method for federated collaborative filtering recommendation integrating explicit and implicit feedback according to claim 1, characterized in that: In step S4, on the basis of constructing an explicit matrix decomposition model integrating implicit feedback data, SGD is used to perform matrix decomposition, explicit feedback data and implicit feedback data are used to perform Top-N recommendation, and the parameters of the explicit and implicit feedback models are optimized. The specific process is as follows: S41, using the cache strategy, for each user u, there is a gradient U u· , as shown below: in, To predict deviation, cache S42, SGD-based U u· Update the rules as follows: S43, using the adaptive moment estimation method, the gradient descent process is carried out by two independent parts, recording the past gradient m u and squared gradient v u The exponentially decaying average of is as follows: Among them, 0<μ1,μ2<1,m u and v u Initialized to 0; S44, in order to offset the deviation, m u and v u The corrected version is as follows: S45, U u· Update the rules as follows: Among them, 0<γ u <1.0 is a constant learning rate, 0<ε u <<1; S46, using the cache strategy, for each item i, there is a gradient V i· , as shown below: Among them, cache S47, using the adaptive moment estimation method, obtain V i· The update rules are as follows: in, is the exponentially decaying average of the corrected past gradients, is the exponentially decaying average of the modified square gradient, 0<γ i <1.0 is a constant learning rate, 0<ε i <<1; S48, bias b for user u's rating u , with gradient b u , as shown below: Adopting the adaptive moment estimation method, we can get b u The update rules are as follows: in, is the exponentially decaying average of the corrected past gradients, is the exponentially decaying average of the modified square gradient, 0<γ bu <1.0 is a constant learning rate, 0<ε bu <<1; S49, bias b for scoring item i i , with gradient b i , as shown below: Adopting the adaptive moment estimation method, we can get b i The update rules are as follows: in, is the exponentially decaying average of the corrected past gradients, is the exponentially decaying average of the modified square gradient, 0<γ bi <1.0 is a constant learning rate, 0<ε bi <<1.
6. The method for federated collaborative filtering recommendation integrating explicit and implicit feedback according to claim 1, characterized in that: In step S5, the federated collaborative filtering recommendation method is used to decompose the iterative update process of model parameters into two parts, which are executed on the client and server respectively. The server provides the latest project profile matrix V for all users to download. Each user u has the latest downloaded V, its own implicit factors, explicit factors and weight parameters, and then performs local updates and calculates the gradient. i , and then Gradient i Sent to the server to update the project configuration file.
7. The method for federated collaborative filtering recommendation integrating explicit and implicit feedback according to claim 1, characterized in that: In step S5, Paillier semi-homomorphic encryption is integrated into the federated learning framework to encode the gradient. The specific process is as follows: First, in the key generation phase, semi-homomorphic encryption operations are used to generate public and private keys. The key generation process is only performed on one of the users, and all participants, including the server, know the public key. The private key is only shared between users and is not protected by the server. After the key is generated, different TLS / SSL secure channels are established to send the public and private keys to the corresponding participants. Then, before starting the matrix decomposition, the parameters are initialized. The explicit project profile matrix is initialized on the server, and the explicit user profile matrix, implicit project profile matrix, and explicit user project profile matrix parameters are initialized by each user on the local client; Finally, matrix decomposition is performed.
8. The method for federated collaborative filtering recommendation integrating explicit and implicit feedback according to claim 7, characterized in that: The specific process of matrix decomposition is as follows: (1) The server uses the public key to encrypt the explicit project configuration file V and obtains the ciphertext C v From this moment on, the server is ready for the latest C v Available for download by all users; (2) Each user downloads the latest C v , and decrypt it with the private key to obtain the plaintext V of the explicit project configuration file; calculate and update the user configuration file U and gradient G on the local client to complete the following operations; First, calculate and update the implicit user vector X u· and the implicit item vector Y i· , the implicit user vector X u· As the explicit user vector U u· , the implicit item vector Y i· As an explicit item vector V i· Then, the user's rating bias b is calculated and updated locally u and the rating bias b of the item i , used to correct the rating bias of users and items; finally, perform local updates with V to locally calculate and update the explicit user profile U and gradient G; Afterwards, G is encrypted with the public key to obtain the ciphertext C G ; After getting the ciphertext C G After that, a TLS / SSL secure channel is established, and C G Send to the server; (3) After receiving the encrypted gradient C from the user, the server G Finally, update the project configuration file with this ciphertext: Server update latest C v Available for download by all users; (4) Iterate steps (2) and (3) until a certain number of iterations is reached.