A Bayesian personalized matrix factorization recommendation method based on differential privacy

Abstract

The present invention relates to a Bayesian personalized matrix decomposition recommendation method based on differential privacy, belonging to the fields of computer recommendation systems and information security. The method specifically comprises: converting a user's implicit feedback data into an implicit feedback matrix with a value of 0 or 1; constructing an objective function L(W, H), solving the matrix W and the matrix H based on a recommendation algorithm under Bayesian personalized ranking, and storing the user latent factor matrix W; substituting the obtained user latent factor matrix W and the item matrix into the objective function L(W, H), rewriting the objective function to solve the latent factor matrix using a gradient descent method, considering the gradient of each item latent factor and perturbing the gradient using a small amount of Laplace noise; performing gradient descent on the objective function L(H|D) to obtain the matrix. The present invention has good privacy protection and model accuracy.

Description

Technical Field

[0001] The present invention belongs to the fields of computer recommendation systems and information security, and relates to a Bayesian personalized matrix decomposition recommendation method based on differential privacy. Background Art

[0002] Recommender systems, as an effective technology for alleviating information overload, have been widely used in a variety of fields, including online shopping, social networking, and online movies. Typically, recommendation models model user preferences based on their personal information and provide personalized services. Recommendation technology greatly facilitates various user activities. However, this convenience comes at the expense of user privacy. Some studies have shown that attackers can use auxiliary information and recommendation results to infer user behavior history and analyze sensitive user characteristics, posing a significant threat to user privacy. With growing awareness of privacy protection, how to provide recommendation services while preserving user privacy has become a hot topic. In this context, recommendation systems must consider user privacy.

[0003] Differential privacy is a privacy-preserving framework that quantifies the level of privacy protection. Its low computational cost and strong scalability make it a promising privacy-preserving approach. Within this framework, changes to a single piece of data have minimal impact on the processed dataset, making it impossible for an attacker to infer the private information of a specific data sample from a publicly available dataset. Furthermore, differential privacy assumes that the attacker possesses maximum background knowledge—that is, information about all data other than the target data. Under this maximum background knowledge assumption, differential privacy can protect the privacy of a single piece of data without considering any information the attacker may possess. Furthermore, differential privacy provides a rigorous definition of privacy protection based on mathematical statistics and proposes a quantitative evaluation method, making the privacy protection levels of datasets processed with different parameters comparable. Summary of the Invention

[0004] In view of this, the purpose of the present invention is to provide a Bayesian personalized matrix decomposition recommendation method based on differential privacy to achieve a good balance between security and usability of the collaborative filtering recommendation model.

[0005] In order to achieve the above object, the present invention provides the following technical solutions:

[0006] A Bayesian personalized matrix factorization recommendation method based on differential privacy includes the following steps:

[0007] S1: The implicit feedback matrix with a value of 0 or 1 represents the user's implicit feedback data on the project. 0 represents the user's negative feedback on the project, and 1 represents the user's positive feedback on the project.

[0008] S2: Randomly initialize matrices W, H, W is the user latent factor matrix, represents the u-th row of the matrix W, representing the latent factor vector of the u-th user; H is the item latent factor matrix, The i-th row of the matrix H represents the latent factor vector of the i-th item; is the noisy item latent factor matrix, Representation matrix The i-th row represents the latent factor vector of the i-th item with noise added; d represents the dimension of the latent factor vector;

[0009] S3: Construct the dataset D and the objective function L(W, H), use the matrix decomposition recommendation method based on Bayesian personalized ranking to solve the matrix W and matrix H, and save the matrix W;

[0010] S4: Fix the matrix W obtained in S3 and substitute it into L(W, H), and reconstruct The objective function calculate about Gradient Determine the gradient sensitivity;

[0011] S5: Gradient sensitivity Do a norm clipping, sample noise from the Laplace distribution, and adjust the gradient Add noise to get the noisy gradient use renew

[0012] S6: According to the formula Make predictions, sort the prediction results of the non-positive feedback items in the implicit feedback matrix described in S1, and recommend the TOP-N items to the user.

[0013] Furthermore, in step S1, the user's implicit feedback data on the project is converted into an implicit feedback matrix with a value of 0 or 1. r ui ∈R represents the implicit feedback of user u on item i, the value 0 represents the user's negative feedback on the item, the value 1 represents the user's positive feedback on the item, U is the set of all users, I is the set of all items, |U| represents the number of users in U, and |I| represents the number of items in I.

[0014] Furthermore, randomly initialize the matrices W, H, W is the user latent factor matrix, represents the u-th row of the matrix W, representing the latent factor vector of the u-th user; H is the item latent factor matrix, The i-th row of the matrix H represents the latent factor vector of the i-th item; is the noisy item latent factor matrix, Representation matrix The i-th row represents the latent factor vector of the i-th item plus noise; d represents the dimension of the latent factor vector.

[0015] Furthermore, in step S3, the objective function L(W, H) is constructed, the matrix W and the matrix H are solved using the matrix decomposition recommendation method based on Bayesian personalized ranking, and the matrix W is saved. Specifically, the following steps are included:

[0016] S31: Construct a dataset D, which includes multiple triples<u,i,j> , where u represents the user, i represents the item that has interacted with user u, and j represents the item that has not interacted with user u; for each non-zero element r in the implicit feedback matrix R ui , randomly sample from the set of items that user u has not interacted with items to construct triples<u,i,j> Based on the constructed dataset D, the implicit feedback matrix is ​​decomposed into two low-dimensional matrices W and H, and the objective function of the user matrix W and the item matrix H is constructed:

[0017]

[0018] in, is the loss function, λ W represents the regularization coefficient of the user vector, represents the regularization coefficient of the item vector that the user has interacted with, represents the regularization coefficient of the item vector with no interaction with the user, and ||·||2 represents the l2 norm.

[0019] S32: Use stochastic gradient descent to minimize the objective function L(W, H), randomly selecting triplets in the dataset D<u,i,j> , calculate w respectively u , h i , h j Gradient Then use the gradient Update the user latent factor vector w u and the item latent factor vector h i , h j ; For user latent factor vector w u and the item latent factor vector h i , h j The update rules are as follows:

[0020]

[0021]

[0022]

[0023] Where η1 represents the learning rate, and the formula for calculating the gradient is as follows:

[0024]

[0025]

[0026]

[0027] Repeat step S32 to iterate W and H. When W and H meet the stopping condition, the iteration ends and the matrix W is saved.

[0028] Further, in step S4, the matrix W obtained in S3 is fixed and substituted into L(W, H) to construct The objective function calculate about Gradient Determine the gradient The sensitivity of the test is as follows:

[0029] S41: Fix the matrix W obtained in S3 and substitute it into L(W, H) to construct The objective function

[0030]

[0031] in, represents the vector of item i that user u has interacted with, represents the vector of item j that user u has not interacted with, represents the regularization coefficient of the item vector that has interacted with user u, represents the regularization coefficient of the item vector that has no interaction with user u, and ||·||2 represents the l2 norm;

[0032] S42: For any triple in the dataset D<u,i,j> , respectively find the objective function about and Gradient and The calculation formula is as follows:

[0033]

[0034]

[0035] in, represents the gradient of item i that interacts with user u, represents the gradient of item j with no interaction with user u.

[0036] S43: Calculate the sensitivity Δ of the gradient of the project latent factor vector; the formula is as follows:

[0037] Δ=max{Δ + , Δ -}

[0038]

[0039]

[0040] Among them, w max and h max denote the upper bound of the l1 norm of the user vector and item vector, Δ + express sensitivity, express sensitivity.

[0041] S44: For all triples in dataset D<u,i,j> , calculate the gradient of item i and item j respectively and According to the sensitivity Δ Perform l1 norm clipping, the clipping formula is as follows:

[0042]

[0043]

[0044] Aggregate all triples<u,i,j> Calculated and

[0045]

[0046]

[0047] in From Laplace distribution Sampling noise ∈ is the total privacy budget, T2 is the number of iterations. Use noise B to aggregate the gradient results Perturbation, gradient aggregation matrix after perturbation as follows:

[0048]

[0049] use Pair Matrix Make an update

[0050]

[0051] Where η2 is the learning rate; repeat step S44 for Iterate, The iteration ends when the stopping condition is met.

[0052] The beneficial effects of the present invention are as follows: the present invention addresses the privacy leakage problem in the Bayesian personalized matrix decomposition recommendation method based on differential privacy, and proposes a Bayesian personalized matrix decomposition recommendation method based on differential privacy, which decomposes the user's implicit feedback data matrix for the project into two low-dimensional matrices, representing the user latent factor matrix and the project latent factor matrix respectively. Since directly using Laplace noise to perturb the objective function will make the added noise too large, it will seriously affect the usability of the recommendation model. Therefore, the objective function is solved by using a gradient descent solution method to reduce the sensitivity of the objective function, reduce the impact of noise on the objective function, and maximize the accuracy of the model. This algorithm can ensure the security of user privacy data while ensuring the usability of the recommendation model.

[0053] Other advantages, objects, and features of the present invention will be described in part in the following description and, in part, will be apparent to those skilled in the art upon examination of the following description or may be learned from practice of the present invention. The objects and other advantages of the present invention may be realized and obtained through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0054] In order to make the purpose, technical solutions and advantages of the present invention more clear, the present invention will be described in detail below with reference to the accompanying drawings, in which:

[0055] Figure 1 Schematic diagram of the solution of matrix W and matrix H by the recommendation algorithm based on Bayesian personalized ranking;

[0056] Figure 2 Schematic diagram of solving H using the gradient descent method (SGD) to add noise to the gradient;

[0057] Figure 3 Schematic diagram for recommending items to user u;

[0058] Figure 4 The comparison results of the present invention and other methods on HR indicators under different privacy budgets are shown;

[0059] Figure 5 The comparison results of the NDCG indicator between the present invention and other methods under different privacy budgets are shown in Figure 2. DETAILED DESCRIPTION

[0060] The following describes the embodiments of the present invention by means of specific examples, and those skilled in the art can easily understand other advantages and effects of the present invention from the contents disclosed in this specification. The present invention can also be implemented or applied through other different specific embodiments, and the details in this specification can also be modified or changed in various ways based on different viewpoints and applications without departing from the spirit of the present invention. It should be noted that the illustrations provided in the following embodiments are only schematic illustrations of the basic concept of the present invention, and the following embodiments and features in the embodiments can be combined with each other without conflict.

[0061] Among them, the accompanying drawings are only for illustrative purposes and represent only schematic diagrams rather than actual pictures, and should not be understood as limiting the present invention. In order to better illustrate the embodiments of the present invention, some parts of the accompanying drawings may be omitted, enlarged or reduced, and do not represent the dimensions of actual products. For those skilled in the art, it is understandable that some well-known structures and their descriptions may be omitted in the accompanying drawings.

[0062] The same or similar numbers in the drawings of the embodiments of the present invention correspond to the same or similar parts; in the description of the present invention, it should be understood that if there are terms such as "upper", "lower", "left", "right", "front", "back", etc. indicating directions or positional relationships, they are based on the directions or positional relationships shown in the drawings. They are only for the convenience of describing the present invention and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific direction, be constructed and operate in a specific direction. Therefore, the terms describing the positional relationship in the drawings are only used for illustrative purposes and cannot be understood as limiting the present invention. For ordinary technicians in this field, the specific meanings of the above terms can be understood according to specific circumstances.

[0063] See also Figures 1 to 3 , specifically including the following steps:

[0064] Step 1: The recommendation system converts the collected user implicit feedback data into an implicit feedback matrix with a value of 0 or 1. The value 0 indicates that the user has given a negative feedback to the project, and the value 1 indicates that the user has given a positive feedback to the project. U is the set of all users, I is the set of all projects, |U| is the number of users in U, and |I| is the number of projects in I.

[0065] Step 2: Recommendation system based on implicit feedback matrix and the dimension d of the latent factor vector to generate matrices W, H, And initialize.

[0066] Step 3: Construct a dataset D, which includes multiple triples<u,i,j> , where u represents the user, i represents the item that has interacted with user u, and j represents the item that has not interacted with user u; for each non-zero element r in the implicit feedback matrix R ui , randomly sample from the set of items that user u has not interacted with items to construct triples<u,i,j> Based on the constructed dataset D, the implicit feedback matrix is ​​decomposed into two low-dimensional matrices W and H, and the objective function of the user matrix W and the item matrix H is constructed:

[0067]

[0068] in, is the loss function, λ W represents the regularization coefficient of the user vector, represents the regularization coefficient of the item vector that the user has interacted with, represents the regularization coefficient of the item vector with no interaction with the user, and ||·||2 represents the l2 norm.

[0069] Step 4: Use stochastic gradient descent to minimize the objective function L(W, H), randomly selecting triples in the dataset D<u,i,j> , calculate w respectively u , h i , h j Gradient Then use the gradient Update the user latent factor vector w u and the item latent factor vector h i , h j ; For user latent factor vector w u and the item latent factor vector h i , h j The update rules are as follows:

[0070]

[0071]

[0072]

[0073] Where η1 represents the learning rate, and the formula for calculating the gradient is as follows:

[0074]

[0075]

[0076]

[0077] Repeat step S32 to iterate W and H. When W and H meet the stopping condition, the iteration ends and the matrix W is saved.

[0078] Step 5: Fix the matrix W obtained in step 4 and substitute it into L(W, H) to construct The objective function Build About The objective function calculate about Gradient Determine the gradient The sensitivity of the test is as follows:

[0079]

[0080] in, represents the vector of item i that user u has interacted with, represents the vector of item j that user u has not interacted with, represents the regularization coefficient of the item vector that has interacted with user u, represents the regularization coefficient of the item vector that has no interaction with user u, and ||·||2 represents the l2 norm;

[0081] For any triple in the dataset D<u,i,j> , respectively find the objective function about and Gradient and The calculation formula is as follows:

[0082]

[0083]

[0084] in, represents the gradient of item i that interacts with user u, represents the gradient of item j with no interaction with user u.

[0085] Step 6: Calculate the sensitivity Δ of the gradient of the project latent factor vector; the formula is as follows:

[0086] Δ=max{Δ + , Δ +}

[0087]

[0088]

[0089] Among them, w max and h maxrepresents the upper bound of the l1 norm of the user vector and item vector, Δ + Indicates h i The sensitivity of the gradient, Δ - Indicates h j The sensitivity of the gradient.

[0090] Step 7: For all triples in dataset D<u,i,j> , calculate the gradient of item i and item j respectively and According to the sensitivity Δ Perform l1 norm clipping, the clipping formula is as follows:

[0091]

[0092]

[0093] Aggregate all triples<u,i,j> Calculated and

[0094]

[0095]

[0096] in From Laplace distribution Sampling noise ∈ is the total privacy budget, T2 is the number of iterations. Use noise B to aggregate the gradient results Perturbation, gradient aggregation matrix after perturbation as follows:

[0097]

[0098] use Pair Matrix Make an update

[0099]

[0100] Where η2 is the learning rate; repeat step S44 for Iterate, The iteration ends when the stopping condition is met.

[0101] Step 8: Calculate the prediction vector of the uth user Sort the prediction results of non-positive feedback items in the implicit feedback matrix R and recommend the TOP-N items to the u-th user

[0102] Example

[0103] The Bayesian personalized matrix decomposition recommendation method based on differential privacy described in the present invention specifically includes the following steps:

[0104] Step 1: Convert the implicit feedback data of users to the project into an implicit feedback matrix with a value of 0 or 1. The value 0 represents the user's negative feedback to the project, and the value 1 represents the user's positive feedback to the project. According to the number of users |U| and the number of projects |I| contained in the implicit inverse matrix, initialize the user latent factor matrix W and the project latent factor matrix H. Construct the objective function L(W, H), and solve W and H using the gradient descent method and save them. The solution process is as follows: Figure 1 shown.

[0105] Step 2: Substitute W obtained in step 1 into L(W, H), reinitialize H, and get The objective function calculate about Gradient Determine the gradient Sensitivity; respectively from the Laplace distribution Sampling noise ∈ is the total privacy budget. Noise Gradient The specific process is as follows: Figure 2 shown.

[0106] Step 3: Calculate the user's prediction vector according to the formula in S6, sort the prediction results of non-positive feedback items in the implicit feedback matrix R, and recommend the TOP-N items to the u-th user. The recommendation process is as follows: Figure 3 shown.

[0107] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not limiting. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical solutions of the present invention can be modified or replaced by equivalents without departing from the purpose and scope of the technical solutions, which should all be included in the scope of the claims of the present invention.

Claims

1. A Bayesian personalized matrix decomposition recommendation method based on differential privacy, characterized by: The method specifically comprises the following steps: S1: The implicit feedback matrix with a value of 0 or 1 represents the user's implicit feedback data on the project. 0 represents the user's negative feedback on the project, and 1 represents the user's positive feedback on the project. S2: Randomly initialize matrices W, H, W is the user latent factor matrix, represents the u-th row of the matrix W, representing the latent factor vector of the u-th user; H is the item latent factor matrix, The i-th row of the matrix H represents the latent factor vector of the i-th item; is the noisy item latent factor matrix, Representation matrix The i-th row represents the latent factor vector of the i-th item with noise added; d represents the dimension of the latent factor vector; S3: Construct the dataset D and the objective function L(W,H), use the matrix decomposition recommendation method based on Bayesian personalized ranking to solve the matrix W and matrix H, and save the matrix W; S4: Fix the matrix W obtained in S3 and substitute it into L(W,H), and reconstruct The objective function calculate about Gradient Determine the gradient sensitivity; S5: Gradient sensitivity Do a norm clipping, sample noise from the Laplace distribution, and adjust the gradient Add noise to get the noisy gradient use renew S6: According to the formula Make predictions, sort the prediction results of the non-positive feedback items in the implicit feedback matrix described in S1, and recommend the TOP-N items to the user.

2. The Bayesian personalized matrix decomposition recommendation method based on differential privacy according to claim 1, characterized in that: In step S1, the user's implicit feedback data on the project is converted into an implicit feedback matrix with a value of 0 or 1. r ui ∈R represents the implicit feedback of user u on item i, the value 0 represents the user's negative feedback on the item, the value 1 represents the user's positive feedback on the item, U is the set of all users, I is the set of all items, |U| represents the number of users in U, and |I| represents the number of items in I.

3. The Bayesian personalized matrix decomposition recommendation method based on differential privacy according to claim 1, characterized in that: In step S2, the matrices W, H, W is the user latent factor matrix, represents the u-th row of the matrix W, representing the latent factor vector of the u-th user; H is the item latent factor matrix, The i-th row of the matrix H represents the latent factor vector of the i-th item; is the noisy item latent factor matrix, Representation matrix The i-th row represents the latent factor vector of the i-th item plus noise; d represents the dimension of the latent factor vector.

4. The Bayesian personalized matrix decomposition recommendation method based on differential privacy according to claim 1, characterized in that: In step S3, construct the objective function L(W,H), use the matrix decomposition recommendation method based on Bayesian personalized ranking to solve the matrix W and the matrix H, and save the matrix W; The specific steps include: S31: Construct a dataset D, which includes multiple triples<u,i,j> , where u represents the user, i represents the item that has interacted with user u, and j represents the item that has not interacted with user u; for each non-zero element r in the implicit feedback matrix R ui , randomly sample from the set of items that user u has not interacted with items to construct triples<u,i,j> Based on the constructed dataset D, the implicit feedback matrix is ​​decomposed into two low-dimensional matrices W and H, and the objective function of the user matrix W and the item matrix H is constructed: in, is the loss function, λ W represents the regularization coefficient of the user vector, represents the regularization coefficient of the item vector that the user has interacted with, represents the regularization coefficient of the item vector that has no interaction with the user, ‖·‖2 represents the l2 norm; S32: Use stochastic gradient descent to minimize the objective function L(W,H), randomly selecting triplets in the dataset D<u,i,j> , calculate w respectively u , h i , h j Gradient Then use the gradient Update the user latent factor vector w u and the item latent factor vector h i , h j ; For user latent factor vector w u and the item latent factor vector h i , h j The update rules are as follows: Where η1 represents the learning rate, and the formula for calculating the gradient is as follows: Repeat step S32 to iterate W and H. When W and H meet the stopping condition, the iteration ends and the matrix W is saved.

5. The Bayesian personalized matrix decomposition recommendation method based on differential privacy according to claim 1, characterized in that: In step S4, the matrix W obtained in S3 is fixed and substituted into L(W,H) to construct The objective function calculate about Gradient Determine the gradient sensitivity; The specific steps include: S41: Fix the matrix W obtained in S3 and substitute it into L(W,H) to construct The objective function in, represents the vector of item i that user u has interacted with, represents the vector of item j that user u has not interacted with, represents the regularization coefficient of the item vector that has interacted with user u, represents the regularization coefficient of the item vector that has no interaction with user u, ‖·‖2 represents the l2 norm; S42: For any triple in the dataset D<u,i,j> , respectively find the objective function about and Gradient and The calculation formula is as follows: in, represents the gradient of item i that interacts with user u, represents the gradient of item j with no interaction with user u; S43: Calculate the sensitivity Δ of the gradient of the project latent factor vector; the formula is as follows: Δ=max{Δ + ,D - } D + =w max +λ H +h max D - =w max +λ H -h max Among them, w max and h max denote the upper bound of the l1 norm of the user vector and item vector, Δ + express sensitivity, express sensitivity; S44: For all triples in dataset D<u,i,j> , calculate the gradient of item i and item j respectively and According to the sensitivity Δ Perform l1 norm clipping, the clipping formula is as follows: Aggregate all triples<u,i,j> Calculated and in From Laplace distribution Sampling noise ∈ is the total privacy budget, T2 is the number of iterations; use noise B to aggregate the gradient results Perturbation, gradient aggregation matrix after perturbation as follows: use Pair Matrix Make an update Where η2 is the learning rate; repeat step S44 for Iterate, The iteration ends when the stopping condition is met.

6. The Bayesian personalized matrix decomposition recommendation method based on differential privacy according to claim 1, characterized in that: In step S6, the prediction vector of the uth user is calculated Sort the prediction results of non-positive feedback items in the implicit feedback matrix R and recommend the TOP-N items to the u-th user.

Images