A recommendation method and system based on improved federated matrix decomposition

Abstract

This invention discloses a recommendation method based on improved federated matrix factorization, comprising: all users downloading the latent factor vector V of item j from a server. j ;Detect user u i Is this related to project v? j An evaluation was conducted; if the detection result is yes, then it is based on user u. i For project v j Rating r ij Calculate the user gradient ▽U locally i (j), update user u i The new latent factor vector U i Locally calculated project gradient ▽V j (i) will be the project gradient ▽V j (i) and user project interaction value S ij The project is randomly divided into p+1 parts, and the resulting gradient partitions and interaction information partitions are randomly sent to user u. i p users are randomly selected; user u i Update local project gradient ▽V j (i) and user project interaction value S ij And update the project gradient ▽V′ j (i) and user project interaction value S ij The data is sent to the server. By incorporating user and project interaction values ​​along with gradient information as private information, the server receives spurious information from all parties and cannot infer the user's true gradient. However, the final aggregated average parameter information is accurate and does not affect the accuracy.

Description

Technical Field

[0001] This invention belongs to the field of personalized recommendation technology, and more specifically, this invention relates to a recommendation method and system based on improved federated matrix factorization. Background Technology

[0002] With the widespread adoption of mobile devices, more and more consumers are purchasing goods online. The explosive growth in user numbers and the increasing variety of goods offered by suppliers have resulted in a deluge of information surrounding users, making it impossible for them to extract relevant data by browsing all online content. To address this information overload, Recommendation Systems (RecSys) have become widely used. RecSys helps consumers find products, news, friends, and more that may interest them. Therefore, from movie recommendations to product recommendations, recommendation systems are becoming increasingly prevalent in our daily lives. Collaborative Filtering (CF) is a key technology in RecSys, with model-based Matrix Factorization (MF) playing a crucial role in improving recommendation performance and proving to be the most successful CF recommendation strategy. However, predicting individual preferences requires more user privacy information, thus posing a risk of privacy breaches. In practical applications, most data owners face the problem of limited data volume, and there are insurmountable barriers between different data sources, hindering cross-domain sharing and exchange. This has led to a growing number of data silos in the current big data landscape. To address these challenges, Federated Learning (FL) has emerged. Federated learning is a recently proposed distributed framework that offers significant advantages in privacy-preserving machine learning services. Many clients collaboratively train models under the coordination of a central server, while keeping their data within their local environments. The core of Federated Learning (FL) is to address the issues of data silos and privacy protection.

[0003] In recent years, Federated Learning (FL) has received increasing attention for protecting user privacy and complying with laws and regulations. Federated Recommendation Systems (FedRec) can address the problem of traditional recommendation systems where untrusted third parties centrally collect user rating data, protecting the original user rating data (i.e., value privacy) from leakage. Participants protect value privacy by sending parameters to update the model instead of the original data. Training a Federated Learning (MF) model in a federated learning environment can protect the original user data from leakage; the most common method for training MF models is Stochastic Gradient Descent (SGD). However, if the server receives the gradients uploaded by users, it can infer rating information; this gradient represents procedural privacy. Therefore, procedural privacy protection is necessary for FedRec.

[0004] Existing research on privacy protection in FedRec's protection process mainly falls into two categories: encryption-based recommendation methods and perturbation-based recommendation methods. The former primarily uses homomorphic encryption (HE) technology for gradient encryption during client-server interactions; however, the continuous encryption and decryption of HE leads to excessive computational costs. The latter mainly utilizes differential privacy (DP) technology to add perturbations to real data to prevent privacy leaks, but this results in decreased accuracy. Furthermore, from a user perspective, users not only want to protect local information but also interaction information with projects, i.e., whether a user has rated a project—this is considered privacy-related. Summary of the Invention

[0005] This invention provides a recommendation method based on improved federated matrix factorization, which aims to improve the above-mentioned problems.

[0006] This invention is implemented as follows: a recommendation method based on improved federated matrix factorization, the method comprising the following steps:

[0007] S1. The server sends the latent factor vector V of project j to all users. j Update requests are made to enable each user to locally compute the project gradient ▽V j The project gradient and user project interaction values ​​are sharded locally and randomly sent to multiple random users. Each user updates the project gradient and user project interaction values ​​that are sent to the server locally.

[0008] S2. Average and aggregate the received item gradients and user item interaction values ​​sent by each user to obtain the average aggregated item gradient value. Then update the latent factor vector V of project j. j .

[0009] Furthermore, the specific formula for calculating the average aggregation is as follows:

[0010]

[0011] in, Indicates user u i Project v sent to the server j The gradient, S′ ij Indicates user u i Sending to the server with project v j User project interaction values.

[0012] This invention is implemented as follows: a recommendation method based on improved federated matrix factorization, the method comprising the following steps:

[0013] S1, V, the latent factor vector of all users based on item j j The update request downloads the latent factor vector V of project j from the server. j ;

[0014] S2, Detect user u i Is this related to project v? j An evaluation was conducted; if the detection result is yes, then it is based on user u. i For project v j Rating r ij Calculate user gradients locally Update user u i latent factor vector U i And calculate the project gradient locally.

[0015] S3, Project Gradients and user project interaction value S ij The project is randomly divided into p+1 parts, and the resulting gradient partitions and interaction information partitions are randomly sent to user u. i Randomly selected p One user;

[0016] S4, User u i Update local project gradients and user project interaction value S ij and update the project gradient and user project interaction value S ij Send to the server.

[0017] Furthermore, if user u i Project v not specified j To evaluate, the project gradient is then determined.

[0018] Furthermore, the p+1 interactive information fragments are randomly divided. Satisfy the following formula:

[0019]

[0020] in, For user u i Retained interactive information fragments, The interaction information fragment is randomly selected from the k-th users.

[0021] Furthermore, the gradient shards of the p+1 randomly partitioned items are... Satisfy the following formula:

[0022]

[0023] in, For user u i Retained project gradient sharding, Gradient sharding for the project of the randomly selected k-th user.

[0024] Furthermore, the updated user item interaction value S ij The specific formula for calculating ′ is as follows:

[0025]

[0026] Among them, U / u i Indicates excluding user u i Other users in the recommendation system besides those in the user set, U is the user set that stores all users in the recommendation system. Indicates excluding user u i Other users, Indicates user For project v j Interactive information, For interactive information After being randomly divided into p+1 parts, they are allocated to user u. i Interactive information fragments.

[0027] Furthermore, the updated project gradient The specific calculation formula is as follows:

[0028]

[0029] in, Indicates user u i For project v j Locally calculated project gradients For user u i Project gradient After being randomly divided into p+1 parts, they are allocated to user u. i Project gradient sharding.

[0030] Furthermore, the user project interaction value S is adopted. ij Indicates user u i Is this related to project v? j An evaluation was conducted, if user u i For project v j The user project interaction value S was evaluated. ij =1, user u i Project v not specified j The user project interaction value S was evaluated. ij =0.

[0031] The present invention is implemented as follows: a recommendation system based on improved federated matrix factorization, the system comprising: a server, which communicates with n users, and the users storing their local rating data;

[0032] Users update the item gradient and user item interaction values ​​based on the above-mentioned recommendation method based on improved federated matrix factorization, and send them to the server;

[0033] The server updates the updated latent factor vector V of item j based on the aforementioned recommendation method based on improved federated matrix factorization. j .

[0034] The recommendation method based on improved federated matrix factorization provided by this invention has the following beneficial technical effects:

[0035] (1) This invention introduces user item interaction values, which avoids the problem that the number of item gradients received by the server is more than the actual number of item gradients due to users sharing information with other users, and the average item gradient during aggregation is less than the real average item gradient, resulting in the recommendation accuracy being lower than the accuracy of the federated matrix factorization method.

[0036] (2) By introducing user and project interaction values ​​along with gradient information as private information transmission, the information received by the server from all parties is false and cannot infer the user's true gradient, but the final average parameter information after aggregation is true and will not affect the accuracy.

[0037] (3) The server transmits secret information to each participant and to each other using random parameter fragmentation, which greatly improves computational efficiency compared to existing homomorphic encryption techniques for protecting privacy. Attached Figure Description

[0038] Figure 1 A schematic diagram of the structure of a recommendation system based on improved federated matrix factorization provided in an embodiment of the present invention;

[0039] Figure 2 A flowchart of a recommendation method based on improved federated matrix decomposition provided in an embodiment of the present invention;

[0040] Figure 3 The graph shows the influence of the parameter factorNumk on the model accuracy on different datasets, where (a) is the MovieLens dataset, (b) is the filmTrust dataset, and (c) is the Epinions dataset.

[0041] Figure 4 The comparison results of various models on the MovieLens100K dataset provided in the embodiments of the present invention include (a) comparison of MAE of different models and (b) comparison of RMSE of different models.

[0042] Figure 5 The comparison results of various models on the filmTrust dataset provided in the embodiments of the present invention include, (a) comparison of MAE of different models, and (b) comparison of RMSE of different models.

[0043] Figure 6 The comparison results of various models on the Epinions dataset provided in the embodiments of the present invention include (a) comparison of MAE of different models and (b) comparison of RMSE of different models. Detailed Implementation

[0044] The specific embodiments of the present invention will be further described in detail below with reference to the accompanying drawings, so as to help those skilled in the art to have a more complete, accurate and in-depth understanding of the inventive concept and technical solution of the present invention.

[0045] This invention first decomposes the gradient locally on the user's end, then uses a secret sharing technique to transmit intermediate results between users. Next, it introduces user item interaction values, which, after decomposition, are shared along with spurious gradients with randomly selected users. Finally, the server aggregates the spurious information sent by users to obtain the true updated values, without reducing recommendation accuracy. This method protects user value privacy, process privacy, and existence privacy. Information transmitted between users and between users and servers is fragmented using random parameters, which does not introduce additional encryption costs.

[0046] Figure 1 This is a schematic diagram of the structure of a recommendation system based on improved federated matrix factorization provided in an embodiment of the present invention. For ease of explanation, only the parts related to the embodiment of the present invention are shown. The system includes:

[0047] The server communicates with n users, and each user's account stores the user's local rating data.

[0048] Figure 2 The flowchart of the recommendation method based on improved federated matrix factorization provided in the embodiments of the present invention includes the following steps:

[0049] S1. The server sends the latent factor vector V of project j to all users. j Update request;

[0050] In this embodiment of the invention, the server stores a project file V, which consists of potential factor vectors for each project.

[0051] S2, the latent factor vector V of all user items j j The update request downloads the latent factor vector V of project j from the server. j ;

[0052] S3, Detect user u i For project v j User project interaction value S ij If the value of is 1, then proceed to steps S4 and S5; if the result is no, then proceed to step S6.

[0053] If user u i For project v j The user project interaction value S was evaluated. ij =1, user u i Project v not specified j The user project interaction value S was evaluated. ij =0.

[0054] S4, based on user u i For project v j Rating r ij Calculate user gradients locally Calculations are performed based on formula (1), and user u is updated based on formula (2). i latent factor vector U i And calculate the local computation project gradient based on formula (3). The specific calculation formula is as follows:

[0055]

[0056]

[0057]

[0058] Among them, e ij Indicates user u i For project v j Scoring error, For user u i For project v j Predicted score α represents the learning rate, and λ represents the regularization parameter.

[0059] S5. Project gradient and user project interaction value S ij The project is randomly divided into p+1 parts, and the resulting gradient partitions and interaction information partitions are randomly sent to user u. i Randomly selected p One user;

[0060] In this embodiment of the invention, all users in the recommendation system are numbered from 0 to n-1, starting from... We randomly select p integers, which determines p randomly selected users. The resulting project gradient shards and interaction information shards are randomly sent to the p users. The resulting p+1 interaction information shards are... Satisfy the following formula:

[0061]

[0062] in, For user u i Retained interactive information fragments, The interaction information of the randomly selected k-th user is sharded. The resulting p+1 items are then divided into gradient shards. Satisfy the following formula:

[0063]

[0064] in, For user u i Retained project gradient sharding, Gradient sharding for the project of the randomly selected k-th user.

[0065] S6, Project Gradient

[0066] S7, User u i Update local project gradients and user project interaction value S ij and update the project gradient and user project interaction value S ij The updated user project interaction value S is sent to the server. ij The specific formula for calculating ′ is as follows:

[0067]

[0068] Among them, U / u iIndicates excluding user u i Other users in the recommendation system besides those in the user set, U is the user set that stores all users in the recommendation system. Indicates excluding user u i Other users, Indicates user For project v j Interactive information, For interactive information After being randomly divided into p+1 parts, they are allocated to user u. i Interactive information sharding, updated project gradient The specific calculation formula is as follows:

[0069]

[0070] in, Indicates user u i For project v j Locally calculated project gradients To deliver project gradient After being randomly divided into p+1 parts, they are allocated to user u. i Project gradient sharding.

[0071] In this embodiment of the invention, if user u i Project v not specified j An evaluation was conducted, then If user u i No interaction information fragments were received from other users in the recommendation system. and project gradient sharding but

[0072] S8, The server receives project gradients sent by each user. and interactive information S ij The FedAvg algorithm is used to average and aggregate the received gradients to obtain the aggregated total gradient value. Then update the latent factor vector V j .

[0073] In this embodiment of the invention, the formula for calculating the average aggregation of project gradients is as follows:

[0074]

[0075] Updated latent factor vector V j ←V j -α·ΔV j .

[0076] S9. Each user downloads the updated latent factor matrix V of the project from the server, and then obtains the user's rating of each project, recommending the project with the highest rating to the user.

[0077] For the server, the recommendation process based on improved federated matrix factorization is shown in Algorithm 1:

[0078]

[0079]

[0080] After the server initializes the project configuration file V, it sends an update request to the user for V. j The user requests that information be uploaded locally to the server via a secret sharing method for aggregation, thereby enabling the server to update V based on the aggregated information. j Vectors improve recommendation accuracy with the maximum number of iterations.

[0081] The maximum number of iterations is set to the number of projects in the project configuration file V. Each iteration completes the update of one project, where V can be a movie. j If V corresponds to a specific movie ID, or if the project V is a product ID, then V... j The product number corresponds to a specific brand.

[0082] For users, the recommendation process based on improved federated matrix factorization is shown in Algorithm 2:

[0083]

[0084]

[0085] Users download the latest V from the server. j Vector, according to <V j U i >Get user u j For project v j Predicted score r ij The scoring error e is calculated by comparing the predicted score with the actual score. ij Then update vector U locally based on the error. i This leads to the updated local project gradient. After generating the local project gradient, the gradient is randomly divided into p+1 parts, and the gradient and user project interaction information S are then... ij By sharing the information with the remaining p clients and incorporating user project interaction values, the problem of the server receiving more gradients than the actual number of gradients due to users sharing information with other users is solved, thus improving the accuracy of the algorithm.

[0086] Datasets: To verify the recommendation accuracy and model update time of the improved Federation Matrix Factorization (FMFSS) method proposed in this invention, experiments were conducted on four real-world datasets: MovieLens100K, filmTrust, and a subset of the Epinions dataset consisting of 664,824 ratings from 49,289 users on 139,738 items. The relevant attributes of these datasets are shown in Table 2.

[0087] Table 1 Dataset and Attributes

[0088]

[0089] Parameter settings: The learning rate α for all matrix factorizations in the paper was set to 0.05, and the regularization coefficient λ was set to 0.03. The homomorphic encryption method used was Paillier, with a key length of 1024. In the experiment, the training and test sets were randomly divided in a 9:1 ratio. The privacy budget ε for the added differential privacy noise was set to ε0. p =2ε i , ε i =0.25.

[0090] Experimental Environment: All experiments were conducted on a computer with a 2.50GHz 8-core CPU and 32GB of RAM. Accuracy metrics included root mean square error (RMSE) and mean absolute error (MAE).

[0091] (1) Root Mean Square Error (RMSE)

[0092]

[0093] (2) Mean Absolute Error (MAE)

[0094]

[0095] Where, r ij For user u i For project v j The predicted score, r ij For user u i The actual score for project vj, and Dtest is the actual score for the test dataset.

[0096] The comparison models include: (1) FCF: Federated Collaborative Filtering, which trains the matrix factorization model in a federated learning environment without adding any privacy protection methods; (2) FedMF-PartText: The gradient uploaded by the user is calculated based on the rated items, and all gradient values ​​are transmitted using homomorphic encryption to train the federated matrix factorization model; (3) FedMF-FullText: All gradients are uploaded, and the gradients of unrated items are replaced with 0. All gradient values ​​are transmitted using homomorphic encryption to train the federated matrix factorization model; (4) Two Stage RR: The two-stage stochastic response method and federated learning are combined into the matrix factorization framework, and noise is added during model training. The two stages satisfy ε-DP respectively, and intermediate parameters are transmitted using additive homomorphic encryption; (5) FMFSS: The federated matrix factorization algorithm based on secret sharing proposed in this paper, in which the user first decomposes the local gradient and existence information, and shares them with other users, and uses secret sharing technology to transmit spurious parameters.

[0097] Sensitivity Analysis: Parameter `factorNum`. This parameter is an implicit factor in MF (Fast Moving Model), affecting model accuracy and training time. The impact of different values ​​of `factorNum` on the accuracy of the FMFSS model is verified using three real-world datasets presented in the paper. Specific results are shown below. Figure 3 As shown (100 iterations, α = 0.05), (a) is the result on the MovieLens-100K dataset, (b) is the result on the filmTrust dataset, and (c) is the result on the Epinions dataset.

[0098] The value of factorNum is {5, 10, 20, 50, 100}, determined by... Figure 3 It can be seen that the model recommendation accuracy is better as the factorNum increases. When factorNum=100, the accuracy of the FMFSS method reaches the best level. In order to make a trade-off between accuracy and time, factorNum is set to 50 for all subsequent experiments in this paper.

[0099] To verify the recommendation accuracy of the algorithm proposed in this invention, the RMSE and MAE of the model were verified using three real datasets from the paper, with different maximum number of iterations. Specific results are as follows: Figures 4-6 As shown.

[0100] Depend on Figures 4-6It can be concluded that the RMSE and MAE values ​​of the proposed method FMFSS are similar to those of the comparative experiments FCF and FedMF-PartText, and the accuracy of the three is better than the other two: FedMF-FullText and two-stage RR. However, in terms of privacy protection: FCF only trains the recommendation model under the FL framework, protecting value privacy. The intermediate calculated values ​​are sent to the untrusted server in the form of random parameter shards, exposing process privacy, and indirectly exposing existence privacy while sending the real gradient; the FedMF-PartText model uses additive homomorphic encryption to encrypt the real gradient value, which also reveals the existence of the rating; the FedMF-FullText model sets the gradient value to 0 for the user's unrated items. Thus, the total gradient value received by the server remains unchanged, but the number of gradients increases greatly when aggregating parameters, resulting in a smaller average parameter and thus a decrease in model accuracy; the two-stage RR model adds noise to the user's original binary matrix to generate false rating interaction values ​​in order to protect process privacy and existence privacy. During model training, false gradients are sent to items with interaction values ​​but no rating behavior, thus reducing accuracy.

[0101] Table 2 Attributes of 5 subsets of MovieLens-100K

[0102]

[0103] Table 3 Attributes of 5 Subsets on filmTrust

[0104]

[0105] Table 4 shows the attributes of the five subsets of Epinions.

[0106]

[0107] To verify the communication efficiency of our proposed algorithm, we used subsets of the MovieLens100K, filmTrust, and Epinions datasets (the attributes of which are shown in Tables 2-4) to test the time required for each iteration of different models. The specific results are shown in Table 5 (factorNum = 50).

[0108] Table 5 shows the time required for each iteration of different models on 15 subset datasets (in seconds).

[0109]

[0110] Table 5 shows that the training time for models using homomorphic encryption is significantly longer than the other two methods. The FMFSS method proposed in this paper takes slightly longer than the FCF method because the FMFSS method based on secret sharing protects privacy. While sharing data among users takes some time, it is not significant. The FedMF-PartText model uses homomorphic encryption to transmit intermediate parameter information involving user item interactions, increasing the model training time. The FedMF-FullText model uses homomorphic encryption to transmit large amounts of intermediate parameter information, resulting in very high communication costs, which is considered unacceptable in reality. The Two-Stage RR model also requires high communication costs for parameter transmission using homomorphic encryption. The proposed FMFSS method requires users to randomly select users to distribute gradients to before sending their private gradients to the server, using secret sharing. After receiving gradient information from all users, the server aggregates the parameters using the FedAvg method. During this process, the server cannot deduce the actual gradient values ​​of the users, and the gradient information received by the server after aggregation is identical to the actual gradient information, thus protecting process privacy and existence privacy. By incorporating user-generated interaction values, the server can obtain the actual number of gradients when aggregating the model, ensuring recommendation accuracy. Furthermore, all information is transmitted via random parameter partitioning, which avoids introducing additional encryption computations. In summary, the proposed algorithm strikes a good balance between privacy protection, recommendation accuracy, and communication efficiency, achieving a satisfactory level in all three aspects.

[0111] The present invention has been described by way of example. Obviously, the specific implementation of the present invention is not limited to the above-described manner. Any non-substantial improvements made using the inventive concept and technical solution of the present invention, or the direct application of the inventive concept and technical solution of the present invention to other occasions without modification, are all within the protection scope of the present invention.

Claims

1. A recommender system based on improved federated matrix factorization, characterized in that, The system includes: a server, which communicates with n users, and the users store their local rating data; The server sends the latent factor vector of project j to all users. Update the request and average the received project gradients and user project interaction values ​​sent by each user to obtain the averaged project gradient value. This updates the latent factor vector of project j. ; User's latent factor vector based on project j The update request downloads the latent factor vector of project j from the server. ;Detecting users Is the project An evaluation was conducted; if the detection result is yes, then it is based on the user. For the project rating Calculate user gradients locally Update user latent factor vectors And calculate the project gradient locally. , project gradient and user project interaction values Randomly divided into The system will randomly send the divided project gradient fragments and interaction information fragments to users. Randomly selected One user; user Update local project gradients and user project interaction values and update the project gradient and user project interaction values Send to the server; The server receives project gradients sent by each user. and interactive information The FedAvg algorithm is used to average and aggregate the received gradients from the projects, resulting in the aggregated total gradient value. This updates the latent factor vector. ; Each user downloads the updated latent factor matrix of the project from the server. This allows us to obtain user ratings for each project and recommend the highest-rated project to the user. Use user project interaction values Indicates user Is the project An evaluation was conducted, if the user For the project The user project interaction value was evaluated. ,user No project The user project interaction value was evaluated. .

2. The recommender system based on improved federated matrix factorization as described in claim 1, characterized in that, The formula for calculating average aggregation is as follows: ； in, Indicates user Projects sent to the server gradient, Indicates user Project sent to the server User project interaction values.

3. The recommender system based on improved federated matrix factorization as described in claim 1, characterized in that, If user No project To evaluate, the project gradient is then determined. .

4. The recommender system based on improved federated matrix factorization as described in claim 1, characterized in that, After random partitioning Interactive information segment Satisfy the following formula: ， ； in, For users Retained interactive information fragments, The interaction information fragment is randomly selected from the k-th users.

5. The recommender system based on improved federated matrix factorization as described in claim 1, characterized in that, After random partitioning Gradient partitioning for each project Satisfy the following formula: ， ； in, For users Retained project gradient sharding, Gradient sharding for the project of the randomly selected k-th user.

6. The recommender system based on improved federated matrix factorization as described in claim 1, characterized in that, Updated user project interaction values The specific calculation formula is as follows: ； in, Indicates excluding users Other users in the recommendation system besides those in the user set, U is the user set that stores all users in the recommendation system. Indicates excluding users Other users, Indicates user For the project Interactive information, For interactive information Randomly divided into After distribution to users Interactive information fragments.

7. The recommender system based on improved federated matrix factorization as described in claim 1, characterized in that, Updated project gradient The specific calculation formula is as follows: ； in, Indicates user For the project The project gradient is calculated locally. For users Project gradient Randomly divided into After distribution to users Project gradient sharding.

Images