Personalized federal recommendation for privacy protection

Abstract

The invention discloses a privacy protection-oriented personalized federal recommendation system and method, and relates to the field of federal recommendation systems. According to the method, an article preference embedding vector is constructed for each client, and a global article embedding orthogonal mapping matrix and a double-layer perceptron used for generating an article embedding mixed factor matrix are trained, so that the global article embedding matrix and the article preference embedding vector of a user are fused; a mixed article embedding matrix which is more personalized and finer in granularity is constructed for the client, and the mixed article embedding matrix is input into a personalized recommendation prediction perceptron constructed for the client to obtain a recommendation result, so that the final recommendation effect is improved; when the client side uploads data to the central server, local differential privacy encryption is carried out; according to the technical scheme, negative sampling is carried out on each client side, the positive samples and the negative samples are mixed and then jointly used for training, the training effect is improved, meanwhile, the server cannot distinguish the historical interaction situation between the client sides and the articles according to the update data uploaded by the client sides, and user privacy is protected.

Description

Technical Field

[0001] The present invention relates to the field of federated recommendation systems, and in particular to a personalized federated recommendation system oriented to privacy protection. Background Art

[0002] With the rapid advancement of Internet technology, recommendation systems have been widely used in user modeling, aiming to solve the problem of information overload in many real-world fields such as e-commerce, short video recommendations, news push, and medical care. In order to improve the performance of recommendation systems, the system usually collects a large amount of user data, which inevitably contains user privacy information, such as user attributes, user behavior records, and user social relationships.

[0003] Although recommendation systems have achieved remarkable results in improving recommendation accuracy, the collection of user data poses many risks of privacy leakage. User data may be sold to third parties without the user's explicit authorization, or may be illegally stolen by malicious attackers. In addition, with the introduction of a series of privacy protection policies and the continuous strengthening of supervision, the practice of improving recommendation performance by obtaining user data is facing increasing challenges.

[0004] Researchers point out that the federated learning algorithm can effectively address the severe challenge of user privacy data leakage, and the algorithm is increasingly attracting widespread attention from academia and industry. Specifically, the recommendation system under the federated learning framework is committed to mining the joint distribution information contained in data sets stored on different devices, while strictly avoiding direct sharing of the original data on these devices, which greatly reduces the risk of data leakage. The system constructs a user data model based on a virtual global view through the mechanism of exchanging model parameters or intermediate results in the calculation process. On this basis, a distributed machine learning training model is further established to achieve the dual goals of protecting user privacy and improving model performance.

[0005] In the federated recommendation algorithm, models need to be built for users and items separately. To protect user privacy, user models are strictly retained locally to avoid communication with the central server; shared models such as item models need to be uploaded to the central server for federated aggregation, and then distributed by the server to each client participating in federated learning for the next round of training. However, the data distribution of different clients is often heterogeneous, that is, the local data of the clients participating in federated learning often do not obey independent and identical distribution, resulting in large differences in the item models trained by each client. After these item models are federated by the server, the unified global item model obtained may not be suitable for some clients, thereby damaging the overall recommendation effect of the recommendation system. Intuitively, different users often hold different views on the same item. Constructing a unified item model to meet the cognition of all users will inevitably lead to fuzzy and high-dimensional models; at the same time, different users may adopt different decision logics for the same item, which further leads to more fuzzy and more complex parameters for user decision models that meet the behavioral characteristics of multiple users. In terms of privacy protection, an honest but curious server may infer the historical interactions between the client and the item based on the item category corresponding to the item model uploaded by the client, thereby compromising user privacy; when the client communicates with the server, the information may be intercepted by a malicious third party, and the relevant information of the client may be inferred from it. Summary of the invention

[0006] In order to solve the technical problems in the above background, the present invention proposes a new federal recommendation method that combines privacy protection and personalization. The present invention constructs an item preference model for users that is independent of their decision logic, and realizes the effective fusion of the model with the global item model, thereby realizing the local personalization of the item model to improve the local recommendation effect; in addition, three shallow perceptrons are constructed for each client, which are used for item model localization, item model fusion and recommendation prediction, respectively, to realize the personalization of the client model. In order to improve the privacy protection capability, the present invention randomly adds negative samples to the training set of each user to confuse their item interaction history; when the client communicates with the central server, local differential privacy encryption technology is used to protect the communication content, thereby resisting inference attacks to a certain extent and enhancing the security of the system.

[0007] To achieve the above object, the present invention provides a privacy-preserving personalized federated recommendation method, which realizes the personalization of the item embedding matrix by fusing the global item embedding matrix from the central server with the item preference embedding vector trained by the client. The method comprises the following steps:

[0008] S1. The central server maintains a global item existence table to record the number of item types in the recommendation system; the central server initializes the model weights and the global item embedding matrix and distributes them to each client participating in the federated training;

[0009] S2. Each client obtains the initialized model weights and the global item embedding matrix from the central server, and initializes the weights of the global item embedding orthogonal mapping matrix, the item preference embedding vector, the two-layer perceptron for generating the item embedding mixing factor matrix, and the single-layer perceptron for user recommendation prediction;

[0010] S3. Each client takes all interacted items as positive samples based on the historical interactions with the items, and randomly samples items that have not been interacted with as negative samples. The positive samples and negative samples are mixed in a certain ratio to obtain a training set; the two most recently interacted positive samples are separately reserved and mixed with a certain ratio of negative samples to obtain a test set and a validation set;

[0011] S4. Each client uses the local training set data to train the recommendation model in parallel:

[0012] S401. Each client selects the global embedding vector of the corresponding item from the global item embedding matrix according to the item category included in the training set, concatenates the global item embedding matrix that matches the item category of the client's training set, and performs an orthogonal transformation on it through the global item embedding orthogonal mapping matrix to obtain a migration item embedding matrix;

[0013] S402. Broadcast the user's item preference embedding vector to the same dimension as the migration item embedding matrix to obtain the user's item preference embedding matrix; subtract the migration item embedding matrix from the user's item preference embedding matrix and calculate the absolute value to obtain the absolute value matrix of the item-preference embedding matrix difference;

[0014] S403. Input the absolute value matrix of the item-preference embedding matrix difference into a two-layer perceptron to generate an item embedding mixed factor matrix;

[0015] S404. The item embedding mixed factor matrix and the 1-item embedding mixed factor matrix are used as weights of the global item embedding matrix and the user's item preference embedding matrix, respectively, and a weighted average is obtained as the mixed item embedding matrix;

[0016] S405. Input the mixed item embedding matrix into the user recommendation prediction single-layer perceptron to obtain the predicted score of each item by the client; calculate the loss value, and use small batch stochastic gradient descent to update the model parameters; each client trains the local recommendation system model in parallel;

[0017] S5. After several epochs of local training, each client uses the updated model parameters to make predictions on the validation set and test set, calculates various evaluation indicators, and retains a record containing the training round ID, the evaluation indicator value on the validation set, the evaluation indicator value on the test set, and the local model parameters. The local model includes the updated global item embedding matrix, the user's item preference embedding vector, the global item embedding orthogonal mapping matrix, the two-layer perceptron for generating the item embedding mixing factor matrix, and the user recommendation prediction single-layer perceptron; the client performs local differential privacy encryption on the updated global item embedding matrix and uploads it to the central server together with the client's evaluation indicator calculation values ​​on the validation set and test set;

[0018] S6. After receiving the global item embedding matrix uploaded by each client and encrypted with local differential privacy, the central server aggregates the global embedding vectors of each item using the FedAvg algorithm according to the global item existence table and item ID to obtain a new global item embedding vector; the server calculates the average value of each evaluation indicator on the validation set and the test set for all clients respectively; if the average value of the evaluation indicator on the validation set is better than the best average value of the corresponding indicator in the previous training, the best average value of the indicator is updated and the record is saved; the aggregated global item embedding matrix is ​​distributed to each client, and if the best average value of the evaluation indicator is updated in this round, the client is notified to retain the local model parameters of the current round;

[0019] S7. The client verifies the message content sent by the central server. If it contains a signal to retain the model parameters of the current round, the local model parameters of the current round are saved. Otherwise, the model parameters are directly updated and replaced without saving. The client uses the received aggregated global item embedding matrix to continue the next round of training.

[0020] S8. Take S4 to S7 as a cycle, and periodically cycle S4 to S7 until the number of iteration cycles set at the beginning. The central server determines the final global item embedding matrix according to the training round corresponding to the best average value of the evaluation index on the recorded validation set, and measures the recommendation effect with the average value of the evaluation index on the test set of the corresponding round; each client uses the trained global item embedding matrix and local model to output its own final recommendation result.

[0021] Preferably, a federated learning architecture is adopted, including a single central server and multiple clients. Each client does not need to upload original local data to the server, nor does it need to expose the client model to the server. It only uploads the updated object embedding matrix to the server.

[0022] Preferably, the global item embedding matrix updated locally by the client is uploaded to the central server after being locally differentially privately encrypted.

[0023] Preferably, each client needs to train a set of local models, which include item preference embedding vectors, orthogonal mapping matrices, two-layer perceptrons, and single-layer perceptrons for user recommendation prediction;

[0024] The global item embedding matrix is ​​orthogonally transformed by using an orthogonal mapping matrix to obtain a migration item embedding matrix. The absolute value matrix of the difference between the migration item embedding matrix and the user item preference embedding vector is calculated as the input of the two-layer perceptron, and a mixing factor matrix is ​​output. The item preference embedding vector and the global item embedding matrix are fused by using the mixing factor matrix to obtain a mixed item embedding matrix, thereby realizing local personalization of the item embedding matrix.

[0025] Preferably, the inner product of the user's item preference embedding vector and the mixed item embedding vector is not used as the recommendation prediction score, but the mixed item embedding vector is input into the recommendation prediction single-layer perceptron constructed by the client to obtain the recommendation prediction score.

[0026] Preferably, each client trains three shallow perceptrons, which are used to personalize the global item embedding vector, generate the item embedding mixing factor matrix, and make recommendation predictions.

[0027] Compared with the prior art, the present invention has the following beneficial effects:

[0028] The present invention fuses the global item embedding vector with the user's item preference embedding vector by training a global item embedding orthogonal mapping matrix and a two-layer perceptron for generating an item embedding mixed factor matrix, thereby constructing a more personalized and fine-grained mixed item embedding matrix for the client, thereby improving the final recommendation effect; negative sampling is performed for each client, and the positive samples and negative samples are mixed and used together for training, which improves the training effect while preventing the server from identifying the historical interaction between the client and the item based on the updated data uploaded by the client, thereby protecting the user's privacy; a local differential privacy method is used, which is applied to data communication between a central server and a client, thereby reducing the risk of inference attacks, enhancing data security, and protecting data privacy to a certain extent; the model adopts a total of three shallow perceptrons, which are respectively used to personalize the global item embedding vector, generate the item embedding mixed factor matrix, and make recommendation predictions, so that the model is relatively lightweight and more convenient to be deployed on smart terminals with limited performance. BRIEF DESCRIPTION OF THE DRAWINGS

[0029] In order to more clearly illustrate the technical solution of the present invention, the following briefly introduces the drawings required for use in the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

[0030] Figure 1 It is a schematic diagram of the system structure of the present invention;

[0031] Figure 2 is a training flow chart of the present invention;

[0032] Figure 3 A structural diagram of the client local training model of the present invention. DETAILED DESCRIPTION

[0033] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0034] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, the present invention is further described in detail below with reference to the accompanying drawings and specific embodiments.

[0035] Before describing, the characters appearing in the examples are explained:

[0036] The vectors and matrices with subscript "0" in the characters come from the central server;

[0037] The vectors and matrices with subscripts "0, u" in the characters come from the client.

[0038] Embodiment 1

[0039] This embodiment provides a privacy-preserving personalized federated recommendation method, which personalizes the item embedding matrix by fusing the global item embedding matrix from the central server with the item preference embedding vector trained on the client.

[0040] The above method is to use Figure 1 The recommendation system shown in the figure is implemented, and the system includes: a central server and several clients. This embodiment includes the following three types of entities:

[0041] (1) Central server: The central server is used to initialize the model structure and model parameters and maintain a global item existence table. It is responsible for distributing the initialized model structure and parameters to the clients participating in the federated training, collecting the item embedding matrices uploaded by the clients, and aggregating the item embedding vectors uploaded by the clients according to the item ID.

[0042] (2) Items: This recommendation system constructs a global item embedding vector for each item, which is shared by all clients. At the same time, each client maintains an item preference embedding vector locally. When recommending items to a client, the global item embedding vector and the user's item preference embedding vector are combined for recommendation.

[0043] (3) Client: In this recommendation system, each client corresponds to a user, and three shallow perceptrons are used to personalize the global item embedding vector, generate the item embedding mixed factor matrix, and make recommendation predictions. The client maintains an item preference embedding vector, a global item embedding orthogonal mapping matrix, a two-layer perceptron for generating the item embedding mixed factor matrix, and a single-layer perceptron for user recommendation prediction. These models are personalized models of the client and are always kept locally and do not participate in federated aggregation.

[0044] The workflow of this system includes the following three stages:

[0045] (1) Model initialization and dataset construction: The central server constructs an item existence table based on the existence of items in the recommendation system to record the number of item types in the recommendation system. Each item constitutes a record, and each record contains information such as the item ID. The central server initializes the item embedding matrix and client model parameters. Each client constructs a local dataset of the form (client ID, item ID, interaction (0 or 1), timestamp) based on its historical interactions with the item, and combines negative sampling to construct training, test, and validation sets.

[0046] (2) Training of the model and item embedding vectors: The central server distributes the initialized client model parameters and the global item embedding matrix to the clients participating in federated learning. The clients train the model using local data. After the training is completed, the updated item embedding matrix is ​​encrypted locally using differential privacy and uploaded to the central server, which aggregates it and distributes it to the clients again for the next round of training until the predetermined number of training times is reached.

[0047] (3) Completing the recommendation: Each client uses the trained model parameters to fuse the global item embedding vector with the user’s item preference embedding vector, inputs the trained prediction model, and obtains the final recommendation result.

[0048] Embodiment 2

[0049] The following will describe in detail how the present invention solves technical problems in real life in conjunction with this embodiment.

[0050] S1. The central server maintains a global item existence table X, which is used to record the number of item types in the recommendation system. The purpose of maintaining a global item existence table X is that, firstly, the item interaction lists of some clients may overlap to varying degrees, and X is prepared for the subsequent aggregation of the updated global item embedding vectors uploaded by different clients by the central server; secondly, when performing random negative sampling, each client needs to know which specific items it has not interacted with through the global item existence table. The central server initializes the global item embedding matrix Global item embedding orthogonal mapping matrix A two-layer perceptron (weights are ) and user recommendation prediction single-layer perceptron And distributed to each client participating in federated training.

[0051] S2. Each client obtains the initialized model weights from the server and initializes the corresponding parts of the local model: global item embedding orthogonal mapping matrix Two-layer perceptron for generating item embedding mixing factor matrix and user recommendation prediction single-layer perceptron Get the global item embedding matrix from the server Initialize a user-item preference embedding vector

[0052] S3. Each client lists X based on its historical interactions with the item u The global items are stored in Table X. All items that have interacted with each other are taken as positive samples, and items that have never interacted with each other are randomly sampled as negative samples. The ratio of positive samples to negative samples is 1:4. The positive samples and negative samples are mixed according to this ratio to obtain the training set; the two positive samples with the most recent interaction time are set aside, and 99 negative samples are randomly sampled to construct the test set and the validation set.

[0053] S4. Each client uses the local training set data to train the recommendation model in parallel, such as Figure 2 and Figure 3 As shown:

[0054] S401. Each client selects the global embedding vector of the corresponding item from the global item embedding matrix according to the item categories included in the training set, splices the global item embedding matrix that matches the item categories in the client's training set, and performs an orthogonal transformation on it through the global item embedding orthogonal mapping matrix to obtain a migration item embedding matrix.

[0055] Specifically, the global item embedding matrix from the central server is the average of the item embedding matrices uploaded by all clients, so it may not necessarily match the current client's personalized view of the item. To this end, this embodiment first learns a mapping relationship from a global item embedding matrix to the client's personalized view to achieve preliminary personalization of the global item embedding matrix; a single-layer perceptron with an orthogonal weight matrix and a bias term of 0 is used The global item embedding matrix is ​​orthogonally transformed. The inner product of the global item embedding vector remains unchanged before and after the orthogonal mapping. While maintaining the similarity of the item embedding vector before and after the transformation, it introduces personalization for the current client. Each client selects the item from the global item embedding matrix according to the item category included in the training set. Select the global item embedding vector i of the corresponding item global , concatenate the global item embedding matrix that matches the item types in the client training set Will Use the global item embedding orthogonal mapping matrix Perform orthogonal mapping to obtain the migration item embedding matrix As shown below:

[0056]

[0057] (4b) Broadcast the user's item preference embedding vector to the same dimension as the migration item embedding matrix to obtain the user's item preference embedding matrix; subtract the migration item embedding matrix from the user's item preference embedding matrix and calculate the absolute value to obtain the absolute value matrix of the item-preference embedding matrix difference.

[0058] Specifically, we further embed the migration item matrix For local personalization, this embodiment will later train an item embedding mixing factor matrix to embed the global item embedding matrix representing the average item preference of the client and the item preference embedding vector representing the individual item preferences of a specific client Weighted averaging is used to extract fine-grained, personalized mixed item embedding matrices.

[0059] First, Broadcast to The same dimension, get the item preference embedding matrix

[0060] Embed the migration items into the matrix and the item preference embedding matrix Subtract and calculate the absolute value to get the absolute value matrix Δ of the item-preference embedding matrix difference 0，u , as shown below:

[0061]

[0062] Among them, Δ 0，u Each element represents and The similarity of elements at corresponding positions, the smaller the value, the higher the similarity.

[0063] (4c) The absolute value matrix of the item-preference embedding matrix differences is input into a two-layer perceptron to generate the item embedding mixing factor matrix.

[0064] Specifically, in order to preserve the original information contained in the global item embedding matrix as much as possible, this embodiment selects the global item embedding matrix Instead of migrating the item embedding matrix and the user's item preference embedding matrix To merge; at the same element position, and may have different contribution levels; therefore we train a two-layer perceptron to learn and Mixing ratio;

[0065] The absolute value matrix Δ of the item-preference embedding matrix difference 0，u Input a two-layer perceptron The activation functions use ReLU and Sigmoid functions to generate the item embedding mixed factor matrix As shown below:

[0066]

[0067] (4d) Embed items into a mixing factor matrix and As global item embedding matrices and the user's item preference embedding matrix The weights of the mixed items are used to obtain the weighted average value. As shown below:

[0068]

[0069] (4e) The mixed item embedding matrix is ​​input into the user recommendation prediction single-layer perceptron to obtain the client's prediction score for each item; the loss value is calculated and the model parameters are updated using mini-batch stochastic gradient descent; each client trains the local recommendation system model in parallel.

[0070] Specifically, from the mixed item embedding matrix Get the mixed item embedding vector of item i Will Input user recommendation prediction single layer perceptron After being activated by the Sigmoid activation function, the predicted score of user u for item i is obtained. As shown below:

[0071]

[0072] The binary cross entropy loss is calculated as follows:

[0073]

[0074] Among them, Item u Represents the set of items that client u has interacted with; Item′ u represents the set of items that client u has not interacted with; Represents the prediction score of user u for negative sample i′.

[0075] Mini-batch stochastic gradient descent is used to update model parameters, and each client trains the local recommendation system model in parallel.

[0076] S5. After completing several epochs of local training, each client uses the updated model parameters to make predictions on the validation set and the test set, calculates various evaluation indicators, and retains a record of the form (training round ID, evaluation indicator value on the validation set, evaluation indicator value on the test set, local model parameters), where the local model includes the updated global item embedding matrix, the user's item preference embedding vector, the global item embedding orthogonal mapping matrix, a two-layer perceptron for generating an item embedding mixing factor matrix, and a single-layer perceptron for user recommendation prediction; the client performs local differential privacy encryption on the updated global item embedding matrix and uploads it to the central server together with the client's calculated evaluation indicator values ​​on the validation set and the test set.

[0077] Specifically, this embodiment mainly evaluates the recommendation quality from two aspects: Hit Ratio (HR) and Normalized Discounted Cumulative Gain (NDCG). HR is used to indicate whether the positive sample of the client is hit in the prediction result list, that is, whether the item the user wants is recommended, emphasizing the "accuracy" of the recommendation; NDCG is used to indicate whether the positive sample of the client is at the front in the prediction result list, that is, whether the item the user wants is recommended first, emphasizing the "priority" of the recommendation. The calculation formulas are as follows:

[0078]

[0079] Among them, |U t| represents the number of clients, hits(u) = 1 indicates that the positive samples in the client's validation set / test set appear in the first K of the recommendation list, otherwise, hits(u) = 0; p u Represents the position of the positive sample in the validation set / test set of client u in the recommended list. When the predicted list does not contain the positive sample, p u →∞. In this embodiment, K=10 is set, that is, only the first 10 recommended items are considered.

[0080] After several epochs of local training, each client uses the updated model parameters to make predictions on the validation set and test set, calculates various evaluation indicators, and retains a (training round ID round , the evaluation index value on the validation set (NR v ,NDCG v ), the evaluation index value on the test set (HR t ,NDCG t ), local model parameters), where the local model includes the updated global item embedding matrix User's item preference embedding vector Global item embedding orthogonal mapping matrix Two-layer object embedding mixed factor generation perceptron and user recommendation prediction single-layer perceptron The client embeds the updated global item into the matrix Perform local differential privacy encryption and upload it to the central server together with the client's evaluation index values ​​on the validation set and test set:

[0081]

[0082] In the formula, La(0,λ) represents Laplace noise with mean 0 and noise intensity λ. The addition of Laplace differential privacy noise reduces the risk of malicious third-party inference attacks.

[0083] S6. After receiving the global item embedding matrix uploaded by each client and encrypted with local differential privacy, the central server After that, according to the global item existence table X and item ID, the FedAvg algorithm is used to aggregate the global embedding vectors of each item to obtain a new global item embedding vector, as shown in the following formula:

[0084]

[0085] Among them, |U i | represents the number of clients that uploaded the global item embedding vector for item i.

[0086] The global item embedding vector after aggregating each item Concatenate into a global item embedding matrix The server calculates the average value of each evaluation indicator of all clients on the validation set and the test set respectively; if the average value of the evaluation indicator on the validation set is better than the best average value of the corresponding indicator in the previous training, the best average value of the indicator is updated and a record of the form (training round ID round , the best average value of the evaluation index on the validation set Average value of evaluation index on the test set Global item embedding matrix ) records; embed the aggregated global items into the matrix Distribute to each client for the next round of training; if the best average value of the evaluation indicator is updated in this round, the client also needs to be notified to retain the local model parameters of the current round.

[0087] S7. The client verifies the message content sent by the central server. If it contains a signal to retain the model parameters of the current round, the local model parameters of the current round are saved. Otherwise, there is no need to save and the model parameters are directly updated and replaced. The client uses the received aggregated global item embedding matrix Continue to the next round of training.

[0088] S8. Take S4 to S7 as a cycle, and periodically cycle S4 to S7 until the number of iterations set at the beginning. The central server records the training round ID corresponding to the best average value of the evaluation index on the validation set. round , determine the final global item embedding matrix The recommendation effect of this system is measured by the average value of the evaluation indicators on the corresponding round test set.

[0089] Finally, the trained prediction model is used to complete the recommendation.

[0090] Embodiment 2

[0091] This embodiment provides an experimental process to demonstrate the superiority of the present invention.

[0092] Table 1 compares the recommendation results of this embodiment with those of FedMF, FedRECON, FedNCF, and PFedRec on four datasets: MovieLens-100k, MovieLens-1M, Lastfm-2K, and Foursquare. Among them, FedMF was proposed by Chai et al. in 2020. It is a federated learning version of the matrix decomposition (MF) recommendation algorithm and a classic federated recommendation system (FedRec) method. It updates the user embedding vector locally and aggregates the item embedding vector gradient globally. In 2021, Singhal et al. proposed the FedRECON federated recommendation algorithm. Between each two rounds of iterations, this method does not inherit the user embedding vector of the previous round, but completely discards the user embedding vector and retrains it using a reconstruction mechanism. FedNCF was proposed by Parifanis et al. in 2022 as a federated learning version of neural collaborative filtering (NCF). Specifically, each user will update the user embedding vector locally, and upload the item embedding vector and scoring function to the server for global update. In 2023, Zhang et al. proposed the PFedRec algorithm, which adopts a new "dual personalization" mechanism to efficiently extract personalized features of users and items. The four data sets used in this embodiment are all real data from the real world and are often used in the field of recommendation systems. Among them, the MovieLens dataset contains movie ratings from users of the MovieLens website, and each user has rated at least 20 movies; the Lastfm-2k dataset originates from Last.fm, one of the world's largest music platforms, and records the interaction between users and artists' works; the Foursquare dataset is obtained through the public API of the Foursquare application, and contains a large number of users' check-in records, social relationships, and venue ratings.

[0093] Table 1

[0094]

[0095] It can be seen from Table 1 that the improved personalized federated recommendation system in this implementation case improves the recommendation performance.

[0096] Table 2 shows the ablation experiment results of the relevant modules in this embodiment on the MovieLens-100k dataset, where NoOrth refers to canceling the global object embedding orthogonal mapping matrix. Orthogonal properties, use a normal linear mapping matrix instead; NoOrthMapping means canceling the global item embedding orthogonal mapping matrix Directly find the absolute value vector of the difference between the global item embedding vector and the user preference embedding vector; OneUserEmbedding means canceling the single-layer perceptron for user recommendation prediction The inner product of the user-item preference embedding vector and the mixed-item embedding vector is directly used as the prediction result.

[0097] Table 2

[0098] method HR@10 NDCG@10 NoOrth 0.7211 0.4485 NoOrthMapping 0.7190 0.4271 OneUserEmbedding 0.7041 0.4219 The present invention 0.7275 0.4497 ,

[0099] Table 2 shows the effectiveness of embedding the global item orthogonal mapping matrix and separating the user's item preference and decision logic in improving the recommendation effect in this implementation case.

[0100] Table 3 shows the model performance of this embodiment by adding local differential privacy on the MovieLens-100k dataset.

[0101] Table 3

[0102] Noise intensity λ=0 λ=0.2 λ=0.4 λ=0.6 λ=0.8 λ=1.0 HR@10 0.7275 0.7237 0.7116 0.7063 0.6935 0.6871 NDCG@10 0.4497 0.4305 0.4219 0.4174 0.4132 0.4094 ,

[0103] It can be seen from Table 3 that as the noise intensity increases, the performance of the model in terms of HR@10 and NDCG@10 will gradually decline; but even if the noise of λ=0.2 is added, the model performance of this embodiment is still better than all the comparison algorithms.

[0104] The embodiments described above are only descriptions of the preferred embodiments of the present invention and are not intended to limit the scope of the present invention. Without departing from the design spirit of the present invention, various modifications and improvements made to the technical solutions of the present invention by ordinary technicians in this field should all fall within the protection scope determined by the claims of the present invention.

Claims

1. A personalized federated recommendation method for privacy protection, which realizes the personalization of the item embedding matrix by fusing the global item embedding matrix from the central server with the item preference embedding vector trained by the client, characterized in that: The method comprises the following steps: S1. The central server maintains a global item existence table to record the number of item types in the recommendation system; the central server initializes the model weights and the global item embedding matrix and distributes them to each client participating in the federated training; S2. Each client obtains the initialized model weights and the global item embedding matrix from the central server, and initializes the weights of the global item embedding orthogonal mapping matrix, the item preference embedding vector, the two-layer perceptron for generating the item embedding mixing factor matrix, and the single-layer perceptron for user recommendation prediction; S3. Each client takes all interacted items as positive samples based on the historical interactions with the items, and randomly samples items that have not been interacted with as negative samples. The positive samples and negative samples are mixed in a certain ratio to obtain a training set; the two most recently interacted positive samples are separately reserved and mixed with a certain ratio of negative samples to obtain a test set and a validation set; S4. Each client uses the local training set data to train the recommendation model in parallel: S401. Each client selects the global embedding vector of the corresponding item from the global item embedding matrix according to the item category included in the training set, splices it into a global item embedding matrix that matches the item category of the client's training set, and performs an orthogonal transformation on it through the global item embedding orthogonal mapping matrix to obtain a migration item embedding matrix; S402. Broadcast the user's item preference embedding vector to the same dimension as the migration item embedding matrix to obtain the user's item preference embedding matrix; Subtract the migration item embedding matrix from the user's item preference embedding matrix, and calculate the absolute value to obtain the absolute value matrix of the item-preference embedding matrix difference; S403. Input the absolute value matrix of the item-preference embedding matrix difference into a two-layer perceptron to generate an item embedding mixed factor matrix; S404. The item embedding mixed factor matrix and the 1-item embedding mixed factor matrix are used as weights of the global item embedding matrix and the user's item preference embedding matrix, respectively, and a weighted average is obtained as the mixed item embedding matrix; S405. Input the mixed item embedding matrix into the user recommendation prediction single-layer perceptron to obtain the predicted score of each item by the client; calculate the loss value, and use small batch stochastic gradient descent to update the model parameters; each client trains the local recommendation system model in parallel; S5. After several epochs of local training, each client uses the updated model parameters to make predictions on the validation set and test set, calculates various evaluation indicators, and retains a record containing the training round ID, the evaluation indicator value on the validation set, the evaluation indicator value on the test set, and the local model parameters. The local model includes the updated global item embedding matrix, the user's item preference embedding vector, the global item embedding orthogonal mapping matrix, the two-layer perceptron for generating the item embedding mixing factor matrix, and the user recommendation prediction single-layer perceptron; the client performs local differential privacy encryption on the updated global item embedding matrix and uploads it to the central server together with the client's evaluation indicator calculation values ​​on the validation set and test set; S6. After receiving the global item embedding matrix uploaded by each client and encrypted with local differential privacy, the central server aggregates the global embedding vectors of each item using the FedAvg algorithm according to the global item existence table and item ID to obtain a new global item embedding vector; the server calculates the average value of each evaluation indicator on the validation set and the test set for all clients respectively; if the average value of the evaluation indicator on the validation set is better than the best average value of the corresponding indicator in the previous training, the best average value of the indicator is updated and the record is saved; the aggregated global item embedding matrix is ​​distributed to each client, and if the best average value of the evaluation indicator is updated in this round, the client is notified to retain the local model parameters of the current round; S7. The client verifies the message content sent by the central server. If it contains a signal to retain the model parameters of the current round, the local model parameters of the current round are saved. Otherwise, the model parameters are directly updated and replaced without saving. The client uses the received aggregated global item embedding matrix to continue the next round of training. S8. Take S4 to S7 as a cycle, and periodically cycle S4 to S7 until the number of iteration cycles set at the beginning. The central server determines the final global item embedding matrix according to the training round corresponding to the best average value of the evaluation index on the recorded validation set, and measures the recommendation effect with the average value of the evaluation index on the test set of the corresponding round; each client uses the trained global item embedding matrix and local model to output its own final recommendation result.

2. A personalized federated recommendation system for privacy protection, the system being used to implement the method of claim 1, characterized in that: It adopts a federated learning architecture, including a unique central server and multiple clients. Each client does not need to upload original local data to the server, nor does it need to expose the client model to the server. It only uploads the updated item embedding matrix to the server.

3. The privacy-preserving personalized federated recommendation system according to claim 2, characterized in that: The global item embedding matrix updated locally on the client is uploaded to the central server after being locally differentially privately encrypted.

4. The privacy-preserving personalized federated recommendation system according to claim 2, characterized in that: Each client needs to train a set of local models, including item preference embedding vectors, orthogonal mapping matrices, two-layer perceptrons, and single-layer perceptrons for user recommendation predictions; Use the orthogonal mapping matrix to perform an orthogonal transformation on the global item embedding matrix to obtain the migration item embedding matrix; The absolute value matrix of the difference between the migration item embedding matrix and the user item preference embedding vector is calculated as the input of the two-layer perceptron, and a mixing factor matrix is ​​output. The item preference embedding vector and the global item embedding matrix are fused using the mixing factor matrix to obtain a mixed item embedding matrix, thereby realizing local personalization of the item embedding matrix.

5. The privacy-preserving personalized federated recommendation system according to claim 2, characterized in that: Instead of using the inner product of the user's item preference embedding vector and the mixed item embedding vector as the recommendation prediction score, the mixed item embedding vector is input into the recommendation prediction single-layer perceptron built on the client to obtain the recommendation prediction score.

6. The privacy-preserving personalized federated recommendation system according to claim 2, characterized in that: Each client trains three shallow perceptrons, which are used to personalize the global item embedding vector, generate the item embedding mixing factor matrix, and make recommendation predictions.

Images