A privacy-preserving personalized federated recommendation

Abstract

The present invention discloses a personalized federated recommendation system and method for privacy protection, relating to the field of federated recommendation systems. The present invention constructs an item preference embedding vector for each client, trains a global item embedding orthogonal mapping matrix, and a two-layer perceptron for generating an item embedding mixing factor matrix. The matrix is ​​used to fuse the global item embedding matrix with the user's item preference embedding vector, constructing a more personalized and fine-grained mixed item embedding matrix for the client. This matrix is ​​then input into the personalized recommendation prediction perceptron constructed for the client to obtain recommendation results, thereby improving the final recommendation effect. When the client uploads data to a central server, local differential privacy encryption is performed. Negative sampling is performed for each client, mixing positive and negative samples for training. This improves the training effect while preventing the server from identifying the client's historical interactions with items based on the updated data uploaded by the client, thereby protecting user privacy.

Description

Technical Field

[0001] The present invention relates to the field of federated recommendation systems, and in particular to a personalized federated recommendation system oriented to privacy protection. Background Art

[0002] With the rapid advancement of internet technology, recommender systems have been widely used for user modeling, aiming to address information overload in a variety of real-world fields, including e-commerce, short video recommendations, news push, and healthcare. To improve the performance of recommender systems, they typically collect large amounts of user data, which inevitably contains private user information such as user attributes, behavior records, and social relationships.

[0003] While recommendation systems have achieved significant success in improving recommendation accuracy, the collection of user data carries numerous potential privacy risks. User data may be sold to third parties without explicit authorization or illegally stolen by malicious attackers. Furthermore, with the introduction of a series of privacy protection policies and the continued strengthening of regulatory oversight, improving recommendation performance through the acquisition of user data is facing increasing challenges.

[0004] Researchers point out that federated learning algorithms can effectively address the severe challenge of user privacy data leakage and are increasingly attracting widespread attention from academia and industry. Specifically, recommendation systems within the federated learning framework strive to mine the joint distribution information contained in data sets stored on different devices while strictly avoiding direct sharing of the raw data on these devices, which greatly reduces the risk of data leakage. By exchanging model parameters or intermediate results during the computation process, the system constructs a user data model based on a virtual global view. On this basis, a distributed machine learning training model is further established, achieving the dual goals of protecting user privacy and improving model performance.

[0005] In federated recommendation algorithms, models must be constructed separately for users and items. To protect user privacy, user models are strictly retained locally, avoiding communication with a central server. Shared models, such as item models, are uploaded to a central server for federated aggregation, which is then distributed to all participating clients for the next round of training. However, data distribution across clients is often heterogeneous. That is, the local data of participating clients often does not follow independent and identical distribution (IID). This results in significant differences in the item models trained across clients. After these item models are aggregated by the server, the resulting unified global item model may not be applicable to certain clients, thus compromising the overall effectiveness of the recommendation system. Intuitively, different users often hold different perspectives on the same item. Constructing a unified item model that matches the perceptions of all users inevitably results in a fuzzy and high-dimensional model. Furthermore, different users may employ different decision-making logic for the same item, further leading to even more fuzziness and complex parameters in user decision models that reflect the behavioral characteristics of multiple users. In terms of privacy protection, an honest but curious server may infer the client's historical interactions with the item based on the item category corresponding to the item model uploaded by the client, thereby compromising user privacy; when the client communicates with the server, the information may be intercepted by a malicious third party, and relevant information about the client may be inferred from it. Summary of the Invention

[0006] To solve the technical problems in the above background, the present invention proposes a new federated recommendation method that combines privacy protection and personalization. The present invention constructs an item preference model for users that is independent of their decision-making logic, and realizes the effective fusion of this model with the global item model, thereby realizing local personalization of the item model to improve the local recommendation effect; in addition, three shallow perceptrons are constructed for each client, which are used for item model localization, item model fusion and recommendation prediction respectively, thereby realizing the personalization of the client model. In order to improve the privacy protection capability, the present invention randomly adds negative samples to the training set of each user to confuse their item interaction history; when the client communicates with the central server, local differential privacy encryption technology is used to protect the communication content, thereby resisting inference attacks to a certain extent and enhancing the security of the system.

[0007] To achieve the above objectives, the present invention provides a privacy-preserving personalized federated recommendation method. The method personalizes the item embedding matrix by fusing the global item embedding matrix from a central server with the item preference embedding vectors trained on the client. The method comprises the following steps:

[0008] S1. The central server maintains a global item table to record the number of item types in the recommendation system. The central server initializes the model weights and the global item embedding matrix and distributes them to all clients participating in federated training.

[0009] S2. Each client obtains the initialized model weights and global item embedding matrix from the central server, and initializes the weights of the global item embedding orthogonal mapping matrix, the item preference embedding vector, the two-layer perceptron used to generate the item embedding mixing factor matrix, and the single-layer perceptron for user recommendation prediction.

[0010] S3. Based on their historical interactions with items, each client selects all interacted items as positive samples and randomly samples items they have not interacted with as negative samples. These positive and negative samples are mixed in a certain ratio to form a training set. The two most recently interacted positive samples are set aside and mixed with a certain ratio of negative samples to form a test set and a validation set.

[0011] S4. Each client uses the local training set data to train the recommendation model in parallel:

[0012] S401. Each client selects the global embedding vector of the corresponding item from the global item embedding matrix based on the item category in the training set, concatenates the global item embedding matrices that match the item categories in the client's training set, and performs an orthogonal transformation on them using the global item embedding orthogonal mapping matrix to obtain a transferred item embedding matrix.

[0013] S402. Broadcast the user's item preference embedding vector to the same dimension as the transferred item embedding matrix to obtain the user's item preference embedding matrix. Subtract the transferred item embedding matrix from the user's item preference embedding matrix and calculate the absolute value to obtain the absolute value matrix of the item-preference embedding matrix difference.

[0014] S403. Input the absolute value matrix of the item-preference embedding matrix difference into a two-layer perceptron to generate an item embedding mixing factor matrix;

[0015] S404. Use the item embedding mixing factor matrix and the 1-item embedding mixing factor matrix as weights for the global item embedding matrix and the user's item preference embedding matrix, respectively, and calculate the weighted average to obtain the mixed item embedding matrix.

[0016] S405. Input the mixed item embedding matrix into the user recommendation prediction single-layer perceptron to obtain the client's predicted score for each item; calculate the loss value and update the model parameters using mini-batch stochastic gradient descent; and train the local recommendation system model in parallel on each client.

[0017] S5. After completing several epochs of local training, each client uses the updated model parameters to perform predictions on the validation and test sets, calculate various evaluation metrics, and retain a record containing the training epoch ID, the evaluation metric values ​​on the validation and test sets, and the local model parameters. The local model includes the updated global item embedding matrix, the user's item preference embedding vectors, the global item embedding orthogonal mapping matrix, a two-layer perceptron used to generate the item embedding mixing factor matrix, and a single-layer perceptron for user recommendation prediction. The client locally differentially encrypts the updated global item embedding matrix and uploads it to the central server along with the client's calculated evaluation metric values ​​on the validation and test sets.

[0018] S6. After receiving the global item embedding matrix uploaded by each client and encrypted using local differential privacy, the central server aggregates the global embedding vectors of each item using the FedAvg algorithm based on the global item presence table and item ID to obtain a new global item embedding vector. The server calculates the average value of each evaluation metric for all clients on the validation set and test set. If the average value of an evaluation metric on the validation set is better than the best average value of the corresponding metric in the previous training, the best average value of that metric is updated and saved. The aggregated global item embedding matrix is ​​distributed to each client. If the best average value of the evaluation metric is updated in this round, the client is notified to retain the local model parameters of the current round.

[0019] S7. The client verifies the message sent by the central server. If it contains a signal to retain the model parameters for the current round, the client saves the local model parameters for the current round. Otherwise, the client directly updates or replaces the model parameters without saving. The client then uses the received aggregated global item embedding matrix to proceed to the next round of training.

[0020] S8. Take S4 to S7 as one cycle, and periodically cycle S4 to S7 until the number of iterations set at the beginning. The central server determines the final global item embedding matrix based on the training round corresponding to the best average value of the evaluation index on the recorded validation set, and measures the recommendation effect with the average value of the evaluation index on the test set of the corresponding round; each client uses the trained global item embedding matrix and local model to output its own final recommendation result.

[0021] Preferably, a federated learning architecture is adopted, including a unique central server and multiple clients. Each client does not need to upload original local data to the server, nor does it need to expose the client model to the server. It only uploads the updated item embedding matrix to the server.

[0022] Preferably, the global item embedding matrix updated locally by the client is uploaded to the central server after being locally differentially privately encrypted.

[0023] Preferably, each client needs to train a set of local models, which include item preference embedding vectors, orthogonal mapping matrices, two-layer perceptrons, and single-layer perceptrons for user recommendation prediction.

[0024] An orthogonal mapping matrix is ​​used to perform an orthogonal transformation on the global item embedding matrix to obtain a migration item embedding matrix. The absolute value matrix of the difference between the migration item embedding matrix and the user item preference embedding vector is calculated as the input of a two-layer perceptron, and a mixing factor matrix is ​​output. The item preference embedding vector and the global item embedding matrix are fused using the mixing factor matrix to obtain a mixed item embedding matrix, thereby realizing local personalization of the item embedding matrix.

[0025] Preferably, the inner product of the user's item preference embedding vector and the mixed item embedding vector is not used as the recommendation prediction score, but the mixed item embedding vector is input into the recommendation prediction single-layer perceptron constructed by the client to obtain the recommendation prediction score.

[0026] Preferably, each client trains three shallow perceptrons, which are used to personalize the global item embedding vector, generate the item embedding mixing factor matrix, and make recommendation predictions.

[0027] Compared with the prior art, the present invention has the following beneficial effects:

[0028] The present invention trains a global item embedding orthogonal mapping matrix and a two-layer perceptron for generating an item embedding mixing factor matrix, fuses the global item embedding vector with the user's item preference embedding vector, and constructs a more personalized and fine-grained mixed item embedding matrix for the client, thereby improving the final recommendation effect; negative sampling is performed for each client, and positive samples and negative samples are mixed and used together for training. While improving the training effect, it prevents the server from identifying the historical interaction between the client and the item based on the updated data uploaded by the client, thereby protecting user privacy; a local differential privacy method is used for data communication between the central server and the client, reducing the risk of inference attacks, enhancing data security, and protecting data privacy to a certain extent; the model adopts a total of three shallow perceptrons, which are used for personalizing the global item embedding vector, generating the item embedding mixing factor matrix, and recommendation prediction. The model is relatively lightweight and more convenient to deploy on smart terminals with limited performance. BRIEF DESCRIPTION OF THE DRAWINGS

[0029] In order to more clearly illustrate the technical solution of the present invention, the following briefly introduces the drawings required for use in the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0030] Figure 1 Schematic diagram of the system structure of the present invention;

[0031] Figure 2 It is a training flow chart of the present invention;

[0032] Figure 3 Structural diagram of the client local training model of the present invention. DETAILED DESCRIPTION

[0033] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of the present invention.

[0034] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, the present invention is further described in detail below with reference to the accompanying drawings and specific embodiments.

[0035] Before describing, the characters appearing in the examples are explained:

[0036] The vectors and matrices with the subscript "0" in the characters come from the central server;

[0037] The vectors and matrices with subscripts "0, u" in the characters come from the client.

[0038] Example 1

[0039] This embodiment provides a privacy-preserving personalized federated recommendation method, which personalizes the item embedding matrix by fusing the global item embedding matrix from a central server with the item preference embedding vector trained on the client.

[0040] The above method is to use Figure 1 The recommendation system implementation shown in the figure includes: a central server and several clients. This embodiment includes the following three types of entities:

[0041] (1) Central server: The central server is used to initialize the model structure and model parameters and maintain a global item existence table. It is responsible for distributing the initialized model structure and parameters to the clients participating in the federated training, collecting the item embedding matrices uploaded by the clients, and aggregating the item embedding vectors uploaded by the clients according to the item ID.

[0042] (2) Items: This recommendation system constructs a global item embedding vector for each item, which is shared by all clients. At the same time, each client maintains a local item preference embedding vector. When recommending items to a client, the global item embedding vector and the user's item preference embedding vector are combined for recommendation.

[0043] (3) Client: In this recommendation system, each client corresponds to a user. Three shallow perceptrons are used to personalize the global item embedding vector, generate the item embedding mixed factor matrix, and predict recommendations. The client maintains an item preference embedding vector, a global item embedding orthogonal mapping matrix, a two-layer perceptron for generating the item embedding mixed factor matrix, and a single-layer perceptron for user recommendation prediction. These models serve as the client's personalized models and are always kept locally and do not participate in federated aggregation.

[0044] The workflow of this system includes the following three stages:

[0045] (1) Model initialization and dataset construction: The central server constructs an item existence table based on the existence of items in the recommendation system, which is used to record the number of types of items in the recommendation system. Each item constitutes a record, and each record contains information such as the ID of the item; the central server initializes the item embedding matrix and client model parameters; each client constructs a local dataset in the form of (client ID, item ID, interaction (0 or 1), timestamp) based on the historical interaction with the item, and combines negative sampling to construct a training set, test set, and validation set.

[0046] (2) Training of the model and item embedding vectors: The central server distributes the initialized client model parameters and the global item embedding matrix to the clients participating in federated learning. The clients train the model using local data. After the training is completed, the updated item embedding matrix is ​​encrypted locally using differential privacy and uploaded to the central server. The central server aggregates it and then distributes it to the clients for the next round of training until the predetermined number of training times is reached.

[0047] (3) Complete the recommendation: Each client uses the trained model parameters to fuse the global item embedding vector with the user's item preference embedding vector, inputs the trained prediction model, and obtains the final recommendation result.

[0048] Example 2

[0049] The following will describe in detail how the present invention solves technical problems in real life in conjunction with this embodiment.

[0050] S1. The central server maintains a global item presence table X, which is used to record the number of item types in the recommendation system. The purpose of maintaining a global item presence table X is, first, that the item interaction lists of some clients may overlap to varying degrees, and X is used to prepare for the subsequent aggregation of updated global item embedding vectors uploaded by different clients by the central server; second, when performing random negative sampling, each client needs to know which specific items it has not interacted with through the global item presence table. The central server initializes the global item embedding matrix Global item embedding orthogonal mapping matrix A two-layer perceptron (weights are ) and user recommendation prediction single-layer perceptron And distributed to each client participating in federated training.

[0051] S2. Each client obtains the initialized model weights from the server and initializes the corresponding parts of the local model: global item embedding orthogonal mapping matrix Two-layer perceptron for generating item embedding mixing factor matrix and user recommendation prediction single-layer perceptron Get the global item embedding matrix from the server Initialize a user-item preference embedding vector

[0052] S3. Each client lists X based on its historical interactions with the item u The global items are stored in Table X. All items that have interacted with each other are taken as positive samples, and items that have never interacted with each other are randomly sampled as negative samples. The ratio of positive samples to negative samples is 1:4. Positive samples and negative samples are mixed according to this ratio to obtain the training set; the two positive samples with the most recent interaction time are set aside separately, and 99 negative samples are randomly sampled for each to construct the test set and validation set.

[0053] S4. Each client uses the local training set data to train the recommendation model in parallel, such as Figure 2 and Figure 3 As shown:

[0054] S401. Each client selects the global embedding vector of the corresponding item from the global item embedding matrix based on the item categories included in the training set, concatenates the global item embedding matrices that match the item categories in the client's training set, and performs an orthogonal transformation on them using the global item embedding orthogonal mapping matrix to obtain the migration item embedding matrix.

[0055] Specifically, the global item embedding matrix from the central server It is the average of the item embedding matrices uploaded by all clients, so it does not necessarily match the current client's personalized view of the item. Therefore, this embodiment first learns the mapping relationship between a global item embedding matrix and the client's personalized view to achieve preliminary personalization of the global item embedding matrix; a single-layer perceptron with an orthogonal weight matrix and a bias term of 0 is used. Perform an orthogonal transformation on the global item embedding matrix. The inner product of the global item embedding vector remains unchanged before and after the orthogonal mapping. While maintaining the similarity of the item embedding vector before and after the transformation, it introduces personalization for the current client. Each client selects the item from the global item embedding matrix according to the item category included in the training set. Select the global item embedding vector i of the corresponding item global , splicing the global item embedding matrix that matches the item types in the client training set Will Use global item embedding orthogonal mapping matrix Perform orthogonal mapping to obtain the migration item embedding matrix As shown in the following formula:

[0056]

[0057] (4b) Broadcast the user's item preference embedding vector to the same dimension as the migrated item embedding matrix to obtain the user's item preference embedding matrix; subtract the migrated item embedding matrix from the user's item preference embedding matrix and calculate the absolute value to obtain the absolute value matrix of the item-preference embedding matrix difference.

[0058] Specifically, we further embed the migration item matrix Perform local personalization. Later in this embodiment, an item embedding mixing factor matrix will be trained to embed the global item embedding matrix representing the average item preference of the client. and the item preference embedding vector representing the individual item preferences of a specific client Weighted averaging is used to extract fine-grained, personalized mixed item embedding matrices.

[0059] First, Broadcast to The same dimension, get the item preference embedding matrix

[0060] Embed the migration items into the matrix and item preference embedding matrix Subtract and calculate the absolute value to obtain the absolute value matrix Δ of the item-preference embedding matrix difference 0，u , as shown below:

[0061]

[0062] Among them, Δ 0，u Each element represents and The similarity of elements at corresponding positions, the smaller the value, the higher the similarity.

[0063] (4c) The absolute value matrix of the item-preference embedding matrix differences is input into a two-layer perceptron to generate the item embedding mixing factor matrix.

[0064] Specifically, in order to preserve the original information contained in the global item embedding matrix as much as possible, this embodiment selects the global item embedding matrix Instead of transferring the item embedding matrix and the user's item preference embedding matrix To merge; at the same element position, and may have different contribution levels; therefore we train a two-layer perceptron to learn and Mixing ratio;

[0065] The absolute value matrix Δ of the item-preference embedding matrix difference 0，u Input a two-layer perceptron The activation functions use ReLU and Sigmoid functions to generate the item embedding mixing factor matrix As shown in the following formula:

[0066]

[0067] (4d) Embed items into the mixing factor matrix and As global item embedding matrix and the user's item preference embedding matrix The weights of the mixed items are used to obtain the weighted average value and embed the matrix As shown in the following formula:

[0068]

[0069] (4e) The mixed item embedding matrix is ​​input into the user recommendation prediction single-layer perceptron to obtain the client's prediction score for each item; the loss value is calculated and the model parameters are updated using mini-batch stochastic gradient descent; each client trains the local recommendation system model in parallel.

[0070] Specifically, from the mixed item embedding matrix Get the mixed item embedding vector of item i Will Input user recommendation prediction single layer perceptron And after activation by Sigmoid activation function, we get the predicted score of user u for item i As shown in the following formula:

[0071]

[0072] Calculate the binary cross entropy loss as shown below:

[0073]

[0074] Among them, Item u Represents the set of items that client u has interacted with; Item′ u represents the set of items that client u has not interacted with; Represents the predicted score of user u for negative sample i′.

[0075] Mini-batch stochastic gradient descent is used to update model parameters, and each client trains the local recommendation system model in parallel.

[0076] S5. After completing several epochs of local training, each client uses the updated model parameters to make predictions on the validation set and test set, calculate various evaluation indicators, and retain a record of the form (training round ID, evaluation indicator value on the validation set, evaluation indicator value on the test set, local model parameters). The local model includes the updated global item embedding matrix, the user's item preference embedding vector, the global item embedding orthogonal mapping matrix, a two-layer perceptron for generating the item embedding mixing factor matrix, and a single-layer perceptron for user recommendation prediction. The client performs local differential privacy encryption on the updated global item embedding matrix and uploads it to the central server together with the client's calculated evaluation indicator values ​​on the validation set and test set.

[0077] Specifically, this embodiment mainly evaluates the recommendation quality from two aspects: the hit ratio (HR) and the normalized discounted cumulative gain (NDCG). HR is used to indicate whether the client's positive samples are hit in the prediction result list, that is, whether the items the user wants are recommended, emphasizing the "accuracy" of the recommendation; NDCG is used to indicate whether the client's positive samples are at the top of the prediction result list, that is, whether the items the user wants are recommended first, emphasizing the "priority" of the recommendation. The calculation formulas are as follows:

[0078]

[0079] Among them, |U t| represents the number of clients, hits(u) = 1 indicates that the positive samples in the client's validation set / test set appear in the first K of the recommendation list, otherwise, hits(u) = 0; p u Represents the position of the positive sample in the client u validation set / test set in the recommended list. When the predicted list does not contain the positive sample, p u →∞. In this embodiment, K=10 is set, that is, only the first 10 recommended items are considered.

[0080] After completing local training for several epochs, each client uses the updated model parameters to make predictions on the validation set and test set, calculates various evaluation indicators, and retains a record of the form (training round ID round , the evaluation index value on the validation set (NR v ,NDCG v ), the evaluation index value on the test set (HR t ,NDCG t ), local model parameters), where the local model includes the updated global item embedding matrix User item preference embedding vector Global item embedding orthogonal mapping matrix Two-layer item embedding mixed factor generation perceptron and user recommendation prediction single-layer perceptron The client embeds the updated global item into the matrix Perform local differential privacy encryption and upload it to the central server together with the client's evaluation index values ​​on the validation set and test set:

[0081]

[0082] Where La(0,λ) represents Laplace noise with mean 0 and noise intensity λ. The addition of Laplace differential privacy noise reduces the risk of malicious third-party inference attacks.

[0083] S6. After receiving the global item embedding matrix uploaded by each client and encrypted with local differential privacy, the central server Finally, according to the global item existence table X and item ID, the FedAvg algorithm is used to aggregate the global embedding vectors of each item to obtain a new global item embedding vector, as shown in the following formula:

[0084]

[0085] Among them, |U i | represents the number of clients that uploaded the global item embedding vector of item i.

[0086] The global item embedding vector after aggregating each item Spliced ​​into a global item embedding matrix The server calculates the average value of each evaluation indicator on the validation set and test set for all clients respectively; if the average value of the evaluation indicator on the validation set is better than the best average value of the corresponding indicator in the previous training, the best average value of the indicator is updated and a record of the form (training round ID round , the best average value of the evaluation index on the validation set Average value of evaluation indicators on the test set Global item embedding matrix ) records; embed the aggregated global items into the matrix Distribute to each client for the next round of training; if the best average value of the evaluation indicator is updated in this round, the client also needs to be notified to retain the local model parameters of the current round.

[0087] S7. The client verifies the message content sent by the central server. If it contains a signal to retain the model parameters of the current round, the local model parameters of the current round are saved. Otherwise, no need to save, the model parameters are directly updated and replaced. The client uses the received aggregated global item embedding matrix Continue to the next round of training.

[0088] S8. Take S4 to S7 as a cycle, and periodically cycle S4 to S7 until the number of iterations set at the beginning. The central server records the training round ID corresponding to the best average value of the evaluation index on the validation set. round , determine the final global item embedding matrix The recommendation effect of this system is measured by the average value of the evaluation indicators on the corresponding round test set.

[0089] Finally, the trained prediction model is used to complete the recommendation.

[0090] Example 2

[0091] This embodiment provides an experimental process to demonstrate the superiority of the present invention.

[0092] Table 1 compares the recommendation results of this embodiment with those of FedMF, FedRECON, FedNCF, and PFedRec on the MovieLens-100k, MovieLens-1M, Lastfm-2K, and Foursquare datasets. Among them, FedMF was proposed by Chai et al. in 2020. It is a federated learning version of the matrix decomposition (MF) recommendation algorithm and a classic federated recommendation system (FedRec) method. It updates the user embedding vector locally and aggregates the item embedding vector gradient globally. In 2021, Singhal et al. proposed the FedRECON federated recommendation algorithm. Between each two rounds of iteration, this method does not inherit the user embedding vector of the previous round, but completely discards the user embedding vector and retrains it using a reconstruction mechanism. FedNCF was proposed by Parifanis et al. in 2022 as a federated learning version of neural collaborative filtering (NCF). Specifically, each user will update the user embedding vector locally and upload the item embedding vector and scoring function to the server for global update. In 2023, Zhang et al. proposed the PFedRec algorithm, which adopts a new "dual personalization" mechanism to efficiently extract personalized features of users and items. The four data sets used in this embodiment are all real data from the real world and are often used in the field of recommendation systems. Among them, the MovieLens dataset contains user ratings of movies from the MovieLens website, and each user has rated at least 20 movies; the Lastfm-2k dataset comes from Last.fm, one of the world's largest music platforms, and records the interaction between users and artists' works; the Foursquare dataset is obtained through the public API of the Foursquare application, and contains a large number of users' check-in records, social relationships and venue ratings.

[0093] Table 1

[0094]

[0095] It can be seen from Table 1 that the improved personalized federated recommendation system in this implementation case improves the recommendation performance.

[0096] Table 2 shows the ablation experiment results of the relevant modules on the MovieLens-100k dataset of this embodiment, where NoOrth refers to canceling the global object embedding orthogonal mapping matrix. Orthogonal properties, use a normal linear mapping matrix instead; NoOrthMapping means canceling the global item embedding orthogonal mapping matrix Directly find the absolute value vector of the difference between the global item embedding vector and the user preference embedding vector; OneUserEmbedding refers to canceling the user recommendation prediction single-layer perceptron The inner product of the user-item preference embedding vector and the mixed-item embedding vector is directly used as the prediction result.

[0097] Table 2

[0098] method HR@10 NDCG@10 NoOrth 0.7211 0.4485 NoOrthMapping 0.7190 0.4271 OneUserEmbedding 0.7041 0.4219 The present invention 0.7275 0.4497 ,

[0099] Table 2 shows the effectiveness of embedding the global item orthogonal mapping matrix and separating the user's item preference and decision logic in improving the recommendation effect in this implementation case.

[0100] Table 3 shows the model performance of this embodiment with local differential privacy added to the MovieLens-100k dataset.

[0101] Table 3

[0102] Noise intensity λ=0 λ=0.2 λ=0.4 λ=0.6 λ=0.8 λ=1.0 HR@10 0.7275 0.7237 0.7116 0.7063 0.6935 0.6871 NDCG@10 0.4497 0.4305 0.4219 0.4174 0.4132 0.4094 ,

[0103] As shown in Table 3, as the noise intensity increases, the performance of the model in terms of HR@10 and NDCG@10 gradually decreases; however, even with the addition of noise of λ = 0.2, the model of this embodiment still outperforms all the comparison algorithms.

[0104] The embodiments described above are merely descriptions of preferred embodiments of the present invention and are not intended to limit the scope of the present invention. Without departing from the spirit of the present invention, various modifications and improvements made to the technical solutions of the present invention by persons skilled in the art should fall within the scope of protection defined by the claims of the present invention.

Claims

1. A personalized federated recommendation method for privacy protection, which personalizes the item embedding matrix by fusing the global item embedding matrix from the central server with the item preference embedding vector trained on the client. The method is characterized by: The method comprises the following steps: S1. The central server maintains a global item table to record the number of item types in the recommendation system. The central server initializes the model weights and the global item embedding matrix and distributes them to all clients participating in federated training. S2. Each client obtains the initialized model weights and global item embedding matrix from the central server, and initializes the weights of the global item embedding orthogonal mapping matrix, the item preference embedding vector, the two-layer perceptron used to generate the item embedding mixing factor matrix, and the single-layer perceptron for user recommendation prediction. S3. Based on their historical interactions with items, each client selects all interacted items as positive samples and randomly samples items they have not interacted with as negative samples. These positive and negative samples are mixed in a certain ratio to form a training set. The two most recently interacted positive samples are set aside and mixed with a certain ratio of negative samples to form a test set and a validation set. S4. Each client uses the local training set data to train the recommendation model in parallel: S401. Each client selects the global embedding vectors of the corresponding items from the global item embedding matrix according to the item categories included in the training set, and splices them into a global item embedding matrix that matches the item categories in the client's training set. This is then orthogonally transformed using the global item embedding orthogonal mapping matrix to obtain a migration item embedding matrix. First, a mapping relationship between the global item embedding matrix and the client's personalized views is learned to achieve preliminary personalization of the global item embedding matrix. A single-layer perceptron with an orthogonal weight matrix and a bias term of 0 is used. Perform an orthogonal transformation on the global item embedding matrix to introduce personalization for the current client; each client extracts the item from the global item embedding matrix according to the item category included in the training set. Select the global item embedding vector i of the corresponding item global , splicing the global item embedding matrix that matches the item types in the client training set Will Use global item embedding orthogonal mapping matrix Perform orthogonal mapping to obtain the migration item embedding matrix As shown in the following formula: S402. Further embedding the migration item into the matrix Perform local personalization; embed client's personal item preferences into vectors Broadcast to The same dimension, get the item preference embedding matrix Embed the migration items into the matrix and item preference embedding matrix Subtract and calculate the absolute value to obtain the absolute value matrix Δ of the item-preference embedding matrix difference 0,u , as shown below: Among them, Δ 0,u Each element represents and similarity of elements in corresponding positions; S403. Embed the absolute value matrix Δ of the item-preference matrix difference 0,u Input a two-layer perceptron Training to learn the global item embedding matrix and the user's item preference embedding matrix The fusion mixing ratio, the activation function uses ReLU and Sigmoid functions respectively, to generate the item embedding mixing factor matrix As shown in the following formula: S404. Embed items into the mixing factor matrix and As global item embedding matrix and the user's item preference embedding matrix The weights of the mixed items are used to obtain the weighted average value and embed the matrix As shown in the following formula: S405. Input the mixed item embedding matrix into the user recommendation prediction single-layer perceptron to obtain the client's predicted score for each item; calculate the loss value and use mini-batch stochastic gradient descent to update the model parameters; train the local recommendation system model in parallel on each client; Get the mixed item embedding vector of item i Will Input user recommendation prediction single layer perceptron And after activation by Sigmoid activation function, we get the predicted score of user u for item i As shown in the following formula: Calculate the binary cross entropy loss as shown below: Among them, Item u Represents the set of items that client u has interacted with; Item′ u represents the set of items that client u has not interacted with; Represents the predicted score of user u for negative sample i′; S5. After completing several epochs of local training, each client uses the updated model parameters to perform predictions on the validation and test sets, calculate various evaluation metrics, and retain a record containing the training epoch ID, the evaluation metric values ​​on the validation and test sets, and the local model parameters. The local model includes the updated global item embedding matrix, the user's item preference embedding vectors, the global item embedding orthogonal mapping matrix, a two-layer perceptron used to generate the item embedding mixing factor matrix, and a single-layer perceptron for user recommendation prediction. The client locally differentially encrypts the updated global item embedding matrix and uploads it to the central server along with the client's calculated evaluation metric values ​​on the validation and test sets. S6. After receiving the global item embedding matrix uploaded by each client and encrypted using local differential privacy, the central server aggregates the global embedding vectors of each item using the FedAvg algorithm based on the global item presence table and item ID to obtain a new global item embedding vector. The server calculates the average value of each evaluation metric for all clients on the validation set and test set. If the average value of an evaluation metric on the validation set is better than the best average value of the corresponding metric in the previous training, the best average value of that metric is updated and saved. The aggregated global item embedding matrix is ​​distributed to each client. If the best average value of the evaluation metric is updated in this round, the client is notified to retain the local model parameters of the current round. S7. The client verifies the message sent by the central server. If it contains a signal to retain the model parameters for the current round, the client saves the local model parameters for the current round. Otherwise, the client directly updates or replaces the model parameters without saving. The client then uses the received aggregated global item embedding matrix to proceed to the next round of training. S8. Take S4 to S7 as one cycle, and periodically cycle S4 to S7 until the number of iterations set at the beginning. The central server determines the final global item embedding matrix based on the training round corresponding to the best average value of the evaluation index on the recorded validation set, and measures the recommendation effect with the average value of the evaluation index on the test set of the corresponding round; each client uses the trained global item embedding matrix and local model to output its own final recommendation result.

2. A privacy-preserving personalized federated recommendation system, the system being used to implement the method of claim 1, characterized in that: It adopts a federated learning architecture, including a single central server and multiple clients. Each client does not need to upload original local data to the server, nor does it need to expose the client model to the server. It only uploads the updated item embedding matrix to the server.

3. The privacy-preserving personalized federated recommendation system according to claim 2, characterized in that: The global item embedding matrix updated locally on the client is uploaded to the central server after being encrypted with local differential privacy.

4. The privacy-preserving personalized federated recommendation system according to claim 2, characterized in that: Each client needs to train a set of local models, including item preference embedding vectors, orthogonal mapping matrices, two-layer perceptrons, and single-layer perceptrons for user recommendation predictions; Use the orthogonal mapping matrix to perform an orthogonal transformation on the global item embedding matrix to obtain the migration item embedding matrix; The absolute value matrix of the difference between the migration item embedding matrix and the user item preference embedding vector is calculated as the input of the two-layer perceptron, and a mixing factor matrix is ​​output. The item preference embedding vector and the global item embedding matrix are fused using the mixing factor matrix to obtain a mixed item embedding matrix, thereby realizing local personalization of the item embedding matrix.

5. The privacy-preserving personalized federated recommendation system according to claim 2, characterized in that: Instead of using the inner product of the user's item preference embedding vector and the mixed item embedding vector as the recommendation prediction score, the mixed item embedding vector is input into the recommendation prediction single-layer perceptron built on the client to obtain the recommendation prediction score.

6. The privacy-preserving personalized federated recommendation system according to claim 2, characterized in that: Each client trains three shallow perceptrons, which are used to personalize the global item embedding vector, generate the item embedding mixing factor matrix, and make recommendation predictions.

Images