Cross-domain recommendation method based on federated learning and privacy protection

Abstract

The present invention relates to a cross-domain recommendation method based on federated learning and privacy protection, and belongs to the field of cross-domain recommendation technology. The method comprises: S1: aligning common users in different fields: searching for similar users of cold-start users in the source domain and target domain according to a similarity threshold; S2: constructing a heterogeneous information network according to a meta-path and random walk strategy, and inputting similar users into the heterogeneous information network, extracting the latent features of the cold-start user source domain, and using the features to obtain a user-item interaction feature matrix; S3: constructing and training a federated learning model locally, specifically: using CNN to identify the user-item interaction feature matrix, then using federated learning combined with homomorphic encryption to encrypt and exchange the intermediate results, and using a gradient descent algorithm to optimize the encrypted target; finally, updating the gradient and loss function of the model; S4: using the trained local federated learning model to generate accurate recommendation results for users.

Description

Technical Field

[0001] The present invention belongs to the field of cross-domain recommendation technology and relates to a cross-domain recommendation method based on federated learning and privacy protection. Background Art

[0002] With the continuous development of the economy and technology, and the continuous improvement of online shopping capabilities, consumers are gradually shifting their shopping behavior from physical stores to online shopping platforms. Furthermore, with the rapid growth of short videos in recent years, livestream livestreaming has become a popular consumption method. Consequently, major internet platforms have accumulated a wealth of data on user consumption behaviors and preferences. Based on this, many researchers have attempted to develop model systems to analyze user behavior online, aiming to improve the user experience, promote consumer behavior and platform revenue, and ultimately bridge the gap between users and merchants. For example, based on a user's shopping habits, they can recommend stores they might like, thereby enhancing the user's shopping experience.

[0003] With the increasing awareness of user privacy protection in recent years, many users are reluctant to share their historical data with platforms, which has restricted the development of recommendation systems. Since the principle of recommendation systems is to use historical user behavior to predict user preferences and present them to users, it is necessary to develop privacy-protected recommendation systems. Furthermore, based on the characteristics of current big data, user data is characterized by large overall volume and local sparseness, which means that "data silos" may occur in some areas. This makes recommendation systems prone to cold start problems and data sparsity, affecting the accuracy of recommendations. The emergence of cross-domain recommendation systems perfectly solves this problem. Cross-domain recommendation can combine data from multiple domains and introduce information from other domains (source domains) for assistance, enabling better recommendations in the target domain or even multiple domains.

[0004] The common method used by personalized recommendation systems to solve data sparsity and cold start problems is to consider the potential characteristics of data interactions. For example, Reference [1] proposed a systematic method for a personal information recommendation platform based on deep learning tourism. This method enables tourists to obtain tourism information conveniently and quickly through scientific information organization and presentation, and helps tourists to better arrange travel plans and make travel decisions. By effectively aggregating multiple node neighborhoods, high-order collaborative information is embedded into the node embedding vector, and the user's potential preferences are obtained, solving the user data sparsity and cold start problems. Reference [2] designed an English teaching resource personalized recommendation system based on the multi-K nearest neighbor regression algorithm. According to the overall architecture of the teaching resource personalized recommendation system, a resource browsing function module, a teaching resource detailed page recommendation module and a teaching resource database were designed. Based on the basic idea of ​​the multi-K nearest neighbor regression algorithm, in order to avoid the loss of important data in the recommendation of English teaching resources and reduce the data loss rate, an English teaching resource missing data reconstruction algorithm was proposed. Finally, the path interest of student users is considered from the selection of browsing path and access time to achieve personalized recommendation of English teaching resources.

[0005] At present, cross-domain recommendation has not been widely used. Most of the research focuses on social network data and the interaction between users and items. For example, Reference [3] proposed a refined graph representation for cross-domain recommendation (CDR) based on edge pruning, which takes into account the feature distribution in the latent space. This method introduces edge pruning into the latest graph-based CDR to improve the graph. Comparison is made in different domains, a latent space is constructed, and edge pruning is performed based on their correlation. Then a cross-domain specific graph representation is obtained, and the performance is improved by considering the relationship between two items through edge pruning. Reference [4] proposed a new cross-domain sequential re-evaluation method based on graph collaborative filtering (CsrGCF) to alleviate the sparsity problem of user interaction data. Specifically, by designing a time-aware and relationship-aware graph attention mechanism with collaborative filtering to utilize the high-order behavior patterns of users, satisfactory results were achieved in both domains. The time-aware graph attention mechanism (TGAT) aims to learn the domain-level sequence representation of items. The relationship-aware graph attention mechanism (RGAT) is proposed to learn the feature representation of collaborative items and users. Furthermore, to improve recommendation performance in both domains simultaneously, a cross-domain feature bidirectional transfer model (CFBT) is proposed. This model transfers users' shared features between the two domains while retaining the user's domain-specific features in a specific domain. Finally, cross-domain and sequential information are combined to recommend the next item that the user will like. However, the above methods do not have privacy protection capabilities.

[0006] Traditional privacy protection methods for personalized recommendation systems include literature [5]-[7], differential privacy [8], and encryption [9]. Wang et al. [5] protect user privacy by learning user group preferences rather than individual preferences. Their recommendations are made through personalized groups on the user's local device based on non-negative matrix factorization technology. Li et al. [8] use Markov chains to model the user's check-in sequence and protect the user's privacy by adding weighted noise. Wang et al. [9] designed a PLSA encryption algorithm and proved that it is secure under a semi-honest protocol. However, the above methods do not have cross-domain recommendation capabilities.

[0007] Currently, there is very limited research on cross-domain recommendation systems with privacy protection. For example, Reference

[10] discloses a new privacy protection algorithm based on group recommendation. The general model of the recommendation system is analyzed through user modeling, recommendation object modeling and recommendation algorithm, and the privacy issues of the recommendation system are perceived through questionnaire analysis. Based on the classification of user privacy concerns, group recommendation is introduced, and user data is anonymously submitted through the sender anonymity feature provided by the crowdsourcing network to prevent the server and malicious users from identifying good users and complete the research on user data. Reference

[11] discloses a dynamic privacy budget allocation method based on a local differential privacy protection scheme. This method proposes an improved user-based collaborative filtering algorithm. The algorithm uses a matrix-based similarity calculation method when calculating user similarity. The user's privacy is protected during the recommendation process, while ensuring that the recommendation performance will not be greatly lost. The solution proposed in this invention first dynamically adds noise to local sensitive data to ensure user privacy, then sends the noise-added data to the server for similarity calculation, and finally gives movie recommendations through a user-based collaborative filtering algorithm. Reference

[12] discloses an efficient privacy protection collaborative filtering algorithm based on differential privacy protection and time factor. This approach can effectively reduce the risk of leaking private data while obtaining the required privacy protection services. However, none of the above cross-domain recommendation systems consider the correlation between e-commerce and POI domains, and thus cannot achieve accurate cross-domain recommendations.

[0008] References:

[0009] [1]Xuejuan Wang:Implementation of Personalized InformationRecommendation Platform System Based on Deep Learning Tourism.J.Sensors 2022:1-9(2022).

[0010] [2]Yan Tang,Yang Yu:APersonalized Recommendation System for EnglishTeaching Resources Based on Multi-K Nearest Neighbor RegressionAlgorithm.Secur.Commun.Networks 2022:7077123:1-7077123:11(2022).

[0011] [3]Taisei Hirakawa,Keisuke Maeda,Takahiro Ogawa,Satoshi Asamizu,MikiHaseyama:Refining Graph Representation for Cross-Domain Recommendation Basedon Edge Pruning in Latent Space.IEEE Access 10:12503-12509(2022).

[0012] [4]Bingyuan Wang,Baisong Liu,Hao Ren,Xueyuan Zhang,Jiangcheng Qin,Qian Dong,Jiangbo Qian:Exploiting high-order behaviour patterns for cross-domain sequential recommendation.Connect.Sci.34(1):2597-2614(2022).

[0013] [5]X.W.Wang,H.Yang,K.Lim.“Privacy-Preserving POI Recommendation UsingNonnegative Matrix Factorization,”in IEEE Symposium on Privacy-AwareComputing.DC,USA,IEEE,2018,pp.117-118.

[0014] [6]X.X.Li,P.P.Sun,Y.Bai,L.E.Wang.“M-generalization for multipurposetransactional data publication,”Frontiers of Computer Science.2018,12(6):1241-1254.

[0015] [7]L.E Wang,X.X Li.“Agraph-based multifold model for anonymizing datawith attributes of multiple types,”Computers&Security.2018,72C:122-135.

[0016] [8]L.Kuang,S.M.Tu,Y.Q.Zhang,X.X.Yang.“Providing privacy preserving innext POI recommendation for Mobile edge computing,”Journal of CloudComputing.2020,9(1).

[0017] [9]W.Q.Wang,A.Liu,Z.X.Li,X.L.Zhang,Q.Li,X.F.Zhou.“Protecting multi-party privacy in location-aware social point-of-interest recommendation,”World Wide Web.2018,22(2):863-883.

[0018]

[10] Xuefeng Ding,Xuehong Liu:User privacy protection algorithm ofperceptual recommendation system based on group recommendation.Int.J.Auton.Adapt.Commun.Syst.13(2):135-150(2020).

[0019]

[11] Min Li, Yingming Zeng, Yue Guo, Yun Guo: AMovie Recommendation SystemBased on Differential Privacy Protection.Secur.Commun.Networks 2020:6611463:1-6611463:10(2020).

[0020]

[12] Chunyong Yin, Lingfeng Shi, Ruxia Sun, Jin Wang: Improved collaborative filtering recommendation algorithm based on differential privacy protection. J. Supercomput. 76(7):5161-5174(2020). Summary of the Invention

[0021] In view of this, the purpose of this invention is to provide a cross-domain recommendation method based on federated learning and privacy protection to improve the accuracy of the recommendation system. Compared with single-domain recommendation algorithms, this invention can more accurately predict the content of user interest. Moreover, compared with simple cross-domain recommendation algorithms, it not only protects user privacy but also can more fully utilize the potential characteristics of data to achieve accurate recommendations.

[0022] In order to achieve the above object, the present invention provides the following technical solutions:

[0023] A cross-domain recommendation method based on federated learning and privacy protection includes the following steps:

[0024] S1: Align common users and find similar users of the cold-start user in the source and target domains based on the similarity threshold;

[0025] S2: Based on the meta-path and random walk strategies, a heterogeneous information network is constructed. Similar users are input into the heterogeneous information network to extract the latent features of the cold-start user source domain. The features are then used to obtain the user-item interaction feature matrix.

[0026] S3: Build and train the federated learning model locally. Specifically, a convolutional neural network (CNN) is used to identify the user-item interaction feature matrix. Federated learning combined with homomorphic encryption is then used to encrypt and exchange intermediate results. The encrypted results are then optimized using a gradient descent algorithm. Finally, the model's gradient and loss function are updated.

[0027] S4: Use the trained local federated learning model to generate recommendation results for users.

[0028] Furthermore, in step S1, cosine similarity is used to measure the similarity between different users.

[0029] Furthermore, in step S2, constructing a heterogeneous information network specifically includes: after setting the meta-path, using the HIN2Vec method to extract cold start user features. The core of HIN2Vec is a neural network model, through which the vector representation of each node can be obtained; then, for each node v in the path, an n-dimensional feature vector is randomly generated to represent the feature set W∈R |V|×n , and judge the relationship between v1, v2 and d, where v1 and v2 are two adjacent nodes on path d, and r (r∈R) represents the relationship between nodes v1 and v2; then judge according to the meta-path discriminant formula in HIN2Vec, where F(v1, v2, r) represents whether there is a relationship r between v1 and v2. If so, the value is 1, and if not, the value is 0;

[0030] The objective function for learning node features in the meta-path is defined as:

[0031] L(v1,v2,r)=F(v1,v2,r)log[p(r|(v1,v2)]+(1-F(v1,v2,r))log(1-p(r|(v1,v2))(1)

[0032] Where L(v1,v2,r) is the target loss function, F(v1,v2,r) indicates whether there is a relationship r between v1 and v2. If so, the value is 1, and if not, the value is 0. p(r|v1,v2) represents the probability that v1 and v2 have the relationship r.

[0033] Then, the stochastic gradient descent algorithm is used to optimize the objective function L(v1,v2,r), and the node features are updated according to the HIN2Vec formula;

[0034] After obtaining the feature representation of the node set, the user-item interaction feature matrix T u,i It is expressed as follows:

[0035]

[0036] Among them, T u,i ∈W |v|×n , represents matrix multiplication, represents the user feature weight, Represents the item feature weight, .t represents transposition, Represents the characteristic value of the mth meta-path of the i-th item, m represents the mth meta-path in a meta-path set, and M represents the set of all meta-paths.

[0037] Furthermore, in step S3, a federated learning model is constructed and trained, specifically including: setting the objective function L of the vertical federated learning to:

[0038]

[0039] in, It represents the characteristic parameters of field A. It represents the characteristic parameters of the B field, θ A is the model parameter of the A domain, θ B is the model parameter of the B domain, y i is the label, λ is the regularization parameter;

[0040] Assumptions Using [[·]] to represent the homomorphic encryption operation, the encryption objective function is expressed as:

[0041]

[0042]

[0043]

[0044]

[0045] Among them, L A Represents some parameters of the objective function L with respect to domain A, L B Represents some parameters of the objective function L with respect to domain B, L AB Represents some parameters of the objective function L that are common to domains A and B;

[0046] The gradient descent algorithm is used to optimize the target, assuming The gradient of the loss function of the training parameters is expressed as:

[0047]

[0048]

[0049] Among them, L' A Indicates the gradient descent algorithm for L A Derivatives are performed on the parameters to update the gradient, L' B Indicates the gradient descent algorithm for L B The derivative of the parameters is used to update the gradient.

[0050] Furthermore, in step S3, the gradient and loss function of the federated learning model are updated. The specific update iterative process includes:

[0051] ① Each participant aligns the entity information with encrypted entities. Encrypted entity alignment means that the servers of each participant find overlapping entities without knowing each other's information.

[0052] ② The central server creates an encryption key pair and transmits the public key pair to each participant's server, enabling each participant's server to decrypt the information individually;

[0053] ③Auxiliary domain A calculation and [[L A ]]Sent to target domain B, target domain B calculates and [[L]], and Send to auxiliary domain A, and send [[L]] to server C;

[0054] ④ Auxiliary domain A initializes mask R A , calculate [[L' A ]]+[[R A ]]Sent to server C, target domain B initializes R B , calculate [[L' B ]]+[[R B ]] is sent to server C, which decrypts [[L]], [[L' A ]]+[[R A ]] and [[L' B ]]+[[R B ]], L' A +R A Send to A, L' B +R B Send to B;

[0055] ⑤ After receiving the data from server C, auxiliary domain A and target domain B update the model parameters; then, repeat steps ③ to ⑤ until the model converges or the number of iterations reaches the upper limit.

[0056] Furthermore, in step S3, the convolutional neural network used includes:

[0057] Convolutional layer: h = f(ω⊙x+b), where x is the convolutional layer input, h is the convolutional layer output, ω is the convolution kernel, b is the bias, ⊙ is the convolution operation, and f(·) is a nonlinear function;

[0058] Pooling layer: Maximum pooling layer d f =Max(h);

[0059] Fully connected layer: d f Convert to a one-dimensional vector

[0060] Furthermore, based on Input into a multi-layer perceptron model to get the predicted score, which is defined as follows:

[0061]

[0062] Among them, w L Partial weight representation, f θ (·) is the nonlinear activation function ReLU, w is the weight of the fully connected layer, b is the bias of the fully connected layer, and b L It is expressed as partial deviation;

[0063] The model loss function is defined as follows:

[0064]

[0065] Among them, m is the number of samples, r' ui is the prediction score.

[0066] The beneficial effects of the present invention are as follows: the present invention first aligns common users in different fields, then uses heterogeneous information networks to mine hidden relationships between features, and then uses the MLP algorithm to construct a feature mapping function. The homomorphic encryption algorithm is further used to encrypt the interaction between intermediate results and feature gradients to protect user privacy. The present invention mainly utilizes the characteristics of local training data of federated learning to solve the "data island" problem across domains, and then uses heterogeneous information networks to extract and embed hidden features, which not only improves the accuracy of the recommendation system, but also protects user privacy to a certain extent.

[0067] The specific beneficial effects are as follows:

[0068] 1) In order to solve the problem of “data islands” and alleviate the cold start and data sparsity problems of the recommendation system, the present invention introduces a cross-domain recommendation system, which integrates auxiliary domain data into machine learning, that is, the data features of the POI field are migrated to the e-commerce field, which is the target domain. The hidden relationship between data features is deeply excavated to realize the transfer learning of multi-domain data, and the heterogeneous information network embedding is used in the feature extraction module, which can fully extract potential features to improve the accuracy of recommendations. As far as is known, most of the existing auxiliary data analysis work is based on data with highly overlapping user features, and personalized transfer learning cannot be achieved. The present invention takes into account the partial overlap of user features and uses a multi-layer perceptron (MLP) to learn the potential features of different fields (i.e., auxiliary fields and target fields) to achieve personalized knowledge transfer.

[0069] 2) To address user privacy concerns in cross-domain recommendation systems, this paper introduces federated learning. Federated learning allows model training to be completed locally without uploading data to a third-party server. Homomorphic encryption encrypts intermediate results and features. This paper constructs a cross-domain recommendation framework based on federated learning and privacy protection (FL&PP). To our knowledge, this is the first application of federated learning to cross-domain recommendation.

[0070] 3) This paper designs a recommendation algorithm based on cross-domain transfer learning and privacy protection. Before local model training, this paper considers using heterogeneous information networks to extract hidden features between data to further improve recommendation accuracy. This model achieves effective recommendations while protecting user privacy. Experimental results on the Movielens and FourSquare datasets demonstrate that this method can effectively ensure user privacy while improving recommendation accuracy.

[0071] Other advantages, objects, and features of the present invention will be described in part in the following description and, in part, will be apparent to those skilled in the art upon examination of the following description or may be learned from practice of the present invention. The objects and other advantages of the present invention may be realized and obtained through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0072] In order to make the purpose, technical solutions and advantages of the present invention more clear, the present invention will be described in detail below with reference to the accompanying drawings, in which:

[0073] Figure 1 This is a diagram showing the overall process framework of the cross-domain recommendation method based on federated learning and privacy protection in the present invention;

[0074] Figure 2 Flowchart for encryption model training;

[0075] Figure 3 Compare the experimental results of different datasets on different models;

[0076] Figure 4 Comparison of Precision@k performance for different k values ​​on two datasets;

[0077] Figure 5 Comparison of Recall@k performance for different k values ​​on two datasets;

[0078] Figure 6 Comparison of HitRatio@k performance for different k values ​​on two datasets;

[0079] Figure 7 The influence of different sim values ​​on the evaluation indicators. DETAILED DESCRIPTION

[0080] The following describes the embodiments of the present invention by means of specific examples, and those skilled in the art can easily understand other advantages and effects of the present invention from the contents disclosed in this specification. The present invention can also be implemented or applied through other different specific embodiments, and the details in this specification can also be modified or changed in various ways based on different viewpoints and applications without departing from the spirit of the present invention. It should be noted that the illustrations provided in the following embodiments are only schematic illustrations of the basic concept of the present invention, and the following embodiments and features in the embodiments can be combined with each other without conflict.

[0081] See also Figures 1 to 7 ,The present invention provides a cross-domain recommendation method based on federated learning and privacy protection, such as Figure 1 As shown in the figure, common users in different fields are first aligned. Then, a heterogeneous information network is used to mine the hidden relationships between features. Finally, the MLP algorithm is used to construct a feature mapping function. Furthermore, a homomorphic encryption algorithm is used to encrypt the interaction between intermediate results and feature gradients to protect user privacy.

[0082] 1. Build a Federated Learning and Privacy Protection (FL&PP) framework

[0083] In the FL&PP (Federated Learning and Privacy Protection) proposed in this invention, Figure 1 In the framework shown, the raw data can be stored locally. After data analysis, the learned latent feature vectors are encrypted using homomorphic encryption and sent to the server. The server uses a modified activation function to enable the MLP method to operate on the ciphertext, ensuring the validity of the results.

[0084] 1) Construction of heterogeneous information networks

[0085] Heterogeneous information networks can extract rich semantic information, and the combination of different meta-paths can more clearly represent heterogeneous information networks. Based on heterogeneous information networks and meta-paths, some potential vectors and target relationships between nodes can be learned by predicting the relationships between nodes. Therefore, the present invention mainly uses this idea to mine the potential features and hidden relationships between data. In the target domain dataset of the present invention, different meta-paths are established between movies, users, and directors based on the similarity between features. In the source domain dataset, meta-paths are constructed with locations and users as elements, thereby constructing a heterogeneous information network through the similarity between different features. The specific meta-paths are shown in Tables 1 and 2.

[0086] Table 1 Movie metapath

[0087]

[0088] Table 2 POI meta-path

[0089]

[0090]

[0091] After setting the meta-path, the HIN2Vec method is used to extract the features of the movie and POI data. The core of HIN2Vec is a neural network model. Through HIN2Vec, a vector representation of each node can be obtained. Then, an n-dimensional feature vector is randomly generated for each node v in the path to represent the feature set W∈R |V|×n , and determine the relationship between v1, v2, and d. v1 and v2 are two adjacent nodes on path d, and r (r∈R) represents the relationship between nodes v1 and v2. Then, the decision is made based on the meta-path discriminant formula in HIN2Vec, where F(v1, v2, r) indicates whether there is a relationship r between v1 and v2. If so, the value is 1, and if not, the value is 0.

[0092] The objective function for learning node features in the meta-path is defined as follows:

[0093] L(v1,v2,r)=F(v1,v2,r)log[p(r|(v1,v2)]+(1-F(v1,v2,r))log(1-p(r|(v1,v2))(1)

[0094] Among them, L(v1,v2,r) is the target loss function, and p(r|v1,v2) represents the probability that v1 and v2 have the relationship r.

[0095] Then, the stochastic gradient descent algorithm is used to optimize the objective function L, and the node features are updated according to the HIN2Vec formula.

[0096] After obtaining the feature representation of the node set, the potential relationship feature T between users and items is u,i It is expressed as follows:

[0097]

[0098] Among them, T u,i ∈W |v|xn , represents matrix multiplication, represents the user feature weight, Represents the project feature weight. And .t represents the transposition, It represents the characteristic value of the mth meta-path of the i-th item, m represents a meta-path in that set, and M represents the set of all meta-paths.

[0099] This invention addresses two fields: point of interest (POI) and e-commerce. Generally, cold-start users can be associated with other auxiliary domains to obtain relevant historical data. This invention utilizes heterogeneous information networks to analyze the latent features of cold-start users and similar users in these two fields. Furthermore, this invention uses cosine similarity to measure the similarity between different users. Finally, homomorphic encryption is used to encrypt the latent feature vectors and transmit them to a server. Table 3 shows the algorithm for extracting latent features of cold-start users and similar users.

[0100] Table 3 Extracting latent features of cold-start users

[0101]

[0102]

[0103] 2) Building a federated learning framework

[0104] Federated learning is an emerging foundational AI technology designed to enable efficient machine learning across multiple participants or computing nodes while ensuring information security during big data exchange, protecting the privacy of end-user data and personal data, and ensuring legal and regulatory compliance. This paper primarily utilizes a vertical federated learning framework combined with homomorphic encryption technology to protect user privacy. Our framework incorporates the homomorphic encryption scheme of Li et al., a probabilistic encryption scheme that achieves homomorphism in addition and multiplication, improving computational efficiency while adapting to neural network computations.

[0105] In order to achieve model convergence, the objective function of vertical federated learning is set as shown in formula (3):

[0106]

[0107] in, It represents the characteristic parameters of field A. It represents the characteristic parameters of the B field, θ A is the model parameter of the A domain, θ B is the model parameter of the B domain, y i is the label and λ is the regularization parameter.

[0108] Assumptions Using [[·]] to represent the homomorphic encryption operation, the encryption objective function can be expressed as:

[0109]

[0110]

[0111]

[0112]

[0113] The gradient descent algorithm is used to optimize the target, assuming The gradient of the loss function of the training parameters is expressed as:

[0114]

[0115]

[0116] Among them, L A Represents some parameters of the objective function L with respect to domain A, L B Represents some parameters of the objective function L with respect to domain B, L AB Represents some parameters of the objective function L that are common to domains A and B.

[0117] The privacy-preserving linear model of federated learning constructed in this invention mainly consists of three parts, namely auxiliary domain A, target domain B and third-party server C. The encryption model training process is as follows: Figure 2 shown.

[0118] The model iterative update process is as follows:

[0119] ① Each participant aligns the entity information with encrypted entities. Encrypted entity alignment means that the servers of each participant find overlapping entities without knowing each other's information.

[0120] ② The central server creates an encryption key pair and transmits the public key pair to each participant's server, so that each participant's server can decrypt the information individually.

[0121] ③Auxiliary domain A calculation Send to target domain B, target domain B calculates and [[L]], and Send to auxiliary domain A, and send [[L]] to server C.

[0122] ④ Auxiliary domain A initializes mask R A , calculate [[L' A ]]+[[R A ]]Sent to server C, target domain B initializes R B , calculate [[L' B ]]+[[R B ]] is sent to server C, which decrypts [[L]], [[L' A ]]+[[R A ]] and [[L' B ]]+[[R B ]], L'A +R A Send to A, L' B +R B Send to B.

[0123] ⑤ After receiving the data from server C, auxiliary domain A and target domain B update the model parameters. Then, steps ③ to ⑤ are repeated until the model converges or the number of iterations reaches the upper limit.

[0124] The local model training of the present invention is based on a convolutional neural network (CNN). The present invention uses CNN to identify the user-item interaction feature matrix. According to the traditional definition of CNN, it is generally composed of four parts: embedding layer, convolution layer, pooling layer, and fully connected layer. The present invention uses the feature vector obtained by network embedding to replace the embedding layer. The network structure CNN (·) used in the present invention is as follows:

[0125] Convolutional layer: h = f(ω⊙x+b), where ω is the convolution kernel, b is the bias, ⊙ is the convolution operation, and f(·) is a nonlinear function;

[0126] Pooling layer: Maximum pooling layer d f =Max(h);

[0127] Fully connected layer: d f Convert to a one-dimensional vector

[0128] Furthermore, based on Input into a multi-layer perceptron model to get the predicted score, which is defined as follows:

[0129]

[0130] The model loss function is defined as follows:

[0131]

[0132] Among them, m is the number of samples, r' ui is the prediction score.

[0133] Experimental analysis

[0134] In this experiment, two real data sets are used to evaluate the method proposed in this invention, namely the MovieLens data set and the FourSquare data set. Both data sets contain project access relationships and social relationships, and are widely used in the field of recommendation systems. For the Movielens data set, it contains rating data of multiple users on multiple movies, as well as movie metadata information and user attribute information. After data preprocessing, the Movielens data set includes historical data generated by 162,541 users between 1995 and 2015. For the FourSquare data set, it includes users, places, check-ins, social relationships, and ratings assigned to places by users. After data preprocessing, FourSquare includes historical data generated by 2,153,471 users between 2011 and 2013. In order to display the data more clearly, we list the data set information in Table 4.

[0135] Table 4 Introduction of different datasets

[0136] DataSet Users Items Interactive MovieLens 162541 62423 25000095 FourSquare 2153471 1021970 2809581

[0137] This experiment uses three commonly used metrics: Precision@k, Recall@k, and HitRatio@k to evaluate the performance of the recommendation system. These metrics are denoted as Pre@k, Rec@k, and Hit@k, respectively. Pre@k represents the proportion of items in the top-k list that are actually visited by users; Rec@k represents the proportion of items in the top-k list that are visited; and Hit@k measures whether the test item appears in the top-k list. The metrics are defined as follows:

[0138]

[0139]

[0140]

[0141] Among them, r is the number of test users, |·| represents the size of the set, and M i is the set of top-k items recommended to user i, N i is the set of items that user i actually visited in the test set.

[0142] 1) Comparison Model:

[0143] MLP: As a deep learning method, MLP can fully approximate nonlinear relationships and has good generalization ability and fault tolerance. In this invention, MLP is used to construct the feature mapping function.

[0144] COFIN: Collaborative filtering recommendation algorithm.

[0145] LCARS: The LDA topic model is mainly used to infer the topic distribution of documents, and can perform topic clustering or text classification based on the topic.

[0146] EMCDR: A classic embedded mapping framework for cross-domain recommendation. This model first uses latent factor models to perform embedding learning in each domain to learn the specific features of entities in each domain. It then uses mapping techniques between domains to compensate for data sparsity. The mapping techniques involved primarily include linear mapping and multi-layer perceptron (MLP) mapping.

[0147] CDR-HA: A cross-domain recommendation algorithm based on heterogeneous knowledge transfer.

[0148] 2) Parameter settings:

[0149] The algorithm of the present invention is based on a common dataset between two fields. The training set comprises 80% of the dataset, and the test set comprises the remaining 20%. Regarding the model parameter settings, the neural network activation function is a modified activation function, and the optimizer used is Adam. The learning rate is reduced by a factor of 0.5 each time, with a lower limit of 0.001. Furthermore, when obtaining node paths in the heterogeneous information network graph, the random walk length is set to 40.

[0150] 3) Experimental results and analysis

[0151] This experiment first verifies the effectiveness of federated learning combined with homomorphic encryption algorithm for privacy protection. Since federated learning allows data to be trained locally and stored locally, there is no need to upload data for training in a centralized manner. Therefore, it can protect user privacy from being violated to a certain extent. As for the verification of homomorphic encryption performance, this experiment evaluates the results. Figure 3 As shown. Figure 3 It can be seen that although the addition of homomorphic encryption will increase the time of model iteration and sacrifice some performance of the recommendation system, it can ensure the validity of data features and does not have a significant impact on the final recommendation results. The most important thing is that it can protect the security of data during the iterative interaction process and protect user privacy to a certain extent.

[0152] The experimental results of this experimental recommendation are divided into two parts. The first part is based on the results of the evaluation indicators and comparison methods of the present invention, which shows the effectiveness of the method of the present invention; the second part is to analyze the results within the experiment to describe the performance of the model of the present invention.

[0153] Table 5 Comparison experimental results of FL&PP with various models when k is 8

[0154]

[0155] Table 5 shows the experimental results of FL&PP compared with various models when k is 8. M.→F indicates that the MovieLens dataset is used as the auxiliary domain dataset and the FourSquare dataset is used as the target domain dataset, while F.→M is the opposite. Table 5 shows that when k is 8, the proposed model performs best compared to other methods.

[0156] Experimental results show that the comparison model CDR-HA and the proposed model FL&PP perform relatively well compared to the other models. This is because both models employ cross-domain recommendation and use a heterogeneous information network embedding approach. However, the proposed model is implemented on a federated learning architecture, effectively protecting user privacy. This is an advantage of the proposed model. On the MovieLens dataset, compared to the MLP method, the proposed method improves Pre@8 and Rec@8 by 25.2% and 26.7%, respectively. Compared to the COFIN method, the proposed method improves Pre@8 and Hit@8 by 16.4% and 9.8%, respectively. This is because COFIN only incorporates interestingness in its feature module, failing to fully exploit the hidden relationships between the data. In contrast, our proposed model transfers data features from the POI domain to the e-commerce domain, the target domain, thereby deeply exploring the hidden relationships between the data features. Although LCARS considers the correlation between the two different domains, it fails to protect user privacy. On the FourSquare dataset, compared to LCARS, our FL&PP framework improves the Pre@8 and Rec@8 metrics by 12.3% and 12.2%, respectively, while protecting user privacy. Finally, compared to the heterogeneous information network-based cross-domain recommendation model CDR-HA, the Pre@8 and Rec@8 metrics improve by 2.3% and 0.51%, respectively. This is because, compared to CDR-HA, the FL&PP model not only considers the characteristics of cold-start users in the auxiliary domain but also incorporates the latent characteristics of similar users. Furthermore, the proposed model effectively protects user privacy.

[0157] Since the present invention uses the cosine similarity algorithm when selecting similar users, the present invention tests the impact of different sim values ​​on the results. Figure 7 As can be seen, model performance improves as sim increases. This is because common user features with high similarity can better assist in recommendations. When sim is 0.3, the model performs best. As similarity continues to increase, model performance decreases. Preliminary analysis suggests that as user similarity continues to increase, the model will experience overfitting, affecting recommendation performance.

[0158] Experimental results show that FL&PP outperforms many related techniques. In particular, it effectively improves the accuracy of recommendations under a medium-density set of common users, which means it reduces computational overhead and the possibility of privacy leakage.

[0159] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not limiting. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical solutions of the present invention can be modified or replaced by equivalents without departing from the purpose and scope of the technical solutions, which should all be included in the scope of the claims of the present invention.

Claims

1. A cross-domain recommendation method based on federated learning and privacy protection, characterized in that: The method specifically comprises the following steps: S1: Align common users in different fields: Find similar users of cold-start users in the source and target domains based on similarity thresholds; S2: Based on the meta-path and random walk strategies, a heterogeneous information network is constructed. Similar users are input into the heterogeneous information network to extract the latent features of the cold-start user source domain. The features are then used to obtain the user-item interaction feature matrix. S3: Build and train the federated learning model locally. Specifically, a convolutional neural network is used to identify the user-item interaction feature matrix. Federated learning combined with homomorphic encryption is then used to encrypt and exchange intermediate results. The encrypted results are then optimized using a gradient descent algorithm. Finally, the model's gradient and loss function are updated. Construct and train the federated learning model, specifically including: setting the objective function L of vertical federated learning to: in, It represents the characteristic parameters of field A. It represents the characteristic parameters of the B field, θ A is the model parameter of the A domain, θ B is the model parameter of the B domain, y i is the label, λ is the regularization parameter; Assumptions Using [[·]] to represent the homomorphic encryption operation, the encryption objective function is expressed as: Among them, L A Represents some parameters of the objective function L with respect to domain A, L B Represents some parameters of the objective function L with respect to domain B, L AB Represents some parameters of the objective function L that are common to domains A and B; The gradient descent algorithm is used to optimize the target, assuming The gradient of the loss function of the training parameters is expressed as: Among them, L' A Indicates the gradient descent algorithm for L A Derivatives are performed on the parameters to update the gradient, L' B Indicates the gradient descent algorithm for L B Derivative the parameters to update the gradient; Update the gradient and loss function of the federated learning model. The specific update iterative process includes: ① Each participant aligns the entity information with encrypted entities. Encrypted entity alignment means that the servers of each participant find overlapping entities without knowing each other's information. ② The central server creates an encryption key pair and transmits the public key pair to each participating server, so that each participating server can decrypt the information individually; ③Auxiliary domain A calculation and [[L A ]]Sent to target domain B, target domain B calculates and [[L]], and Send to auxiliary domain A, and send [[L]] to server C; ④ Auxiliary domain A initializes mask R A , calculate [[L' A ]]+[[R A ]]Sent to server C, target domain B initializes R B , calculate [[L' B ]]+[[R B ]] is sent to server C, which decrypts [[L]], [[L' A ]]+[[R A ]] and [[L' B ]]+[[R B ]], L' A +R A Send to A, L' B +R B Send to B; ⑤ After receiving the data from server C, auxiliary domain A and target domain B update the model parameters. Then, repeat steps ③ to ⑤ until the model converges or the number of iterations reaches the upper limit. S4: Use the trained local federated learning model to generate recommendation results for users.

2. The cross-domain recommendation method according to claim 1, characterized in that In step S1, cosine similarity is used to measure the similarity between different users.

3. The cross-domain recommendation method according to claim 1, characterized in that In step S2, the construction of the heterogeneous information network specifically includes: after setting the meta-path, the cold start user features are extracted using the HIN2Vec method. HIN2Vec is a neural network model that can obtain the vector representation of each node; then, an n-dimensional feature vector is randomly generated for each node v in the path to represent the feature set W∈R |v|×n , and judge the relationship between v1, v2 and d, where v1 and v2 are two adjacent nodes on path d, and r (r∈R) represents the relationship between nodes v1 and v2; then judge according to the meta-path discriminant formula in HIN2Vec, where F(v1, v2, r) represents whether there is a relationship r between v1 and v2. If so, the value is 1, and if not, the value is 0; The objective function for learning node features in the meta-path is defined as: L(v1,v2,r)=F(v1,v2,r)log[p(r|(v1,v2)]+(1-F(v1,v2,r))log(1-p(r|(v1,v2)) Where L(v1,v2,r) is the target loss function; F(v1,v2,r) indicates whether there is a relationship r between v1 and v2. If so, the value is 1, and if not, the value is 0; p(r|v1,v2) represents the probability that v1 and v2 have a relationship r. Then, the stochastic gradient descent algorithm is used to optimize the objective function L(v1,v2,r), and the node features are updated according to the HIN2Vec formula; After obtaining the feature representation of the node set, the user-item interaction feature matrix T u,i It is expressed as follows: Among them, T u,i ∈W |v|×n , represents matrix multiplication, represents the user feature weight, Represents the item feature weight, .t represents transposition, Represents the characteristic value of the mth meta-path of the i-th item, m represents the mth meta-path in a meta-path set, and M represents the set of all meta-paths.

4. The cross-domain recommendation method according to claim 1, characterized in that In step S3, the convolutional neural network used includes: Convolutional layer: h = f(ω⊙x+b), where x is the convolutional layer input, h is the convolutional layer output, ω is the convolution kernel, b is the bias, ⊙ is the convolution operation, and f(·) is a nonlinear function; Pooling layer: Maximum pooling layer d f =Max(h); Fully connected layer: d f Convert to a one-dimensional vector Furthermore, based on Input into a multi-layer perceptron model to get the predicted score, which is defined as follows: Among them, w L Partial weight representation, f θ (·) is the nonlinear activation function ReLU, w is the weight of the fully connected layer, b is the bias of the fully connected layer, and b L It is expressed as partial deviation; The model loss function is defined as follows: Where m is the number of samples, r′ ui is the prediction score.

Images