An efficient adaptive secure cross-domain recommendation method

Abstract

The application belongs to the field of cross-domain, cross-system user interest prediction and recommendation, and particularly relates to an efficient adaptive safe cross-domain recommendation method, which comprises the following steps: obtaining information of users and items, inputting the information of the users and the items into a trained cross-domain recommendation model to obtain a rating result of the users to the items, and recommending the items to the users after sorting the items according to the rating result; the cross-domain recommendation model comprises a modeling domain-independent user preference module, a JLT differential privacy encryption module, a domain relevance extraction module and a domain adaptation module; the application guarantees the realization of domain pluggability through the private guarantee of the domain-independent user preference representation and the autoencoder, and only needs to dynamically fine-tune the model when a new domain is added or an old domain is exited; the application introduces a domain relevance coefficient to adjust the loss of the discriminator, overcomes the problem of large domain span, and further improves the accuracy of the cross-domain recommendation method.

Description

Technical Field

[0001] This invention belongs to the field of cross-domain and cross-system user interest prediction and recommendation, specifically involving an efficient adaptive and secure cross-domain recommendation method. Background Technology

[0002] Personalized recommendation systems are now applied across various fields. It's common for different fields to exhibit a degree of homogeneity in functionality and information provision; for example, a user who likes rock music might prefer science fiction movies. Therefore, methods have emerged to leverage knowledge from information-rich fields to improve the performance of recommendation systems in information-sparse fields—this is known as cross-domain recommendation.

[0003] Compared to traditional recommender systems, cross-domain recommender systems are more complex. First, they need to consider the heterogeneity of user and item data across different domains, as well as the existence of overlapping user scenarios. Second, cross-domain recommender systems not only need to model user interests based on user behavior, but also need to consider knowledge transfer between different domains and the security issues during the transfer process. Finally, cross-domain recommender systems need to consider how to integrate knowledge from the source and target domains to maximize the recommendation performance of the target domain.

[0004] Cross-domain recommendation involves different fields and service providers. The data distribution across different fields can influence each other. For example, if a user recently watched a movie they were very interested in, they might develop an interest in the movie's theme song or background music. Therefore, the current assumption that cross-domain data is independent and identically distributed is untenable. Furthermore, different service providers model user interests differently for different projects. Considering these two points, it is essential to consider data heterogeneity in cross-domain recommendation.

[0005] Existing cross-domain recommendation methods mainly employ multi-task learning, transfer learning, clustering, reinforcement learning, deep neural networks, relation learning, and semi-supervised learning. The paper "Differential Private Knowledge Transfer for Privacy-Preserving Cross-Domain Recommendation" proposes a model combining a deep transfer-based autoencoder and differential privacy to achieve a secure cross-domain recommendation model.

[0006] Although a great deal of research has been conducted on secure cross-domain recommendation models and considerable progress has been made, some challenges remain:

[0007] 1. The differences in the number and characteristics of user-item interactions in the source and target domains, as well as the sparsity of user interactions, pose significant challenges to the fusion of knowledge vectors from different dimensions in the source and target domains.

[0008] 2. During the knowledge transfer process, how can we ensure that the published knowledge vectors retain as many features of the original data as possible, while protecting user privacy and corporate data security?

[0009] 3. How to improve the accuracy of cross-domain recommendation methods when the domain span is large;

[0010] 4. Given encrypted raw data, how to obtain domain relevance and securely deliver relevant information to the target domain without increasing model complexity?

[0011] 5. When new domains are introduced, how can we improve the flexibility of the model and reduce its training cost? Summary of the Invention

[0012] To address the aforementioned technical problems, this invention employs an efficient adaptive secure cross-domain recommendation method, comprising: acquiring user and item information; inputting the user and item information into a trained cross-domain recommendation model to obtain user ratings of items; and ranking the items according to the ratings and recommending them to the user. The cross-domain recommendation model includes: a module for modeling domain-independent user preferences, a JLT differential privacy encryption module, a domain relevance extraction module, and a domain adaptation module.

[0013] The training process for cross-domain recommendation models includes:

[0014] S1. Obtain training data and input it into the domain-independent user preference modeling module to obtain the pre-release matrix R; the pre-release matrix R includes the source domain pre-release matrix R. d and target domain pre-release matrix R s ;

[0015] S2. Input the pre-release matrix R into the JLT differential privacy encryption module to obtain the release matrix. Release Matrix Including source domain publishing matrix and target domain publishing matrix

[0016] S3. Input the pre-release matrix R into the domain relevance extraction module to obtain the domain relevance coefficient;

[0017] S4, Matrix to be released Input the domain adaptation module to obtain the discrimination result and the prediction result.

[0018] S5. Calculate the loss function value based on the domain relevance coefficient, discrimination result, and prediction result. Update the model parameters based on the loss function value. When the loss function value is minimized, the trained cross-domain recommendation model is obtained.

[0019] The domain-specific user preference module for modeling includes autoencoders from various domains; the domain-specific user preference module processes training data including:

[0020] S11. Clean the training data to obtain the user unique identifier field, the item unique identifier field, and the user's rating information for the item. Construct a rating matrix S for each domain based on the user unique identifier field, the item unique identifier field, and the user's rating information for the item.

[0021] S12. Input the scoring matrix S of each domain into its respective autoencoder to obtain the pre-release matrix R. d .

[0022] The JLT differential privacy encryption module processes the pre-release matrix R as follows:

[0023] S21. Perturb the eigenvalues ​​of the pre-release matrix R to obtain the perturbation matrix;

[0024] S22. The perturbation matrix is ​​randomly transformed using the JLT algorithm to obtain the release matrix. The JLT is the Johnson-Lindenstrauss transform.

[0025] Eigenvalue perturbations on the pre-release matrix R include:

[0026] S211. Calculate the average value of the pre-release matrix R, and subtract the average value from the pre-release matrix R to obtain the new pre-release matrix;

[0027] S212, Calculate the eigenvalue perturbation p;

[0028] S213. Calculate the eigenvalues ​​of the new pre-release matrix and construct a diagonal matrix Σ based on the eigenvalues.

[0029] S214. Modify the eigenvalues ​​of the new pre-release matrix based on the diagonal matrix Σ and the eigenvalue perturbation p to obtain the release matrix.

[0030] Modifying the eigenvalues ​​of the new pre-release matrix based on the diagonal matrix Σ and the eigenvalue perturbation p involves: calculating new eigenvalues ​​based on the diagonal matrix Σ and the eigenvalue perturbation p, and replacing the eigenvalues ​​of the new pre-release matrix with the new eigenvalues ​​to obtain the release matrix.

[0031] The domain relevance extraction module processes the pre-release matrix R in the following ways:

[0032] S31. Perform matrix decomposition on the pre-release matrix R to obtain the user feature matrix and the item feature matrix;

[0033] S32. Perform k-means clustering on the user feature matrix to obtain the source domain user set and the target domain user set; both the source domain user set and the target domain user set include multiple sub-user sets; each sub-user set represents users with similar interests and preferences;

[0034] S33. Calculate the domain relevance coefficient between the source domain and the target domain based on the source domain user set and the target domain user set.

[0035] The calculation of the domain relevance coefficient between the source domain and the target domain includes: calculating the similarity between each sub-user set in the source domain user set and each sub-user set in the target domain user set using the Jaccard similarity algorithm; calculating the average similarity between the source domain and the target domain based on the similarity of the sub-user sets; and normalizing the average similarity to obtain the domain relevance coefficient between the target domain and the source domain.

[0036] The domain adaptation module includes: an adversarial network, a gradient inversion layer, a domain discriminator, and a classifier; the domain adaptation module publishes a source domain matrix. and target domain publishing matrix The processing includes:

[0037] S41. Publish the matrix in the source domain. and target domain publishing matrix Add a domain identifier to each row vector to obtain the potential secret vector v;

[0038] S42. Input the potential secret vector v into the adversarial network;

[0039] S43. Input the output of the adversarial network into the gradient inversion layer and the classifier respectively to obtain the result after forward propagation of the gradient inversion layer and the prediction result.

[0040] S44. Input the result of the gradient inversion layer after forward propagation into the neighborhood discriminator to obtain the discrimination result.

[0041] The initial weights of the classifier are the weights of the autoencoder in the target domain.

[0042] The loss function is:

[0043] L=min(L c +L d )

[0044]

[0045] Among them, L c Let L be the loss function of the classifier. dLet w' be the loss function of the domain discriminator. dj This represents the domain relevance coefficient between the source and target domains. This indicates that user u is in the source domain d. j The domain label of the vector. Let U be the discrimination result of the discriminator, D be the set of users, φ be the set of source domains, and φ be the identity function of the gradient inversion layer.

[0046] Beneficial effects:

[0047] 1. This invention uses an autoencoder to encode and decode the rating matrix, avoiding direct publication of the rating matrix and unifying the dimensions of user representation; 2. This invention perturbs and encrypts the pre-published matrix after autoencoder encoding to prevent model theft attacks; 3. This invention ensures pluggable domain implementation through domain-independent user preference representation and autoencoder privatization. When a new domain is added or an old domain is removed, only the correlation coefficient between the new domain and other domains needs to be calculated, and then the model can be dynamically fine-tuned based on the new domain correlation coefficient, improving the model's flexibility and reducing the model's training cost; 4. This invention introduces a domain correlation coefficient to adjust the discriminator's loss, overcoming the problem of large domain spans and further improving the accuracy of this cross-domain recommendation method. Attached Figure Description

[0048] Figure 1 The diagram illustrates the structure of an efficient adaptive secure cross-domain recommendation method provided in this embodiment of the invention.

[0049] Figure 2 The diagram shows the structure of an autoencoder for an efficient adaptive secure cross-domain recommendation method provided in this embodiment of the invention.

[0050] Figure 3 The diagram shows the structure of the domain relevance extraction module in an efficient adaptive secure cross-domain recommendation method provided in this embodiment of the invention.

[0051] Figure 4 This is a structural diagram of the domain adaptation module of an efficient adaptive secure cross-domain recommendation method provided in an embodiment of the present invention. Detailed Implementation

[0052] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0053] like Figure 1As shown, the input of this invention is a user rating matrix, and the output after model prediction is the user's rating of the target domain items. Based on this result, the items are sorted and recommended to the corresponding users.

[0054] This invention comprises the following four steps:

[0055] S1: Modeling Domain-Independent User Preference Representations: Leveraging the powerful feature extraction capabilities of autoencoders and their ability to unify feature dimensions, this invention designs an autoencoder that conforms to the application scenario of this invention; the autoencoder compresses the rating matrices of different domains to the same dimension and obtains domain-independent user preference representations, facilitating domain-specific pluggability.

[0056] like Figure 2 As shown, modeling domain-independent user preference representation mainly involves the following two steps:

[0057] S11: Obtain raw data from a public dataset, then clean the raw data to obtain the user unique identifier field, the item unique identifier field, and the user's rating information for the item. Construct a user-item rating matrix based on these three fields, denoted as S; where... m is the number of users, and n is the number of items;

[0058] S12: An autoencoder (AutoED) consists of an encoder and a decoder; the training process mainly consists of the following three steps:

[0059] S121: Use the encoder to generate the scoring matrix of domain i Each vector is mapped to a unified low-dimensional latent vector space, resulting in the pre-release matrix R. i , represented as:

[0060] R i =Encoder(S i )

[0061] in, This represents the pre-release matrix, where k represents the uniform dimension size of the embeddings across different domains, and n... i Let i be the number of items in domain i.

[0062] S122: Recover the scoring matrix using the decoder:

[0063]

[0064] in, This represents the restored score matrix.

[0065] S123: The autoencoder loss La can be expressed as:

[0066]

[0067] Among them, y u This represents user u in the original rating matrix S. i The vector, The rating matrix representing the recovery of user u in this domain. The vector; each domain obtains AutoED with higher recommendation accuracy by minimizing the mean square loss La.

[0068] S13: Private AutoED. After training your own AutoED for each domain, you can use Decoder to initialize the prediction classifier and then save your own AutoED for subsequent encoding and prediction.

[0069] This invention privatizes the autoencoders in various fields, avoiding the direct publication of the scoring matrix, and to a certain extent guarantees the privacy and security of user and enterprise data.

[0070] S2: JLT Differential Privacy Encryption: Perturbs and encrypts the pre-release matrix encoded by the autoencoder to prevent model theft attacks.

[0071] The JLT differential privacy encryption in step S2 above mainly consists of the following two steps.

[0072] S21: Eigenvalue perturbation of the pre-release matrix; through eigenvalue perturbation, a perturbation matrix with the same eigenvectors can be obtained, but its spectrum is different from that of the pre-release matrix; therefore, perturbing the eigenvalues ​​can achieve privacy protection and can maintain utility by keeping the eigenvectors unchanged; the main steps include:

[0073] S211: Subtract the average value from the pre-release matrix R to obtain the new pre-release matrix;

[0074] S212: Calculate the eigenvalue perturbation p; Compared to the original JLT differential privacy, this invention does not require calculating the compression dimension because the pre-release matrix is ​​already the matrix after the scoring matrix is ​​compressed by an autoencoder;

[0075] S213: Calculate the eigenvalues ​​of the new pre-release matrix and construct a diagonal matrix Σ based on the eigenvalue elements;

[0076] S214: Through Modify the eigenvalues ​​of the new pre-release matrix, where σ i =Σ ii ;

[0077] S22: By performing a random transformation on the perturbed matrix using JLT, appropriate randomness is added to the pre-release matrix to obtain the release matrix. JLT can achieve restricted equivalence properties, which preserves user characteristics by transforming the user's pre-published matrix, while also protecting user privacy through differential privacy.

[0078] Release Matrix The pre-release matrix R and the pre-release matrix are adjacent matrices. For adjacent matrices, R = [r ij ] n×k and The following relationship must be satisfied:

[0079]

[0080] Where n and k represent the row and column of the matrix, r ij The i-th row and j-th column of the pre-release matrix R, For publishing matrix Let i be the row and j be the first row and j be the first column.

[0081] S3: Domain Relevance Extraction: First, cluster users from different domains. Then, traverse the user category sets in different domains. Divide the intersection of users in different category sets by the union to obtain the similarity between the two user sets. Finally, calculate the average of the similarity of all user sets to obtain the similarity between the two domains.

[0082] The above step S3 domain relevance extraction, such as Figure 3 As shown, it mainly consists of the following two steps.

[0083] S31: Perform matrix decomposition on the pre-release matrix R to obtain the dense user feature matrix and item feature matrix respectively;

[0084] S32: Utilize user similarity to measure domain similarity. This mainly consists of the following four steps:

[0085] S321: First, k-means clustering is applied to the user feature matrices of the source domain and the target domain respectively to obtain the user set of the source domain and the user set of the target domain respectively; both the user set of the source domain and the user set of the target domain include multiple sub-user sets; each sub-user set represents users with similar interests and preferences.

[0086] S322: Calculate the similarity between each source domain user set and the target domain user set;

[0087] Use Jaccard similarity to measure the similarity between two sets:

[0088]

[0089] J(A,B) represents the similarity between the set of sub-users of source domain category A and the set of sub-users of target domain category B.

[0090] S323: Calculate the average of the Jaccard results for all sub-user sets to obtain the total similarity between the source and target domains.

[0091]

[0092] Where k is the number of sub-user sets in the source domain user set, and T a For source domain d j The a-th subset of the user set, S b Let b be the b-th subset of the target domain user set.

[0093] S324: Finally, normalization is performed to obtain the correlation coefficient vector of the target domain.

[0094]

[0095] in, For the source domain d j The correlation coefficient with the target domain, where D represents the number of source domains, d j This represents the j-th source domain.

[0096] S4: Target Domain User Preference Prediction; predicting user preferences based on domain similarity vectors. Specifically includes:

[0097] S41: Domain adaptation module, the domain adaptation module is as follows Figure 4 As shown, it consists of an adversarial network f(·), a gradient inversion layer GRL, a neighborhood discriminator Dis(·), and a classifier Cls(·). The training process mainly consists of the following 7 steps.

[0098] S411: Obtain Input: The inputs are the source domain publication matrix. and target domain publishing matrix

[0099] S412: Add an identifier: Publish the matrix to the source domain in the target domain space. and target domain publishing matrix Each row vector is labeled with a domain identifier (Flag), where 0 represents the target domain and the remaining numbers indicate different source domains. This is used to match the corresponding correlation coefficients from the correlation coefficient vector. We obtain the user's secret vector v in their domain, making it impossible to deduce the user's true rating matrix S.

[0100] S413: Input the labeled latent secret vector v into the adversarial network f(·), which is a fully connected network with parameter set θ. f .

[0101] S414: The discriminator is a model used to determine the neighborhood of an input vector. The output of the adversarial network is input into the gradient inversion layer and the classifier, respectively. The result of the forward propagation of the gradient inversion layer is input into the discriminator. The neighborhood of the input vector is predicted based on the output of the discriminator, and the loss Ld is defined.

[0102]

[0103] in, This indicates that user u is in the j-th source domain d j The domain label of the vector. Let U be the discriminant result, where U is the user set and D is the source domain set.

[0104] S415: The classifier Cls(·) is initialized by a decoder pre-trained in the target domain and is used to predict user ratings of item interactions in the target domain; the classifier loss can be expressed as follows:

[0105]

[0106] Among them, y u,s For user u, the actual rating in the target domain s. The prediction score is given to the classifier.

[0107] S416: To further improve recommendation accuracy, this invention introduces domain relevance by alternately training the adversarial network by maximizing the discriminator loss Ld and minimizing the classifier loss Lc. The extracted domain relevance coefficient is used to coordinate the proportion of the discriminator loss in the total loss, thereby endowing the model with the ability to determine domain relevance. This allows the latent secret vector of the source domain to be... A better simulation of the target domain vector v u,s .

[0108] S417: The total loss of the domain adaptation module can be expressed as:

[0109]

[0110] Where, θ c θ is the parameter of the classifier. d These are the parameters of the discriminator.

[0111] To address the aforementioned training problem using a method similar to stochastic gradient descent (SGD), this invention introduces a gradient inversion layer (GRL). The GRL is a simple and easily understood structure. In forward propagation, it simply acts as an identity function to allow data to pass through. However, in backward propagation, it can compute gradients from subsequent network layers, multiply by -μ, and then pass them to the previous layer. Therefore, the GRL can be represented as:

[0112] φ(x) = x(forward propagation)

[0113]

[0114] Where x is the input data of the gradient inversion layer, φ(x) is the identity function of the gradient inversion layer, I is the identity matrix, and μ is the hyperparameter.

[0115] Therefore, the total loss of the domain adaptation module can be modified as follows:

[0116]

[0117] Where φ is the identity function of the gradient inversion layer.

[0118] When a new domain is added, this invention does not require retraining the model separately for each new domain. Instead, it only needs to calculate the correlation coefficient between the new domain and other domains, update the original correlation coefficient, and use the new correlation coefficient to adjust and train the model. This achieves very good benefits in terms of flexibility and training cost.

[0119] The above-described embodiments further illustrate the purpose, technical solution, and advantages of the present invention. It should be understood that the above-described embodiments are merely preferred embodiments of the present invention and are not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc., made to the present invention within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. An efficient adaptive secure cross-domain recommendation method, characterized in that, include: The system acquires user and item information, inputs this information into a trained cross-domain recommendation model, obtains user ratings for items, and then sorts and recommends items to users based on these ratings. The cross-domain recommendation model includes: a module for modeling domain-independent user preferences, a JLT differential privacy encryption module, a domain relevance extraction module, and a domain adaptation module; The training process for cross-domain recommendation models includes: S1. Obtain training data. Input the training data into the domain-independent user preference modeling module to obtain the pre-release matrix. Pre-release matrix Including source domain pre-release matrix and target domain pre-release matrix ; The domain-specific user preference module for modeling includes autoencoders from various domains; the domain-specific user preference module processes training data including: S11. Clean the training data to obtain the user unique identifier field, the item unique identifier field, and the user rating information for the items. Construct a rating matrix for each domain based on the user unique identifier field, the item unique identifier field, and the user rating information for the items. ; S12, Calculate the rating matrix for each domain. Input their respective autoencoders to obtain the pre-release matrix. ; S2, Pre-release matrix Inputting each value into the JLT differential privacy encryption module yields the publication matrix. Release Matrix Including source domain publishing matrix and target domain publishing matrix ; S3, Pre-release matrix Input the domain relevance extraction module to obtain the domain relevance coefficient; S4, Matrix to be released The input domain adaptation module yields discrimination and prediction results. The domain adaptation module includes: an adversarial network, a gradient inversion layer, a domain discriminator, and a classifier; the domain adaptation module publishes a source domain matrix. and target domain publishing matrix The processing includes: S41. Publish the matrix in the source domain. and target domain publishing matrix Add a domain identifier to each row vector to obtain the potential secret vector. ; S42, Transfer the potential secret vector Input into the adversarial network; S43. Input the output of the adversarial network into the gradient inversion layer and the classifier respectively to obtain the result after forward propagation of the gradient inversion layer and the prediction result. S44. Input the result of the gradient inversion layer after forward propagation into the neighborhood discriminator to obtain the discrimination result; S5. Calculate the loss function value based on the domain relevance coefficient, discrimination result and prediction result, update the model parameters based on the loss function value, and obtain the trained cross-domain recommendation model when the loss function value is minimized. The loss function is: in, Let the loss function of the classifier be . Let the loss function be that of the neighborhood discriminator. is the domain relevance coefficient between the source domain and the target domain. Indicates user u in the source domain The domain label of the vector. The discriminant's result is given by U, where U is the user set and D is the source domain set. This is the identity function for the gradient inversion layer.

2. The efficient adaptive secure cross-domain recommendation method according to claim 1, characterized in that, JLT differential privacy encryption module for pre-release matrix The processing includes: S21, Regarding the pre-release matrix The perturbation matrix is ​​obtained by performing eigenvalue perturbation; S22. The perturbation matrix is ​​randomly transformed using the JLT algorithm to obtain the release matrix. The JLT is the Johnson-Lindenstrauss transform.

3. The efficient adaptive secure cross-domain recommendation method according to claim 2, characterized in that, For the pre-release matrix Eigenvalue perturbations include: S211, Calculate the pre-release matrix The average value in the pre-release matrix Subtracting the average from the mean yields the new pre-release matrix; S212, Calculate eigenvalue perturbation ; S213. Calculate the eigenvalues ​​of the new pre-release matrix and construct a diagonal matrix based on the eigenvalues. ; S214, Based on the diagonal matrix and eigenvalue perturbation Modify the eigenvalues ​​of the new pre-release matrix to obtain the release matrix. .

4. The efficient adaptive secure cross-domain recommendation method according to claim 3, characterized in that, According to the diagonal matrix and eigenvalue perturbation Modifying the eigenvalues ​​of the new pre-release matrix includes: based on the diagonal matrix and eigenvalue perturbation Calculate the new eigenvalues, replace the eigenvalues ​​of the new pre-release matrix with the new eigenvalues, and obtain the release matrix. .

5. The efficient adaptive secure cross-domain recommendation method according to claim 1, characterized in that, Domain relevance extraction module for pre-release matrix The processing includes: S31, Regarding the pre-release matrix Perform matrix decomposition to obtain the user feature matrix and the item feature matrix; S32. Perform k-means clustering on the user feature matrix to obtain the source domain user set and the target domain user set; both the source domain user set and the target domain user set include multiple sub-user sets; the sub-user sets represent users with similar interests and preferences. S33. Calculate the domain relevance coefficient between the source domain and the target domain based on the source domain user set and the target domain user set.

6. The efficient adaptive secure cross-domain recommendation method according to claim 5, characterized in that, The calculation of the domain relevance coefficient between the source domain and the target domain includes: calculating the similarity between each sub-user set in the source domain user set and each sub-user set in the target domain user set using the Jaccard similarity algorithm; calculating the average similarity between the source domain and the target domain based on the similarity of the sub-user sets; and normalizing the average similarity to obtain the domain relevance coefficient between the target domain and the source domain.

7. The efficient adaptive secure cross-domain recommendation method according to claim 1, characterized in that, The initial weights of the classifier are the weights of the autoencoder in the target domain.

Images