A cross-domain product recommendation method based on three-step cross attention

Abstract

The present invention discloses a cross-domain commodity recommendation method based on three-step cross-attention. The method deeply analyzes the historical behavior data of users through a self-attention mechanism, calculates a weight vector representing the user's interest, improves the accuracy of extracting diversified interests from user behavior, reduces the dependence on prior knowledge, and thus avoids the problem of limiting the generalization ability and practicality of the model due to difficult-to-obtain or inapplicable external information in practical applications; the user behavior feature embedding is carefully adjusted through a three-step cross-attention mechanism, thereby ensuring the effective integration of information in the film field and the commodity field, suppressing irrelevant noise information, and reducing the risk of negative transfer; based on the interest matrix, the overall embedding vector and the key behavior embedding matrix, the interest-level preference transfer bridge function and the overall preference transfer bridge function are obtained, which can better combine the user's personalized preferences with the key behavior features.

Description

Technical Field

[0001] The present invention relates to the field of data analysis technology, and in particular to a cross-domain product recommendation method based on three-step cross-attention. Background Art

[0002] In the field of product recommendation systems, the cold start problem remains a major challenge, especially when new users or new items lack sufficient interaction data. Cross-domain recommendation effectively solves the cold start and data sparsity problems in recommendation systems by migrating user preferences from the source domain to the target domain, significantly improving the quality of recommendations and attracting widespread attention from relevant researchers.

[0003] However, existing cross-domain recommendations have some limitations when dealing with the diversity of user interests and product domain characteristics. There are two main problems: 1) Most methods rely mainly on source domain data to learn user preference representations, and fail to fully utilize information in the product domain to optimize the learning process. Simply migrating user embeddings from one domain to the product domain may introduce noise, leading to negative transfer phenomena; 2) User interests are often multidimensional. Although some methods model users' multiple interests by integrating metadata such as item categories and brands, this method that relies heavily on prior knowledge (specific meta-paths) to extract user interests may be limited in practical applications, because this prior knowledge may be difficult to obtain or inapplicable in some application scenarios, thereby limiting the generalization ability and practicality of the model. Summary of the invention

[0004] The purpose of the present invention is to provide a cross-domain product recommendation method based on three-step cross attention to reduce the risk of negative transfer and improve the generalization ability and practicality of the model.

[0005] A cross-domain product recommendation method based on three-step cross-attention includes steps S1 to S4:

[0006] Step S1, obtaining the user's historical behavior sequence on the movie rating website, inputting the user's historical behavior sequence and user ID into the latent factor model to obtain the user's behavior sequence embedding matrix and overall embedding vector respectively, calculating the weight matrix through the self-attention mechanism, and then obtaining the user's interest matrix in the movie field according to the behavior sequence embedding matrix and the weight matrix;

[0007] Step S2, obtaining the item ID on the product shopping website, inputting the item ID into the latent factor model to obtain the item embedding vector, constructing a three-step item-level cross-attention module, inputting the behavior sequence embedding matrix and the item embedding vector into the three-step item-level cross-attention module, and obtaining the user's key behavior embedding matrix;

[0008] Step S3, based on the interest matrix, the overall embedding vector and the key behavior embedding matrix, obtain a plurality of interest-level preference transfer bridge functions and an overall preference transfer bridge function, and map the interest matrix and the overall embedding vector to the representation space of the commodity domain through these bridge functions to obtain the mapped interest matrix and the mapped overall embedding vector;

[0009] Step S4, using mean aggregation to calculate the mapped interest matrix to obtain the interest transfer embedding vector, using weighted aggregation to calculate the interest transfer embedding vector and the mapped overall embedding vector to obtain the final user transfer embedding, based on the final user transfer embedding, calculate the user's predicted score for the candidate product items, sort the product items according to the predicted score, and obtain a product recommendation list.

[0010] According to the cross-domain commodity recommendation method based on three-step cross attention provided by the present invention, the historical behavior data of the user is deeply analyzed through the self-attention mechanism, and the weight vector representing the user's interest is calculated, thereby improving the accuracy of extracting diversified interests from user behavior and reducing the dependence on prior knowledge, thereby avoiding the problem of limiting the generalization ability and practicality of the model due to difficult to obtain or inapplicable external information in practical applications; the user behavior feature embedding is carefully adjusted through the three-step cross attention mechanism, ensuring the effective integration of information in the film field and the commodity field, suppressing irrelevant noise information, and reducing the risk of negative migration; based on the interest matrix, the overall embedding vector and the key behavior embedding matrix, the interest-level preference transfer bridge function and the overall preference transfer bridge function are obtained, which can better combine the user's personalized preferences with the key behavior characteristics. The present invention can better avoid information loss in the process of user preference migration, has a stronger ability to capture user personalized needs, and can improve the accuracy of commodity recommendation. BRIEF DESCRIPTION OF THE DRAWINGS

[0011] Figure 1 Schematic diagram of the process of the cross-domain product recommendation method based on three-step cross-attention of the present invention. DETAILED DESCRIPTION

[0012] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0013] See also Figure 1 The cross-domain product recommendation method based on three-step cross-attention provided by an embodiment of the present invention includes steps S1 to S4.

[0014] Step S1, obtain the user's historical behavior sequence on the movie rating website, input the user's historical behavior sequence and user ID into the latent factor model to obtain the user's behavior sequence embedding matrix and overall embedding vector respectively, calculate the weight matrix through the self-attention mechanism, and then obtain the user's interest matrix in the film field according to the behavior sequence embedding matrix and the weight matrix.

[0015] Wherein, step S1 specifically includes:

[0016] Step S11, obtain the user's historical behavior sequence on the movie rating website, input the historical behavior sequence and user ID into the latent factor model to obtain the user's behavior sequence embedding matrix and the overall embedding vector , , ,in, represents the sequence length, d represents the vector dimension, represents the set of real numbers;

[0017] Step S12, calculate the weight vector through the self-attention mechanism , , the expression is as follows:

[0018] ;

[0019] Among them, softmax is the normalized exponential function, tanh is the hyperbolic tangent function, and is the trainable parameter matrix, , , is the attention dimension, T represents transposition;

[0020] Step S13, by multiplying the weight vector with the behavior sequence embedding matrix, the user's interest vector representation in the movie field is obtained , , the expression is:

[0021] ;

[0022] Step S14: Expands to a size of The matrix , K Indicates the number of interests, and then the attention vector Transformed into an attention matrix , , the expression is as follows:

[0023] ;

[0024] Step S15, calculate the user's interest matrix in the movie field using the following formula: , :

[0025] .

[0026] As a specific example, a user's behavior sequence is received as input data, which consists of a series of movie project IDs. These movie project IDs are first input into the embedding layer, during which each movie project ID is converted into a corresponding movie project embedding representation. Subsequently, the multi-interest extraction module is used to identify and generate embedding vectors representing the user's diverse interests from the user's behavior data.

[0027] Step S2, obtain the item ID on the product shopping website, input the item ID into the latent factor model to obtain the item embedding vector, build a three-step item-level cross-attention module, input the behavior sequence embedding matrix and the item embedding vector into the three-step item-level cross-attention module, and obtain the user's key behavior embedding matrix.

[0028] Wherein, step S2 specifically includes:

[0029] Step S21, obtain the product item ID on the product shopping website, input the product item ID into the latent factor model to obtain the product item embedding vector ;

[0030] Step S22, construct a three-step item-level cross-attention module to embed the behavior sequence into the matrix and item embedding vector Input to the three-step item-level cross attention module, the expression is:

[0031] ;

[0032] ;

[0033] ;

[0034] ;

[0035] ;

[0036] ;

[0037] ;

[0038] in, represents the historical behavior embedding after the first cross-attention operation, Represents the updated item embedding, represents the cross attention mechanism function, is the learnable parameter matrix, m =1,2,3; m Representative m Calculate the cross attention score, C and h Represent the embedding dimension and the number of attention heads, respectively. represents the query vector, represents the key vector, represents a value vector, and is the input quantity, Embedding matrix for key user behaviors, It contains knowledge in the field of movies and is also suitable for the field of commodities.

[0039] Step S3, based on the interest matrix, the overall embedding vector and the key behavior embedding matrix, obtain multiple interest-level preference transfer bridge functions and an overall preference transfer bridge function, and use these bridge functions to map the interest matrix and the overall embedding vector to the representation space of the commodity field to obtain the mapped interest matrix and the mapped overall embedding vector.

[0040] Wherein, step S3 specifically includes:

[0041] Step S31, calculate user No. n Interest vector Embedding matrix with key behaviors The similarity between them is used to determine the interest weight vector , , , the expression is as follows:

[0042] ;

[0043] Step S32, calculate user No. n Transferable Feature Embedding , , the expression is:

[0044] ;

[0045] Step S33: Input to the shared meta-network To generate the interest-level preference transfer bridge function The parameter vector , , the expression is:

[0046] ;

[0047] in, It is a shared meta-network Parameters;

[0048] Then the parameter vector Reshape into parameter matrix , ;

[0049] Step S34, using the interest-level preference transfer bridge function, obtain the interest transfer embedding in the commodity domain, the expression is as follows:

[0050]

[0051] in, Indicates user In the commodity sector, through n The interest embedding after the preference transfer bridge function at the interest level is processed. , and then based on Get the mapped interest matrix ;

[0052] Step S35, based on user The overall embedding and key behavior embedding matrix The similarity between them determines the overall weight vector , , the expression is as follows:

[0053] ;

[0054] Step S36, calculate user The overall transferable feature embedding , , the expression is:

[0055] ;

[0056] Step S37: Input to the shared meta-network To generate the overall preference transfer bridge function The parameter vector , the expression is:

[0057] ;

[0058] Then the parameter vector Reshape into parameter matrix ;

[0059] Step S38, using the overall preference transfer bridge function, obtain the mapped overall embedding vector , , the expression is as follows:

[0060] .

[0061] Step S4, using mean aggregation to calculate the mapped interest matrix to obtain the interest transfer embedding vector, using weighted aggregation to calculate the interest transfer embedding vector and the mapped overall embedding vector to obtain the final user transfer embedding, based on the final user transfer embedding, calculate the user's predicted score for the candidate product items, sort the product items according to the predicted score, and obtain a product recommendation list.

[0062] Wherein, step S4 specifically includes:

[0063] Step S41, using the mean aggregation method, calculate each vector in the mapped interest matrix to obtain the interest transfer embedding vector , the expression is as follows:

[0064] ;

[0065] Step S42, using weighted aggregation, calculate the interest transfer embedding vector and the mapped overall embedding vector to obtain the final user transfer embedding, which is expressed as follows:

[0066] ;

[0067] in, is the final user transfer embed, , is the balance adjustment parameter;

[0068] Step S43, based on the final user transfer embedding, the user's predicted score for the candidate product item is calculated, and the expression is as follows:

[0069] ;

[0070] in, Is a user For candidate product items 's prediction score;

[0071] Step S44, sorting the commodity items according to the predicted scores to obtain a commodity recommendation list.

[0072] In this embodiment, in step S4, the loss function in the process of calculating the user's predicted rating of the candidate product item is calculated based on the final user transfer embedding. L for:

[0073] ;

[0074] in, min Indicates taking the minimum value, is the set of interactions between overlapping users and product items in the product domain, express The total number of elements in , yes The elements in Is a user For candidate product items By minimizing and The difference between them can optimize the prediction accuracy.

[0075] In order to prove the feasibility and superiority of the present invention, the following comparative experiments were carried out.

[0076] Experiment 1:

[0077] The experiment used the Amazon review dataset for algorithm testing. The dataset covers 24 categories. This experiment selected three popular categories: Movies and TV, Music (CDs and Vinyl), and Books. Based on these categories, three cross-domain recommendation tasks were constructed: Movies → Music (Task 1), Books → Movies (Task 2), and Books → Music (Task 3). The Amazon review dataset contains rating data (0 - 5 points). Mean Absolute Error (MAE) and Root Mean Square Error (RMSE) are used as metrics. In order to evaluate the performance of the present invention, the MF model (TGT) trained using only data from the target domain, Relational learning via collective matrix factorization (CMF), Cross-Domain Recommendation: An Embedding and Mapping Approach (EMCDR), A deep framework for cross-domain and cross-system recommendations (DCDCSR), Semi-supervised learning for cross-domain recommendation to cold-start users (SSCDR), Personalized Transfer of User Preferences for Cross-domain Recommendation (PTUPCDR), and Reinforced Multi-Interest Transfer for Cross-Domain Recommendation (REMIT) are selected as the baselines for comparison. All ratings of a small number of overlapping users in the target domain are randomly deleted and regarded as test users, and samples of other overlapping users are used to train the interest bridge function. In the experiment, the proportion of test (cold start) users is set to β The overlap ratios are set to 80%, 50%, and 20% of the total number of overlapping users, respectively. For each task, the average results of five random runs are reported. The experimental results are shown in Table 1:

[0078] Table 1

[0079]

[0080] As shown in Table 1, as the proportion of cold start users increases β With the improvement of the model, the mean absolute error (MAE) and root mean square error (RMSE) of the model both show an upward trend, which is mainly due to the larger β The value means that the amount of data used for training is reduced. In contrast, the present invention achieves efficient cross-domain migration of user preferences by comprehensively considering the diverse interests of users in the source domain and the characteristics of the target domain, thereby showing good performance in multiple cross-domain recommendation tasks.

[0081] Experiment 2:

[0082] This experiment selects REMIT as the baseline and proposes two model variants: 1) w / oCA (Remove the three-step item-level cross-attention module): Directly use the user's behavior sequence embedding matrix in the interest vector mapping task Generate an attention distribution for each query vector instead of using a key behavior embedding matrix that contains both source domain knowledge and is suitable for the target domain 2) w / oMI (Remove the multi-interest extraction module): Only use the single-interest meta-network to map the overall embedding of the user without additionally using the multi-interest transfer meta-network to transfer user preferences. β Under the setting, the test was conducted on Task 1 of Experiment 1. The experimental results are shown in Table 2:

[0083] Table 2

[0084]

[0085] As shown in Table 2, when the item-level cross attention (CA) module is removed, the contextual information of the target domain cannot be effectively utilized when processing the interaction between the user historical behavior embedding and the target domain item embedding. The lack of this information leads to a significant drop in the performance of the model when predicting the user's interest in the target domain items. This result emphasizes that the information of the target domain is crucial for optimizing the migration of user preferences. In addition, it is observed that when the multi-interest extraction (MI) module is removed and only a single interest meta-network is used to map the user embedding, the performance of the model is also affected. This phenomenon shows that the user's interests are multi-dimensional, and a single interest meta-network cannot fully capture the user's diverse interests in the source domain, thereby limiting the improvement of the model's recommendation accuracy in the target domain. Therefore, the results of Experiment 2 show that the item-level cross attention module and the multi-interest extraction module are the key factors for the performance improvement of the present invention. The combination of these two modules enables the model to make full use of the information of the source and target domains, realize the effective migration of user preferences, and thus achieve excellent performance in cross-domain recommendation tasks, which further confirms the effectiveness of the present invention.

[0086] In summary, according to the cross-domain commodity recommendation method based on three-step cross attention provided by the present invention, the historical behavior data of the user is deeply analyzed through the self-attention mechanism, and the weight vector representing the user's interest is calculated, which improves the accuracy of extracting diversified interests from user behavior and reduces the dependence on prior knowledge, thereby avoiding the problem of limiting the generalization ability and practicality of the model due to difficult to obtain or inapplicable external information in practical applications; the user behavior feature embedding is carefully adjusted through the three-step cross attention mechanism to ensure the effective integration of information in the film field and the commodity field, suppress irrelevant noise information, and reduce the risk of negative migration; based on the interest matrix, the overall embedding vector and the key behavior embedding matrix, the interest-level preference transfer bridge function and the overall preference transfer bridge function are obtained, which can better combine the user's personalized preferences with the key behavior characteristics. The present invention can better avoid the information loss in the process of user preference migration, has a stronger ability to capture user personalized needs, and can improve the accuracy of commodity recommendation.

[0087] In the description of this specification, the description with reference to the terms "one embodiment", "some embodiments", "example", "specific example", or "some examples" means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present invention. In this specification, the schematic representation of the above terms does not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described can be combined in any one or more embodiments or examples in a suitable manner.

[0088] Although the embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions and variations may be made to the embodiments without departing from the principles and spirit of the present invention, and that the scope of the present invention is defined by the claims and their equivalents.

Claims

1. A cross-domain product recommendation method based on three-step cross attention, characterized in that: It includes steps S1 to S4: Step S1, obtaining the user's historical behavior sequence on the movie rating website, inputting the user's historical behavior sequence and user ID into the latent factor model to obtain the user's behavior sequence embedding matrix and overall embedding vector respectively, calculating the weight matrix through the self-attention mechanism, and then obtaining the user's interest matrix in the movie field according to the behavior sequence embedding matrix and the weight matrix; Step S2, obtaining the item ID on the product shopping website, inputting the item ID into the latent factor model to obtain the item embedding vector, constructing a three-step item-level cross-attention module, inputting the behavior sequence embedding matrix and the item embedding vector into the three-step item-level cross-attention module, and obtaining the user's key behavior embedding matrix; Step S3, based on the interest matrix, the overall embedding vector and the key behavior embedding matrix, obtain a plurality of interest-level preference transfer bridge functions and an overall preference transfer bridge function, and map the interest matrix and the overall embedding vector to the representation space of the commodity domain through these bridge functions to obtain the mapped interest matrix and the mapped overall embedding vector; Step S4, using mean aggregation to calculate the mapped interest matrix to obtain the interest transfer embedding vector, using weighted aggregation to calculate the interest transfer embedding vector and the mapped overall embedding vector to obtain the final user transfer embedding, based on the final user transfer embedding, calculate the user's predicted score for the candidate product items, sort the product items according to the predicted score, and obtain a product recommendation list; Step S2 specifically includes: Step S21, obtain the product item ID on the product shopping website, input the product item ID into the latent factor model to obtain the product item embedding vector ; Step S22, construct a three-step item-level cross-attention module to embed the behavior sequence into the matrix and item embedding vector Input to the three-step item-level cross attention module, the expression is: ； ； ； ； ； ； ； in, represents the historical behavior embedding after the first cross-attention operation, Represents the updated item embedding, represents the cross attention mechanism function, is the learnable parameter matrix, represents the set of real numbers; m Represents the number of times the cross attention score is calculated, m =1,2,3; C and h Represent the embedding dimension and the number of attention heads, respectively. represents the query vector, represents the key vector, represents a value vector, and is the input quantity, Embedding matrix for key user behaviors, , softmax is the normalized exponential function, and T represents transpose.

2. The cross-domain product recommendation method based on three-step cross-attention according to claim 1 is characterized in that: Step S1 specifically includes: Step S11, obtain the user's historical behavior sequence on the movie rating website, input the historical behavior sequence and user ID into the latent factor model to obtain the user's behavior sequence embedding matrix and the overall embedding vector , , ,in, represents the sequence length, d Represents the vector dimension; Step S12, calculate the weight vector through the self-attention mechanism , , the expression is as follows: ; Among them, tanh is the hyperbolic tangent function, and is the trainable parameter matrix, , , is the attention dimension; Step S13, by multiplying the weight vector with the behavior sequence embedding matrix, the user's interest vector representation in the movie field is obtained , , the expression is: ； Step S14: Expands to a size of The matrix , K Indicates the number of interests, and then the attention vector Transformed into an attention matrix , , the expression is as follows: ； Step S15, calculate the user's interest matrix in the movie field using the following formula: , : 。 3. The cross-domain product recommendation method based on three-step cross-attention according to claim 2 is characterized in that: Step S3 specifically includes: Step S31, calculate user No. n Interest vector Embedding matrix with key behaviors The similarity between them is used to determine the interest weight vector , , , the expression is as follows: ； Step S32, calculate user No. n Transferable Feature Embedding , , the expression is: ； Step S33: Input to the shared meta-network To generate the interest-level preference transfer bridge function The parameter vector , , the expression is: ； in, It is a shared meta-network Parameters; Then the parameter vector Reshape into parameter matrix , ; Step S34, using the interest-level preference transfer bridge function, obtain the interest transfer embedding in the commodity domain, the expression is as follows: in, Indicates user In the commodity sector, through n The interest embedding after the preference transfer bridge function at the interest level is processed. , and then based on Get the mapped interest matrix ; Step S35, based on user The overall embedding and key behavior embedding matrix The similarity between them determines the overall weight vector , , the expression is as follows: ； Step S36, calculate user The overall transferable feature embedding , , the expression is: ； Step S37: Input to the shared meta-network To generate the overall preference transfer bridge function The parameter vector , the expression is: ； Then the parameter vector Reshape into parameter matrix ; Step S38, using the overall preference transfer bridge function, obtain the mapped overall embedding vector , , the expression is as follows: 。 4. The cross-domain product recommendation method based on three-step cross-attention according to claim 3 is characterized in that: Step S4 specifically includes: Step S41, using the mean aggregation method, calculate each vector in the mapped interest matrix to obtain the interest transfer embedding vector , the expression is as follows: ； Step S42, using weighted aggregation, calculate the interest transfer embedding vector and the mapped overall embedding vector to obtain the final user transfer embedding, which is expressed as follows: ； in, is the final user transfer embed, , is the balance adjustment parameter; Step S43, based on the final user transfer embedding, the user's predicted score for the candidate product item is calculated, and the expression is as follows: ； in, Is a user For candidate product items 's prediction score; Step S44, sorting the commodity items according to the predicted scores to obtain a commodity recommendation list.

5. The cross-domain product recommendation method based on three-step cross-attention according to claim 4 is characterized in that: In step S4, the loss function in the process of calculating the user's predicted rating of the candidate product item is calculated based on the final user transfer embedding L for: ； in, min Indicates taking the minimum value, is the set of interactions between overlapping users and product items in the product domain, express The total number of elements in , yes The elements in Is a user For candidate product items The real rating.

Images