Multi-Preference Collaborative Filtering Recommendation System with Co-Attention Memory Mechanism

Abstract

The present invention discloses a multi-preference collaborative filtering recommendation system with a co-attention memory mechanism, including an interaction information acquisition module, a user feature extraction module, a user latent preference extraction module, a user preference filtering module, and a recommendation module; the present invention captures high-level information of user interactions through a memory module, and extracts the latent interaction preferences of users by decomposing and reorganizing the edges in multiple graphs.

Description

Technical Field

[0001] The present invention relates to the field of data mining, and specifically to a multi-preference collaborative filtering recommendation system with a co-attention memory mechanism. Background Art

[0002] With the increase in the variety and quantity of items, there is a serious information overload problem in current recommendation systems. How to quickly and effectively select items that users may be interested in has become a key issue for enterprise development. A recommendation system (RS) can infer users' interests and hobbies through users' historical behaviors and help users obtain the information they need, which can effectively alleviate the problem of information overload. This system has become an important tool in other application fields, such as information retrieval, tourism, management science, approximation theory, and prediction theory. It can bring huge commercial value, economic value, and service value to enterprises.

[0003] Among many recommendation algorithms, collaborative filtering (CF) has attracted wide attention due to its simplicity and efficiency. Its basic idea is that users who have purchased similar items in the past tend to purchase similar items in the future. In recent years, many researchers have begun to apply graph convolutional networks (GCNs) to collaborative filtering. GCNs are deep learning models for graph-structured data. They have powerful feature extraction and representation learning capabilities and can capture graph dependencies through message passing between graph nodes. Essentially, user-item interactions can be naturally modeled as a user-item bipartite graph. Therefore, graph convolutional collaborative filtering has become a popular method in current recommendation systems.

[0004] The most popular method in collaborative filtering is matrix factorization (MF). However, since most of its data is collected from users' implicit feedback, data sparsity has become an inherent challenge for this method. This makes it difficult to design effective algorithms: most users have few interactions with the system, and there is not enough data available for learning.

[0005] Existing collaborative filtering has the following defects: First, different users have different interaction preferences. For example, some users like high cost performance, while other users like eye-catching appearances. At the same time, the preferences for a commodity can be summarized from the characteristics of the users who purchased the commodity. This information is contained in users' interaction behaviors but cannot be represented by marginal information. Therefore, how to obtain this information remains a meaningful topic. Second, recommendations in practical applications usually encounter serious sparsity problems, which lead to insufficient interactions and cannot provide information for model learning. Moreover, the data is highly imbalanced: most users and items have few interactions with the system, which makes the recommendation task more difficult. To alleviate the problem of data sparsity, some studies have established multi-graph information to update the embeddings of users (items). However, they all process different graphs separately without considering the complementarity between different graphs.

[0006] Figure 1 It shows a tony example. The preferences of users u1 and u2 are for higher cost performance, while the preference of user u3 is for a striking appearance. Obviously, if the multiple preferences of users are ignored, it is impossible to distinguish whether I3 or I4 should be recommended to u1. More specifically, users may have multiple different preferences simultaneously. For example, u4 likes both high cost performance and a striking appearance. However, when considering the multiple preferences of users, it is obvious that I1 is more suitable for recommendation to u1, while I2 is more suitable for recommendation to u4 (I2 has both attributes). Therefore, if the potential preferences of users are not considered, it may lead to sub-optimal recommendations.

[0007] Although considering user preferences can improve the recommendation performance, there are still the following challenges: 1) How to extract user preference information? The preference information of users is hidden in the interaction behaviors of users, and how to effectively extract the preference information is still a challenge. 2) How to make full use of the information in the user-item interaction data and the similarity between user pairs and item pairs? It is reasonably assumed that separately processing different graphs will lose some meaningful information. 3) How to reorganize the extracted preference information? Even if the different preference information of users is effectively extracted, how to recombine them to generate the final embedding is still a challenge. Summary of the Invention

[0008] The object of the present invention is to provide a multi-preference collaborative filtering recommendation system with a co-attention memory mechanism, including an interaction information acquisition module, a user feature extraction module, a user potential preference extraction module, a user preference filtering module, and a recommendation module;

[0009] The interaction information acquisition module acquires the interaction information of users and items and models it as a user-item bipartite graph;

[0010] The interaction information acquisition module acquires the similarity between users and the similarity between items, and respectively establishes a user-user graph and an item-item graph;

[0011] The interaction information acquisition module transmits the user-item bipartite graph, the user-user graph, and the item-item graph to the user feature extraction module;

[0012] The user feature extraction module performs feature extraction on the user-user graph, the item-item graph, and the user-item bipartite graph to obtain user feature information and transmits it to the user potential preference extraction module;

[0013] The user potential preference extraction module calculates user preferences based on the user feature information and transmits them to the user preference filtering and recommendation module;

[0014] The user preference filtering and recommendation module filters the user preferences to obtain user embedding information and item embedding information, and transmits them to the recommendation module;

[0015] The recommendation module generates item recommendation information based on the user embedding information and the item embedding information, and sends it to the user.

[0016] Furthermore, the steps for the user potential preference extraction module to calculate the user preferences include:

[0017] 1) Calculate the probability that user u selects item k based on the m-th preference That is:

[0018]

[0019] In the formula, represents a matrix filled with K columns ; represents the neighbor set of user u; represents the specific feature of the m-th preference item; k ∈ {1,..., Ns}; represents the element-wise multiplication of two matrices; represents the matrix related to item i under the m-th preference and the matrix related to user u with the m-th preference;

[0020] Calculate the weight coefficient That is:

[0021]

[0022] In the formula, is the matrix related to item under the m-th preference;

[0023] 2) Calculate item under the m-th preference That is:

[0024]

[0025] 3) Screen out the users who have the same preferences as user u, and the steps include:

[0026] 3.1) Calculate the probability that other users have the same preferences as user u and the weight coefficient That is:

[0027]

[0028]

[0029] In the formula, represents a matrix filled with K columns ; represents the m-th preference user feature; is a matrix related to the probability that other users have the same preferences as user u;

[0030] 3.2) Calculate the set of users with the same preferences as user u That is:

[0031]

[0032] 4) Calculate user preferences That is:

[0033]

[0034] In the formula, represents the element-wise multiplication of two matrices.

[0035] Furthermore, the steps for the user preference filtering module to filter user preferences include:

[0036] 1) Fuse all user preferences to obtain:

[0037]

[0038] In the formula, g is a two-layer neural network; m ∈ {1,..., N m}; N m is the total number of user preferences; ∞ represents the concatenation operation; represents the N-th preference of user u m ;

[0039] 2) Use the softmax layer to normalize it to obtain the importance level of user u for the m-th preference That is:

[0040]

[0041] 3) Generate user embedding information Z u and item embedding information Z i , that is:

[0042]

[0043] In the formula, Gu, Fi are user and item source connection embeddings; represents the representation parameter of item i in the m-th preference; represents the importance level of item i in the m-th preference.

[0044] Furthermore, the steps for the user feature extraction module to extract features include:

[0045] 1) Aggregate the user-user graph and item-item graph using the memory module stored in the user feature extraction module to obtain:

[0046]

[0047] where Q m and R m are the transformation matrices of the m-th preference; p i is the interaction set of user u, and s u is the interaction set of item I; represents the aggregated information of user u and the m-th specific user; represents the aggregated information of user u and the m-th specific item;

[0048] 2) Extract the interaction features between user u and the m-th specific item The interaction features between user u and the m-th specific user That is:

[0049]

[0050] Extract the interaction features between item I and the m-th specific item And the interaction features with the m-th specific user That is:

[0051]

[0052] Furthermore, the user preference filtering and recommendation module stores source connection embeddings, that is:

[0053]

[0054] where ∞ represents the connection operation; layer represents the fully connected layer; G u is the user source connection embedding; F i is the item source connection embedding.

[0055] Furthermore, the user-item bipartite graph includes a number of user nodes and a number of item nodes. Among them, the user nodes are denoted as u i ∈W u , i = {1,..., N u}; the item nodes are denoted as v j ∈W v , j = {1,..., N v}; the node W = W u ∪W v ; the edge connecting the user node and the item node in the user-item bipartite graph is denoted as (u i , r, v j ) ∈ E; the edge (u i, r, v j ) represents user u i and item v j The interaction behavior type is r, where r ∈ R = {1, …, R}; R is the total number of interaction types.

[0056] Furthermore, the recommendation module stores an MLP network;

[0057] The input of the MLP network is the user embedding information Z u and the item embedding information Z i , and the output is the preference level r′ of user u for item i ui ;

[0058] The MLP network outputs the preference level r′ of user u for item i ui The process is as follows:

[0059]

[0060] In the formula, l is the index of the hidden layer; f l represents the hidden layer; w is the weight, and b l is the bias.

[0061] Furthermore, the recommendation module writes the item recommendation information into the item recommendation information set for items with a preference level greater than the threshold ε, and sends it to the user.

[0062] Furthermore, the training objective of the MLP network is to minimize the objective function;

[0063] The objective function is as follows:

[0064]

[0065] In the formula, represents the observed preference level; r ui represents the true preference level of user u for item i; λ and θ represent the regularization weight and the model parameters.

[0066] Furthermore, it also includes a database for storing interaction information acquisition module, user feature extraction module, user latent preference extraction module, user preference filtering module, and recommendation module data.

[0067] It should be noted that the present invention proposes a multi-preference collaborative filtering algorithm based on a co-attention memory mechanism. In addition to the user-item bipartite graph, the MPCF model also includes a user-user graph and an item-item graph. First, high-level information is captured through a storage module. For example, by extracting the memory (historical interactions) of u2, the path from u1 to I4 can be captured: (u1, I1, u2, I4). Three modules are carefully designed to distinguish the potential user preferences under user interactions: a decomposer, a combiner, and a fuser. First, the edges of multiple graphs are decomposed into multiple latent spaces with graph-level co-attention, and then the importance of different preferences is automatically identified through a fusion layer with preference-level attention and combined to obtain a unified user (item) embedding.

[0068] The technical effect of the present invention is beyond doubt. The present invention explicitly models the relationships between users-users and items-items and first explores the complementarity between different graphs.

[0069] The present invention captures high-level information of user interactions through a memory module. The potential interaction preferences of users are extracted by decomposing and reorganizing the edges in multiple graphs. Description of the Drawings

[0070] Figure 1 For a toy example;

[0071] Figure 2 For the overall framework of the system of the present invention;

[0072] Figure 3 For the experimental results of verifying the multi-graph information fusion effect;

[0073] Figure 4 For the experimental results of verifying the preference extraction effect; Figure 4 (a) For the experimental results of verifying the preference extraction effect under dataset I; Figure 4 (b) For the experimental results of verifying the preference extraction effect under dataset II; Figure 4 (c) For the experimental results of verifying the preference extraction effect under dataset III;

[0074] Figure 5 For the influence results of the embedding dimension on the recommendation effect; Figure 5 (a) For the influence results of the embedding dimension on the recommendation effect under dataset I; Figure 5 (b) For the influence results of the embedding dimension on the recommendation effect under dataset II; Figure 5 (c) For the influence results of the embedding dimension on the recommendation effect under dataset III. Detailed Embodiments

[0075] The present invention will be further described below in conjunction with embodiments, but it should not be understood that the above-mentioned subject matter scope of the present invention is limited to the following embodiments. Without departing from the above technical idea of the present invention, various substitutions and changes made according to common general knowledge and customary means in the art shall be included within the protection scope of the present invention.

[0076] Embodiment 1:

[0077] See Figure 2 , a multi-preference collaborative filtering recommendation system with a co-attention memory mechanism, including an interaction information acquisition module, a user feature extraction module, a user latent preference extraction module, a user preference filtering module, and a recommendation module;

[0078] The interaction information acquisition module acquires the interaction information between users and items and models it as a user-item bipartite graph;

[0079] The interaction information acquisition module acquires the similarity between users and the similarity between items, and respectively establishes a user-user graph and an item-item graph;

[0080] The interaction information acquisition module transmits the user-item bipartite graph, the user-user graph, and the item-item graph to the user feature extraction module;

[0081] The user feature extraction module extracts features from the user-user graph, the item-item graph, and the user-item bipartite graph to obtain user feature information and transmits it to the user latent preference extraction module;

[0082] The user latent preference extraction module calculates the user preference according to the user feature information and transmits it to the user preference filtering and recommendation module;

[0083] The user preference filtering and recommendation module filters the user preference to obtain user embedding information and item embedding information, and transmits it to the recommendation module;

[0084] The recommendation module generates item recommendation information according to the user embedding information and the item embedding information and sends it to the user.

[0085] The steps for the user latent preference extraction module to calculate the user preference include:

[0086] 1) Calculate the probability that user u selects item k based on the m-th preference That is:

[0087]

[0088] In the formula, represents a matrix filled with K columns ; represents the neighbor set of user u; Represents the specific feature of the m-th preference item; k ∈ {1, …, Ns}; Denotes the element-wise multiplication of two matrices; Represents the matrix related to item i under the m-th preference and the matrix related to user u with the m-th preference;

[0089] Calculate the weight coefficient That is:

[0090]

[0091] In the formula, Is the matrix related to the item under the m-th preference Related;

[0092] 2) Calculate the item under the m-th preference That is:

[0093]

[0094] 3) Screen out users with the same preferences as user u, and the steps include:

[0095] 3.1) Calculate the probability that other users have the same preferences as user u And the weight coefficient That is:

[0096]

[0097]

[0098] In the formula, Represents a matrix filled with K columns Filled; Represents the m-th preference user feature; Is the matrix related to the probability that other users have the same preferences as user u;

[0099] 3.2) Calculate the set of users with the same preferences as user u That is:

[0100]

[0101] 4) Calculate user preferences That is:

[0102]

[0103] In the formula, Denotes the element-wise multiplication of two matrices.

[0104] The steps for the user preference filtering module to filter user preferences include:

[0105] 1) Aggregate all user preferences to obtain:

[0106]

[0107] where g is a two - layer neural network; m ∈ {1, …, N m}; N m is the total number of user preferences; ∞ represents the concatenation operation; represents the N m th preference of user u;

[0108] 2) Use the softmax layer to normalize and obtain the importance level of user u for the mth preference, that is:

[0109]

[0110] 3) Generate user embedding information Z u and item embedding information Z i , that is:

[0111]

[0112] where Gu, Fi are user and item source connection embeddings; represents the representation parameter of item i in the mth preference; represents the importance level of item i in the mth preference.

[0113] The steps for the user feature extraction module to perform feature extraction include:

[0114] 1) Use the memory module stored in the user feature extraction module to aggregate the user - user graph and item - item graph to obtain:

[0115]

[0116] where Q m and R m are the transformation matrices for the mth preference; p i is the interaction set of user u, and s u is the interaction set of item I; represents the aggregated information of user u with the mth specific user; represents the aggregated information of user u with the mth specific item;

[0117] 2) Extract the interaction features between user u and the mth specific item and the interaction features

[0118]

[0119] Extract the interaction features between item I and the m-th specific item Interaction features with the m-th specific user That is:

[0120]

[0121] The user preference filtering recommendation module stores source connection embeddings, that is:

[0122]

[0123] In the formula, ∞ represents the connection operation; layer represents the fully connected layer; G u is the user source connection embedding; F i is the item source connection embedding.

[0124] Furthermore, the user-item bipartite graph includes a number of user nodes and a number of item nodes, where the user nodes are denoted as u i ∈W u , i = {1,..., N u}; the item nodes are denoted as v j ∈W v , j = {1,..., N v}; the node W = W u ∪W v ; the edges connecting user nodes and item nodes in the user-item bipartite graph are denoted as (u i , r, v j ) ∈ E; the edge (u i , r, v j ) indicates that the interaction behavior type between user u i and item v j is r, r ∈ R = {1,..., R}; R is the total number of interaction types.

[0125] The recommendation module stores an MLP network;

[0126] The input of the MLP network is the user embedding information Z u and the item embedding information Z i , and the output is the preference level r' of user u for item i ui ;

[0127] The MLP network outputs the preference level r' of user u for item i ui as follows:

[0128]

[0129] In the formula, l is the index of the hidden layer; fl denotes the hidden layer; w is the weight, and b l is the bias.

[0130] The recommendation module writes item recommendation information into the item recommendation information set for items with a preference level greater than the threshold ε and sends it to the user.

[0131] The training objective of the MLP network is to minimize the objective function;

[0132] The objective function is as follows:

[0133]

[0134] In the formula, denotes the observed preference level; r ui denotes the true preference level of user u for item i; λ and θ denote the regularization weight and model parameters.

[0135] It also includes a database for storing interaction information acquisition module, user feature extraction module, user latent preference extraction module, user preference filtering module, and recommendation module data.

[0136] Embodiment 2:

[0137] Refer to Figure 2 , and the usage process of the multi-preference collaborative filtering recommendation system with a co-attention memory mechanism is as follows:

[0138] First, summarize the first-order neighbor features of users from the user-item bipartite graph to obtain item-specific features. Then, construct the user-user graph and the item graph. Here, the interaction history of each user (item) is retained, the high-level information of user interactions is extracted through the memory module, and then the user-specific features are obtained through the multi-graph encoding layer. Next, a new graph-level co-attention mechanism is proposed to reorganize these features to obtain the latent preferences of users, and then the preference-level attention mechanism is used to automatically identify the importance of different preferences to obtain the final embedding of users. When summarizing user features to learn item embeddings, there is a process similar to the above. Finally, the prediction score is output through the MLP layer.

[0139] Specifically,

[0140] The user-item bipartite graph

[0141] The interactions between users and items in the recommendation system can be naturally modeled as a user-item bipartite graph. Among them, W is the node, including user nodes ui ∈ Wu, i = {1,..., Nu} and item nodes vj ∈ Wv, j = {1,..., Nv}, W = Wu ∪ Wv, and the edge (ui, r, vj) ∈ E represents that the interaction behavior type between user ui and vj is r, r ∈ R = {1,..., R}, where R is the total number of interaction types.

[0142] The bipartite graph convolutional network layer aims to aggregate the items purchased by the user. For simplicity, m∈M={M1, …, Mn} is used as a preference indicator, and Mi, i∈{1, 2, …, n} represents the i-th potential interaction preference of the user.

[0143] Multi-layer

[0144] In addition to the user-item bipartite graph, the relationships between users-users and items-items are also explicitly modeled to alleviate the data sparsity problem in collaborative filtering. Generally, this graph is constructed by calculating the pairwise cosine similarity on the rows (columns) of the rating matrix. Considering that the interests of users change over time, users who are more similar in the time dimension are given more weights.

[0145] Existing studies treat each graph separately and then combine them, ignoring an important fact that different graphs are complementary to each other. For example, the items that a target user may be interested in can be inferred based on the characteristics of neighboring users. Therefore, this embodiment carefully designs a graph-level collaborative attention mechanism for multi-graph encoding.

[0146] Considering that the final embedding of the user (item) should also depend on the characteristics of the user (item) itself, source connections are designed to re-enhance these characteristics. Specifically, the original features are passed through a single fully connected layer to generate the source connection embedding. The formula is as follows:

[0147]

[0148] where ∞ represents the concatenation operation. layer means the fully connected layer. Similarly, the source connection embedding of Fi can also be obtained.

[0149] The disentangler assumes that the interactions of the user are driven by M preferences, and Mi represents the i-th user preference. The general idea is as follows: First, the user has a feature matrix where Lu is the dimension of the user features, and the item has a feature matrix where Li is the dimension of the item features. To distinguish different preferences of the user (item), M preference-specific transformation matrices and are designed respectively. Each transformation matrix maps the user (item) features to an aspect-specific semantic space.

[0150] First, the memory module aggregates the user interaction history retained in the user-user graph, which is formulated as follows:

[0151]

[0152] where Qm and Rm are the transformation matrices of the m-th preference, pi is the interaction set of the user's neighbors, and sui is the interaction set of the item's neighbors. For user u, its m-th item-specific feature Gik and m-th user-specific feature Guk can be extracted as:

[0153]

[0154] Similarly, for item I, its m-th item-specific feature and m-th user-specific feature can be extracted as:

[0155]

[0156] Combiner

[0157] Below, focusing on user u and his interaction set P and neighbor set S, user u has item-specific features and user-specific features. Obviously, they come from different graphs. Existing research processes each graph separately and then fuses them, ignoring the complementarity between different graphs. For this reason, the present invention proposes a graph-level co-attention mechanism to reorganize these features, which is mainly divided into the following two parts:

[0158] 1) Item attention based on users: First, use a single mean pooling to summarize the neighbor representation represents the neighbor set of user u; Next:

[0159]

[0160] where Gu represents a matrix filled with K columns of Gu m, represents the m-th preference item-specific feature, and d represents the embedding dimension. represents the element-wise multiplication of two matrices. h ik represents the probability that user u will purchase item k due to the m-th preference. The softmax function is used to obtain the weight coefficient α:

[0161]

[0162] Since the attention probability of each item-specific feature under the m-th preference is calculated from the above formula, the new representation of the item is the weighted sum of the item-specific features:

[0163]

[0164] User attention based on items: User attention based on items can distinguish which items are purchased by user u due to the m-th preference, in the same way as using user attention based on items to distinguish which users have the same purchase preferences as the target user. Use the new item representation combined with the original user-specific function to filter out users who have the same purchase preferences as user u, and the process is as follows:

[0165]

[0166]

[0167] d represents the embedding dimension. Since the attention probability of each user-specific feature under the m-th preference is calculated from the above formula, the new representation of the user is the weighted sum of user-specific features:

[0168]

[0169] Now, there are item aggregated representations Hi and user aggregated representations Hu. The representation of user u in the m-th preference is as follows:

[0170]

[0171] where represents the element-wise multiplication of two matrices.

[0172] F. Fuser

[0173] Obviously, not every preference is equally important. For most people, one or two preferences dominate the interactions of the present invention, which leads to the present invention proposing a preference-level attention mechanism to filter out unimportant preferences.

[0174] 1) Preference-level attention: The present invention proposes a new fusion layer that integrates the set {Xu m}, m ∈ {1,..., Nm} into a unified representation, as follows:

[0175]

[0176] where g is a two-layer neural network that acts as a preference-level attention mechanism. ∞ represents the concatenation operation. Then, through the softmax layer, is normalized to obtain the importance of the m-th preference as follows:

[0177]

[0178] Obviously, the larger β is, the more dominant the user's preference will be. Through these weight factors β, the final embedding of user u is obtained by weighted summation:

[0179]

[0180] where Gu is the source connection embedding.

[0181] 2) Note: Above, the preference extraction and reorganizer in the user part focus on all concerns, and there is a similar process in the item part.

[0182] G. Once the final embeddings of user u and item I are obtained from the user and item parts respectively, they are concatenated and passed through an MLP to predict the rating r0 from u to I:

[0183]

[0184] where l is the index of the hidden layer.

[0185] The objective function for training is formulated as follows:

[0186]

[0187] where λ and θ represent the regularization weight and model parameters. To minimize the objective function, the Adam optimizer is used. Additionally, dropout and L2 regularization terms are used for regularization. To accelerate model optimization, a weighted random sampling strategy is adopted.

[0188] Example 3:

[0189] A verification experiment of a multi-preference collaborative filtering recommendation system with a co-attention memory mechanism is as follows:

[0190] Experimental settings

[0191] Datasets: Extensive experiments were conducted on three real-world datasets: Movielens, Amazon, and Yelp. All these datasets are accessible and differ in terms of domain, size, and sparsity. The statistics of the datasets are summarized in Table I.

[0192] Movielens - 100k: This dataset has been widely used for evaluating recommendations and contains 100,000 ratings from 943 users for 1,682 movies.

[0193] Amazon: A widely used product recommendation dataset containing 65,170 ratings from 1000 users to 1000 items.

[0194] Yelp: A local business recommendation dataset containing 30,838 ratings from 1,286 users for 2,614 items.

[0195] For each dataset, 80% of the historical scores were randomly selected as the training set, and the remaining were used as the test set.

[0196] Baselines: To evaluate the effectiveness of the model of the present invention, MPCF was compared with the following state-of-the-art baselines.

[0197] PMF: A probabilistic algorithm was proposed, which can scale linearly with the number of observations and can handle very sparse and imbalanced datasets well.

[0198] BiasMF: Naturally integrates many key aspects of data, such as multiple forms of feedback, temporal dynamics, and confidence. LLORMA-Local: Based on the assumption that the matrix is locally low-rank, a new low-rank matrix approximation is proposed.

[0199] AutoRec: AutoRec takes the user partial vector r(u) or the item partial vector r(i) as input, aiming to reconstruct them at the output layer. It has two variants: item-based AutoRec and user-based AutoRec.

[0200] GC-MC: A graphical autoencoder framework for matrix completion tasks in recommendation systems.

[0201] MCCF: Explores the differences in purchase motivations under simple edges in the user-item bipartite graph, where the edges are decomposed and then recombined with hierarchical attention to encode the latent semantics based on specific user-item pairs.

[0202] Implementation:

[0203] Pytorch is used to solve the MPCF model of the present invention. The present invention uses the adjacency matrix as the feature matrix x, searches for the learning rate in {0.0001, 0.0005, 0.001, 0.005}, and the L2 normalization coefficient in {1e1, 1e2, 1e3, 1e4}.

[0204] Change the preference number M in the range {1, 2, 3, 4} and the embedding dimension d in the range {8, 16, 32, 64, 128, 256, 512}. Search for the batch size in {64, 128, 256, 512}.

[0205] Initialize the parameters of the model proposed by the present invention to a Gaussian distribution with a mean of 0 and a standard deviation of 0.1. The optimizer is the adam optimizer. At the same time, dropout is used for multi-preference fusion. Test the drop rate in {0.1, 0.4, 0.5, 0.6}. The present invention uses ReLU as the activation function of the neural network.

[0206] Randomly select 80% of the interaction records to train the neural network, and select the remaining interaction records to test the neural network. Adjust the model parameters so that each model can obtain the best performance. Run all models 3 times and output the average results.

[0207] Two widely used evaluation protocols: Root Mean Square Error (RMSE) and Mean Absolute Error (MAE) are used as evaluation metrics.

[0208] Performance comparison

[0209] Table II summarizes the comparison results. It can be seen from this table that:

[0210] MPCF is significantly better than all other baselines in most cases, which verifies the effectiveness of the model of the present invention. In particular, compared with the strongest baseline, on the three datasets, the MAE is improved by 1.17%, 1.39% and 5.60% respectively, and the RMSE is improved by 1.10%, 1.18% and 5.28% respectively. This significant improvement is attributed to the preference extraction ability and multi-graph auxiliary information.

[0211] On the three datasets, the GCN-based model is better than the CF-based and autoencoder-based models. These improvements are attributed to the graph convolutional layer. This operation not only captures local structural information, but also learns the distribution of the neighbor features of each ego node, thus enhancing the expressiveness of the representation.

[0212] The present invention obtains better results on Yelp. It is speculated that the possible reason is that the data on Yelp is relatively sparse, so the auxiliary information of the multi-graph can obtain better results.

[0213] Ablation experiment

[0214] In order to evaluate and verify the effectiveness of different components of the MPCF model proposed by the present invention, an ablation study was conducted.

[0215] Multi-graph information: Explicitly model the user-user relationship and item-item relationship to alleviate the data sparsity problem in collaborative filtering. In order to explore whether multi-graph information can improve the recommendation performance, experiments were conducted on the three datasets. The experimental results are as Figure 3 shown. The results show that after fusing the multi-graph information, the recommendation performance on the three datasets has been significantly improved. This shows that it is meaningful to explicitly model the user-user graph and item-item graph compared with only using the user-item bipartite graph.

[0216] Attention mechanism and source connection: In the model of the present invention, a graph-level co-attention mechanism and a preference-level attention mechanism are used. In order to emphasize the importance of the user (item) itself, the source connection layer is used to re-enhance the function of the user (item). Therefore, an experiment was conducted on Amazon to study the effectiveness of these components, and the experimental results are shown in Table III.

[0217] It can be concluded from Table III that the three main components of the model proposed in the present invention, namely the source connection, the co-attention mechanism, and the preference-level attention mechanism, are all proven to be effective. Among all components, the co-attention mechanism has the greatest impact on performance. This also verifies the hypothesis of the present invention: different graphs complement each other. Through the purchase records of users, the importance of different neighbors can be effectively distinguished, and the items that users may be interested in can be inferred from the characteristics of neighbors. In addition, it can be found that the source connection can significantly improve performance. Combining all three components can further improve performance.

[0218] Hyperparameter study

[0219] In the model of the present invention, the number of user preferences is a key parameter, and the present invention explores its impact on performance. At the same time, the impact of the embedding dimension d on performance is also studied.

[0220] Number of preferences: To study the preference extraction ability of the model, the preference amount M is changed within the range of {1, 2, 3, 4}, while keeping other parameters unchanged. Experiments are carried out on three datasets, and the experimental results are as Figure 4 shown. It can be concluded that for different datasets, when achieving the best recommendation performance, the number of preferences is inconsistent. On the Yelp dataset, most ratings are 1 or 2, indicating that one preference is sufficient to simulate the latent semantics. On the Amazon and Movielens datasets, user interactions are more complex. At this point, the importance of multiple preferences is more prominent. Increasing M leads to performance improvement, which also proves that there are finer-grained user preferences hidden in user-item interactions, and this information cannot be expressed by edge information. However, the designed model can effectively extract this information. At the same time, as the number of preferences increases, the recommendation performance first rises and then falls, possibly due to overfitting.

[0221] 2) Impact of embedding dimension: The dimension of the embedding d is also a key hyperparameter that controls the complexity and capacity of the MPCF proposed in the present invention. For this reason, the impact of the embedding dimension on the recommendation effect is studied, and the embedding size is set to 8 - 512. The experimental results are as Figure 5 shown. They all have a common trend: as the embedding dimension gradually increases, the recommendation performance will first gradually increase and reach a peak, and then as the dimension further increases, the performance will remain stable or even decline. Therefore, the present invention uses an appropriate embedding dimension d to balance the trade-off between performance and complexity.

Claims

1. A multi-preference collaborative filtering recommendation system with a co-attention memory mechanism, characterized in that: It includes an interaction information acquisition module, a user feature extraction module, a user potential preference extraction module, a user preference filtering module, and a recommendation module; The interaction information acquisition module acquires the interaction information between users and items and models it as a user-item bipartite graph; The interaction information acquisition module acquires the similarity between users and the similarity between items, and respectively constructs a user-user graph and an item-item graph; The interaction information acquisition module transmits the user-item bipartite graph, the user-user graph, and the item-item graph to the user feature extraction module; The user feature extraction module extracts features from the user-user graph, the item-item graph, and the user-item bipartite graph to obtain user feature information and transmits it to the user potential preference extraction module; The user potential preference extraction module calculates the user preference according to the user feature information and transmits it to the user preference filtering and recommendation module; The user preference filtering and recommendation module filters the user preference to obtain user embedding information and item embedding information and transmits it to the recommendation module; The recommendation module generates item recommendation information according to the user embedding information and the item embedding information and sends it to the user; The steps for the user potential preference extraction module to calculate the user preference include: 1) Calculate the probability that user u selects item k based on the m-th preference That is: In the formula, represents a matrix filled with K columns ; represents the set of neighbors of user u; represents the m-th preference item-specific feature; k ∈ {1, …, Ns}; represents the element-wise multiplication of two matrices; represents the matrix related to item i under the m-th preference and the matrix related to user u with the m-th preference; Calculate the weight coefficient That is: wherein, is a matrix related to the item under the m-th preference; 2) Calculate the item under the m-th preference That is: 3) Screening out users with the same preference as user u, the steps include: 3.1) Calculate the probability that other users have the same preferences as user u and the weight coefficient That is: In the formula, represents a matrix filled with K columns ; represents the m-th preference user feature; is a matrix related to the probability that other users have the same preferences as user u; 3.2) Calculate the set of users with the same preferences as user u That is: 4) Calculate user preferences That is: wherein, represents the element-wise multiplication of two matrices; The steps for the user feature extraction module to perform feature extraction include: s1) Aggregating the user-user graph and the item-item graph using the memory module stored in the user feature extraction module to obtain: Where Q m and R m are the conversion matrices of the m-th preference; p i is the interaction set of user u, and s u is the interaction set of item I; represents the aggregated information of user u and the m-th specific user; represents the aggregated information of user u and the m-th specific item; s2) Extract the interaction features between user u and the m-th specific item The interaction features between user u and the m-th specific user That is: Extract the interaction features between item I and the m-th specific item Interaction features with the m-th specific user That is:

2. The multi-preference collaborative filtering recommendation system with a co-attention memory mechanism according to claim 1, characterized in that, The steps for the user preference filtering module to filter the user preference include: 1) Integrate all user preferences to obtain: where \(g\) is a two - layer neural network; \(m\in\{1,\ldots,N\}\); m \(N\) m is the total number of user preferences; \(\infty\) represents the concatenation operation; represents the \(N\)th m preference of user \(u\); 2) Use the softmax layer to perform normalization to obtain the importance level of user u's preference for the m-th That is: 3) Generate user embedding information Z u and item embedding information Z i , namely: where Gu and Fi are the user and item source connection embeddings; denotes the representation parameter of item i in the m-th preference; denotes the importance level of item i in the m-th preference.

3. The multi-preference collaborative filtering recommendation system with a co-attention memory mechanism according to claim 1, wherein: The user preference filtering and recommendation module stores the source connection embedding, that is: where ∞ represents the concatenation operation; layer represents the fully connected layer; G u is the user source connection embedding; F i is the item source connection embedding.

4. The multi-preference collaborative filtering recommendation system with a co-attention memory mechanism according to claim 1, characterized in that: The user-item bipartite graph includes a number of user nodes and a number of item nodes, where the user nodes are denoted as u i ∈W u , i = {1, …, N u}; the item nodes are denoted as v j ∈W v , j = {1, …, N v}; the set of nodes W = W u ∪W v ; the edges connecting user nodes and item nodes in the user-item bipartite graph are denoted as (u i , r, v j ) ∈ E; the edge (u i , r, v j ) indicates that the interaction behavior type between user u i and item v j is r, where r ∈ R = {1, …, R}; R is the total number of interaction types.

5. The multi-preference collaborative filtering recommendation system with a co-attention memory mechanism according to claim 1, wherein The recommendation module stores an MLP network; The input of the MLP network is the user embedding information Z u and the item embedding information Z i , and the output is the preference level r' of user u for item i ui ; The process of the MLP network outputting the preference level r' of user u for item i is as follows: ui is as follows: where l is the index of the hidden layer; f l represents the hidden layer; w is the weight, b l is the bias.

6. The multi-preference collaborative filtering recommendation system with a co-attention memory mechanism according to claim 5, characterized in that, The recommendation module writes the item recommendation information into the item recommendation information set for items with a preference level greater than the threshold ε and sends it to the user.

7. The multi-preference collaborative filtering recommendation system with a co-attention memory mechanism according to claim 5, wherein The training objective of the MLP network is to minimize the objective function; Objective function is as follows: In the formula, represents the observed preference level; r ui represents the true preference level of user u for item i; λ and θ represent the regularization weight and model parameters.

8. The multi-preference collaborative filtering recommendation system with a co-attention memory mechanism according to claim 1, wherein It also includes a database for storing the data of the interaction information acquisition module, the user feature extraction module, the user potential preference extraction module, the user preference filtering module, and the recommendation module.

Images