A collaborative filtering recommendation system based on a graph neural network

Abstract

The application discloses a collaborative filtering recommendation system based on a graph neural network, comprising a user-item bipartite graph generation module, an embedding information generation module, an implicit relation construction module, a data fusion module and a recommendation module; the application uses a memory network based on attention to learn user-user and item-item graphs constructed respectively, so as to obtain relation information between adjacent pairs. The model simultaneously learns all three graphs, unifies multi-attribute and implicit relation information through an information fusion layer, and realizes end-to-end recommendation.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of data mining, and particularly to a collaborative filtering recommendation system based on graph neural network modeling multi-attribute and implicit relationship information. BACKGROUND

[0002] Since the interaction between users and items in the recommendation system can be naturally modeled as a user-item bipartite graph, many emerging researches have devoted to exploring the graph aspect of collaborative filtering methods. As a result, the field of graph convolutional neural networks has made quite effective achievements in recommendation systems. In fact, the interaction between users and items is usually derived from more complex underlying factors, such as the specific preferences of users. Existing methods have proposed to use user-item bipartite graphs to understand the differences in motivation, but have not explicitly proposed the factors that affect the differences and modeled them, while ignoring the similarity between users and items. Multiple restrictions make them unable to capture the fine-grained user preferences more effectively.

[0003] In the information overload environment of the Internet, users want to more efficiently obtain information of interest, and companies also want products to maximize the attraction and retention of users, so as to achieve development. The recommendation system is thus born, which is designed to produce personalized item recommendations and deal with the problem of information overload. Since the recommendation system has received effective feedback in practice, it has not only aroused great interest in the academic field, but also been widely developed in the industrial field.

[0004] For many modern recommendation systems, a common and effective solution is the collaborative filtering (CF) technique, whose basic assumption is that "people who share similar purchases in the past tend to have similar choices in the future". Matrix factorization algorithms add the concept of implicit vectors on the basis of CF algorithms. The vectors are inferred from the records of user-item interactions, but only consider the characteristics of users and items, lacking explicit combinations of user-item interactions. Essentially, such user-item interaction information can be naturally modeled as a graph. The graph can contain more explicit interaction information, while increasing the connectivity between users and items. Recently, graph convolutional neural networks have become one of the best performance architectures for various graph learning tasks. GC-MC uses two multi-link graph convolutional layers to aggregate user features and item features. NGCF builds a user-item bipartite graph to collect first-order neighborhood information.

[0005] Despite their effectiveness, these methods still have two important limitations. First, they do not distinguish between different users' purchasing motivations. However, in the real world, the motivations behind user decisions are multifaceted. Personal attributes can significantly influence preferences. For example, someone who loves science and technology and makes a living from it is much more likely to purchase high-tech products than simply something aesthetically pleasing. However, someone with artistic attributes may choose the opposite. Individuals may prefer items with different attributes depending on their individual attributes. Attributes stem from a person's personality, occupation, and professional focus, and are not simply acquired over a short period of time. Moreover, choices influenced by attributes tend to be dominant and more stable. Therefore, these methods that do not distinguish between purchasing motivations inevitably miss valuable, fine-grained information. Recent work has employed multiple components to capture more complex interaction features, but still fails to utilize more comprehensive and effective information to provide interpretable recommendations. Second, these methods only consider the characteristics of nodes in a bipartite graph, treating the graph as a single entity. This ignores the user-user and item-item relationships outside the bipartite graph, which are also very important signals. Implicit relationships can be modeled using user-user and item-item graphs to reflect more complex interaction features.

[0006] Figure 1 A simple example is shown. If we ignore the user's latent attributes and disregard differences in purchasing motivations, we cannot compare the likelihood of user u1 purchasing product i3 or i4. However, assuming that we take into account the influence of latent attributes on preferences—users u1, u3, and u4 prefer high-tech products, while user u2 prefers art—we can determine that item i4 is more suitable for u1 than i3. This is because, from the user-item interaction level, item i4 was purchased by a user who prefers high-tech products and is more consistent with user u1's purchasing motivation. From the user-user and item-item levels, based on the similarity of the purchased item attributes, we can infer that u3 and u1 have more similar preferences. This similar relationship can be captured by implicit relational modeling. Therefore, it is necessary to design a recommendation system that can describe fine-grained user preferences at two levels. Summary of the Invention

[0007] The purpose of the present invention is to provide a collaborative filtering recommendation system based on graph neural network, which includes a user-item bipartite graph generation module, a multi-attribute embedding information generation module, an implicit relationship construction module, a data fusion module and a recommendation module;

[0008] The user-item bipartite graph generation module obtains the user's ratings of several recommended items and generates a user-item bipartite graph And transmit it to the multi-attribute embedding information generation module and implicit relationship construction module;

[0009] The multi-attribute embedding information generation module generates the user-item bipartite graph Processing is performed to generate user multi-attribute embedding information and item multi-attribute embedding information, and transmitted to the data fusion module;

[0010] The implicit relationship construction module uses the user-item bipartite graph to construct user-user and item-item graphs to model implicit relationship information, and transmits it to the data fusion module;

[0011] The data fusion module fuses the user multi-attribute embedding information, the project multi-attribute embedding information, and the implicit relationship information to obtain the final embedding representation of the user and the project; the final embedding representation is transmitted to the recommendation module;

[0012] The recommendation module calculates the user's preference rating for the item based on the final embedding representation of the user and the item.

[0013] Furthermore, the user's rating of the project is modeled as a user-item bipartite graph in, and Represent the user set and the set of items to be recommended respectively; and Represents the feature matrices of the user set and the item set to be recommended respectively; N u is the total number of users; N i is the total number of items to be recommended; d represents the feature dimension; the scoring set Contains the user's rating of each interacted item; rating set R is the highest rating level; ε represents the edge set; any edge e=(u, i, r)∈ε in the set ε indicates that user u has a clear rating level r for item i.

[0014] Furthermore, the step of generating user and item embedding information by the multi-attribute embedding information generation module includes:

[0015] 1) Extract M latent attribute spaces from the user-item bipartite graph; where the mth latent attribute space affects the mth interaction possibility in the user-item interaction;

[0016] 2) Extracting user potential attribute space and the project potential attribute space

[0017] Among them, the mth potential attribute space of user u is and the mth latent attribute space of item i They are as follows:

[0018]

[0019]

[0020] where W = {W1, W2, …, WM} is the user latent attribute space transformation matrix; Q = {Q1, Q2, …, QM} is the item latent attribute space transformation matrix; m = 1, …, M; u = 1, …, U; i = 1, …, I. M} is the user latent attribute space transformation matrix; m = 1, …, M; u = 1, …, U; i = 1, …, I. u , p i is the feature of user u, item i.

[0021] 3) Calculate the possibility of user u interacting with item i in the mth latent attribute space i.e.

[0022]

[0023] where att node represents the node-level attention neural network.

[0024] 4) Calculate the weight coefficient using the softmax function i.e.

[0025]

[0026] where σ represents the activation function; a m represents the node-level attention vector in the mth latent attribute space. is the item set.

[0027] 5) Calculate the latent attribute space set of user u where the mth latent attribute feature of user u is as follows:

[0028]

[0029] Calculate the weight of each latent attribute space, i.e.

[0030]

[0031] where represents the weight of the Mth latent attribute space, att spac represents the attribute space-level attention neural network.

[0032] 6) Calculate the joint embedding of the inherent latent attribute feature of user u and its latent attribute space i.e.

[0033]

[0034] where C m represents the weight matrix; b m represents the bias vector.

[0035] Calculate the importance of the mth latent attribute space Right now:

[0036]

[0037] Where q represents the attention vector; b represents the bias;

[0038] Normalized calculation to obtain the weight of the mth potential attribute space Right now:

[0039]

[0040] 7) Generate user embedding information z of the bipartite graph module u and project embedding information z i ,Right now:

[0041]

[0042] Where, is the weight of the mth potential attribute space of item i; is the mth potential attribute feature of item i.

[0043] Furthermore, the step of constructing implicit relationship information by the implicit relationship construction module includes:

[0044] 1) Through the user-item bipartite graph, we construct a user-user graph and an item-item graph based on cosine similarity to represent similar users and similar items, and model the implicit relationship between users and items based on multiple graphs;

[0045] First, calculate the joint embedding c of users with similar preferences u ,Right now:

[0046]

[0047] Where u n represents the nth user; u (n，l) Indicates that the user u n The lth user with similar preferences; ⊙ represents the dot product;

[0048] 2) Establish a joint embedding matrix Right now:

[0049]

[0050] Where, is the memory matrix; j = 1, ..., N; N is the number of similar users in different latent attribute spaces;

[0051] 3) Calculate the attention weights of similar users Right now:

[0052]

[0053] Where K represents the attention key matrix of similar users in different latent attribute spaces; Kj is the element of matrix K;

[0054] 4) Normalize to get the attention weight α of similar users j ,Right now:

[0055]

[0056] 5) Calculate the correlation vector Right now:

[0057]

[0058] In the formula, the correlation vector Used to represent user u n The lth user with similar preferences to user u n The influence of preferences;

[0059] 6) Establish a deep neural network for determining the weight of the association vector, namely:

[0060]

[0061] Where, Represents the association vector The weight of

[0062] The input of the deep neural network is the association vector, and the output is the association vector weight;

[0063] Relevance vector weight As shown below:

[0064]

[0065] Where W is the model parameter;

[0066] 7) Calculate the influence weight of each user with similar preferences, that is:

[0067]

[0068] Where, L n Represents all users with n Users with similar preferences;

[0069] 8) Generate implicit relationship information v u ,Right now:

[0070]

[0071] Furthermore, the data fusion module and the recommendation module store an MLP network;

[0072] The input of the MLP network is user association information U u =[z u ||v u ]·A u Project related information i =[z i ||v i ]·A i , the output is user u’s preference level r’ for item i ui ;Intermediate parameter A u =softmax(σ(W a1 ·z u +W a2 ·v u )), A i =softmax(σ(W a1 ·z i +W a2 ·v i ));v u 、v i are the implicit relationship information of users and items respectively; W a1 、W a2 is the calculation coefficient;

[0073] The MLP network outputs user u’s preference level r’ for item i ui The process is as follows:

[0074] g1=[U u ||I i ],

[0075] g2=σ(W2·g1+b2),

[0076] …

[0077] g l =σ(W l ·g l-1 +b l ),

[0078] r′ ui =w T ·g l , (20)

[0079] Where l is the index of the hidden layer; g l represents the hidden layer; w is the weight, b l For bias.

[0080] Furthermore, the objective function of the MLP network is As shown below:

[0081]

[0082] Where, training parameters θ = {W, Q}; λ and θ represent regularization weights;

[0083] Among them, the function equation As shown below:

[0084]

[0085] Where, represents the observed preference level; r ui represents the true preference level of user u for item i.

[0086] Furthermore, the method of sending recommended item information to the user based on the user's preference level for the item by the recommendation module is: sending items with a preference level greater than a threshold ε to the user.

[0087] Furthermore, it also includes a database for storing user-item bipartite graph generation module, multi-attribute embedding information generation module, implicit relationship construction module, data fusion module and recommendation module data.

[0088] It is worth noting that multi-attribute factors and implicit relational factors are accurately proposed and modeled. For a given user-item bipartite graph, multiple attributes are first extracted. A two-layer attention mechanism is then utilized to distinguish the probability distribution of the attribute space, and finally, the attribute factors are modeled. Furthermore, sparse regularization can mitigate overfitting caused by attribute factors that reflect similar motivations. For the implicit relational component, separately constructed user-user and item-item graphs are used. An attention-based memory module is used to learn specific relation vectors between node pairs, and relation-level attention is then used to automatically select informative neighbors for preference modeling.

[0089] The technical effect of the present invention is unquestionable. This paper proposes a new recommendation model, which explicitly proposes and models two implicit factors that affect user-specific preferences - latent attributes and implicit relationships. MI-CF aggregates the multi-attribute space through the user-item bipartite graph, and additionally establishes user-user and item-item graphs, and models the similarity relationship information of neighbor pairs through the memory module. By incorporating multiple graphs into the embedding learning process, end-to-end recommendation is achieved. In addition, in order to avoid the problem that multi-attribute factors reflect similar motivations, the present invention uses a sparse regularizer to solve possible overfitting. The present invention conducted extensive experiments on three public datasets, showing the performance gain of MI-CF. Further experiments quantitatively verified the effectiveness of each component of the model proposed by the present invention, and proved the effectiveness and interpretability of fine-grained implicit factors.

[0090] This paper proposes a new collaborative filtering method based on graph neural network, MI-CF, which captures the fine-grained implicit factors behind user behavior based on attribute-level attention and implicit relationship aggregation.

[0091] This paper uses an attention-based memory network to learn separately constructed user-user and item-item graphs to capture the relationship information between adjacent pairs. The model learns all three graphs simultaneously, unifying multi-attribute and implicit relationship information through an information fusion layer to achieve end-to-end recommendation.

[0092] We conduct extensive experiments on three public datasets to evaluate our proposed method. The experimental results demonstrate the effectiveness and interpretability of MI-CF. BRIEF DESCRIPTION OF THE DRAWINGS

[0093] Figure 1 Examples of toy purchase records for different purchase motivations;

[0094] Figure 2 It is the system framework of the present invention;

[0095] Figure 3 Provides a partial attention-based memory module for users;

[0096] Figure 4 The impact of latent attribute space on three real datasets; Figure 4 (a) The impact of latent attribute space on the Yelp dataset; Figure 4 (b) The impact of the latent attribute space on the Amazon dataset; Figure 4 (c) The impact of latent attribute space on the MovieLens dataset;

[0097] Figure 5 The impact of embedding information on three real datasets; Figure 5 (a) The impact of embedded information on the Yelp dataset; Figure 5 (b) The impact of embedded information on the Amazon dataset; Figure 5 (c) The impact of embedded information on the MovieLens dataset. DETAILED DESCRIPTION

[0098] The present invention will be further described below with reference to the following examples, but it should not be understood that the scope of the present invention is limited to the following examples. Without departing from the above technical ideas of the present invention, various substitutions and modifications can be made according to common technical knowledge and customary means in the art, and all should be included in the scope of protection of the present invention.

[0099] Example 1:

[0100] A collaborative filtering recommendation system based on graph neural network, including a user-item bipartite graph generation module, a multi-attribute embedding information generation module, an implicit relationship construction module, a data fusion module and a recommendation module;

[0101] The user-item bipartite graph generation module obtains the user's ratings of several recommended items and generates a user-item bipartite graph And transmit it to the multi-attribute embedding information generation module and implicit relationship construction module;

[0102] The multi-attribute embedding information generation module generates the user-item bipartite graph Processing is performed to generate user multi-attribute embedding information and item multi-attribute embedding information, and transmitted to the data fusion module;

[0103] The implicit relationship construction module uses the user-item bipartite graph to construct user-user and item-item graphs to model implicit relationship information, and transmits it to the data fusion module;

[0104] The data fusion module fuses the user multi-attribute embedding information, the project multi-attribute embedding information, and the implicit relationship information to obtain the final embedding representation of the user and the project; the final embedding representation is transmitted to the recommendation module;

[0105] The recommendation module calculates the user's preference rating for the item based on the final embedding representation of the user and the item.

[0106] Furthermore, the user's rating of the project is modeled as a user-item bipartite graph in, and Represent the user set and the set of items to be recommended respectively; and Represents the feature matrices of the user set and the item set to be recommended respectively; N u is the total number of users; N i is the total number of items to be recommended; d represents the feature dimension; the scoring set Contains the user's rating of each interacted item; rating set R is the highest rating level; ε represents the edge set; any edge e=(u, i, r)∈ε in the set ε indicates that user u has a clear rating level r for item i.

[0107] The steps of generating user and item embedding information by the multi-attribute embedding information generation module include:

[0108] 1) Extract M latent attribute spaces from the user-item bipartite graph; where the mth latent attribute space affects the mth interaction possibility in the user-item interaction;

[0109] 2) Extracting user potential attribute space and the project potential attribute space

[0110] Among them, the mth potential attribute space of user u is and the mth latent attribute space of item i They are as follows:

[0111]

[0112]

[0113] Where W={W1,W2,…,W M} is the user potential attribute space transformation matrix; Q = {Q1, Q2, ..., Q M} is the transformation matrix of the project potential attribute space; m=1,…,M; u u 、p i are the characteristics of user u and item i;

[0114] 3) Calculate the possibility of user u interacting with item i in the mth latent attribute space Right now:

[0115]

[0116] Where, att node represents a node-level attention neural network;

[0117] 4) Use the softmax function to calculate the weight coefficient Right now:

[0118]

[0119] Where σ represents the activation function; a m represents the node-level attention vector in the mth latent attribute space; For the project set;

[0120] 5) Calculate the potential attribute space set of user u Among them, the mth potential attribute feature of user u is As shown below:

[0121]

[0122] Calculate the weight of each potential attribute space, that is:

[0123]

[0124] Where, Represents the weight of the Mth potential attribute space, att spacRepresents attribute space-level attention neural network;

[0125] 6) Calculate the joint embedding of user u’s own potential attribute features and its potential attribute space Right now:

[0126]

[0127] Where C m represents the weight matrix; b m represents the bias vector;

[0128] Calculate the importance of the mth latent attribute space Right now:

[0129]

[0130] Where q represents the attention vector; b represents the bias;

[0131] Normalized calculation to obtain the weight of the mth potential attribute space Right now:

[0132]

[0133] 7) Generate user embedding information z of the bipartite graph module u and project embedding information z i ,Right now:

[0134]

[0135] Where, is the weight of the mth potential attribute space of item i; is the mth potential attribute feature of item i.

[0136] The step of constructing implicit relationship information by the implicit relationship construction module includes:

[0137] 1) Through the user-item bipartite graph, we construct a user-user graph and an item-item graph based on cosine similarity to represent similar users and similar items, and model the implicit relationship between users and items based on multiple graphs;

[0138] First, calculate the joint embedding c of users with similar preferences u ,Right now:

[0139]

[0140] Where u n represents the nth user; u (n，l) Indicates that the user u n The lth user with similar preferences; ⊙ represents the dot product;

[0141] 2) Establish a joint embedding matrix Right now:

[0142]

[0143] Where, is the memory matrix; j = 1, ..., N; N is the number of similar users in different latent attribute spaces;

[0144] 3) Calculate the attention weights of similar users Right now:

[0145]

[0146] Where K represents the attention key matrix of similar users in different latent attribute spaces; Kj is the element of matrix K;

[0147] 4) Normalize to get the attention weight α of similar users j ,Right now:

[0148]

[0149] 5) Calculate the correlation vector Right now:

[0150]

[0151] In the formula, the correlation vector Used to represent user u n The lth user with similar preferences to user u n The influence of preferences;

[0152] 6) Establish a deep neural network for determining the weight of the association vector, namely:

[0153]

[0154] Where, Represents the association vector The weight of

[0155] The input of the deep neural network is the association vector, and the output is the association vector weight;

[0156] Relevance vector weight As shown below:

[0157]

[0158] Where W is the model parameter;

[0159] 7) Calculate the influence weight of each user with similar preferences, that is:

[0160]

[0161] Where, L n Represents all users with n Users with similar preferences;

[0162] 8) Generate implicit relationship information v u ,Right now:

[0163]

[0164] The data fusion module and the recommendation module store an MLP network;

[0165] The input of the MLP network is user association information U u =[z u ||v u ]·A u Project related information i =[z i ||v i ]·A i , the output is user u’s preference level r’ for item i ui ;Intermediate parameter A u =softmax(σ(W a1 ·z u +W a2 ·v u )), A i =softmax(σ(W a1 ·z i +W a2 ·v i ));v u 、v i are the implicit relationship information of users and items respectively; W a1 、W a2 is the calculation coefficient;

[0166] The MLP network outputs user u’s preference level r’ for item i ui The process is as follows:

[0167] g1=[U u ||I i ],

[0168] g2=σ(W2·g1+b2),

[0169] …

[0170] g l =σ(W l ·g l-1 +b l ),

[0171] r′ ui =w T ·g l , (20)

[0172] Where l is the index of the hidden layer; g l represents the hidden layer; w is the weight, b l For bias.

[0173] Furthermore, the objective function of the MLP network is As shown below:

[0174]

[0175] Where, training parameters θ = {W, Q}; λ and θ represent regularization weights;

[0176] Among them, the function equation As shown below:

[0177]

[0178] Where, represents the observed preference level; r ui represents the true preference level of user u for item i.

[0179] The method for the recommendation module to send recommended item information to the user based on the user's preference level for the item is: sending items with a preference level greater than a threshold ε to the user.

[0180] Furthermore, it also includes a database for storing user-item bipartite graph generation module, multi-attribute embedding information generation module, implicit relationship construction module, data fusion module and recommendation module data.

[0181] Example 2:

[0182] See also Figure 2 and Figure 3 ,A collaborative filtering recommendation system based on graph neural network, includes a user-item bipartite graph generation module, a multi-attribute embedding information generation module, an implicit relationship construction module, a data fusion module and a recommendation module.

[0183] The user-item bipartite graph generation module obtains the user's ratings of several recommended items and generates a user-item bipartite graph And transmitted to the multi-attribute embedding information generation module and the implicit relationship construction module.

[0184] Modeling user-item ratings as a user-item bipartite graph in, and Represent the user set and the item set to be recommended respectively. and Represent the feature matrices of the user set and the item set to be recommended. u is the total number of users. N i is the total number of items to be recommended. d represents the feature dimension. Contains the user's rating for each item they interacted with. Rating Set R is the highest rating. ε represents the edge set. Any edge e = (u, i, r) ∈ ε in the set ε indicates that user u has a clear rating r for item i.

[0185] The multi-attribute embedding information generation module generates the user-item bipartite graph Processing is performed to generate user multi-attribute embedding information and item multi-attribute embedding information, and then transmitted to the data fusion module.

[0186] The steps of generating user and item embedding information by the multi-attribute embedding information generation module include:

[0187] 1) Extract M latent attribute spaces from the user-item bipartite graph, where the mth latent attribute space affects the mth interaction likelihood in the user-item interaction.

[0188] 2) Extracting user potential attribute space and the project potential attribute space

[0189] Among them, the mth potential attribute space of user u is and the mth latent attribute space of item i They are as follows:

[0190]

[0191]

[0192] Where W={W1,W2,…,W M} is the user potential attribute space transformation matrix. Q={Q1,Q2,…,Q M} is the transformation matrix of the project potential attribute space.

[0193] 3) Calculate the possibility of user u interacting with item i in the mth latent attribute space Right now:

[0194]

[0195] Where, att node Represents a node-level attention neural network.

[0196] 4) Use the softmax function to calculate the weight coefficient Right now:

[0197]

[0198] Where σ represents the activation function. m represents the node-level attention vector under the m-th latent attribute space. For project sets.

[0199] 5) Calculate the potential attribute space set of user u Among them, the mth potential attribute feature of user u is As shown below:

[0200]

[0201] Calculate the weight of each potential attribute space, that is:

[0202]

[0203] Where, Represents the weight of the Mth potential attribute space, att spac Representing attribute space-level attention neural network.

[0204] 6) Calculate the joint embedding of user u’s own potential attribute features and its potential attribute space Right now:

[0205]

[0206] Where C m Represents the weight matrix. b m Represents the bias vector.

[0207] Calculate the importance of the mth latent attribute space Right now:

[0208]

[0209] Where q represents the attention vector and b represents the bias.

[0210] Normalized calculation to obtain the weight of the mth potential attribute space Right now:

[0211]

[0212] 7) Generate user embedding information z of the bipartite graph module u and project embedding information z i ,Right now:

[0213]

[0214] Where, is the weight of the mth potential attribute space of item i. is the mth potential attribute feature of item i. Item embedding information z i The calculation process of is similar to that of user embedded information.

[0215] The implicit relationship construction module uses the user-item bipartite graph to construct user-user and item-item graphs to model implicit relationship information, and transmits the implicit relationship information to the data fusion module.

[0216] The step of constructing implicit relationship information by the implicit relationship construction module includes:

[0217] 1) Through the user-item bipartite graph, we construct a user-user graph and an item-item graph based on cosine similarity to represent similar users and similar items, and model the implicit relationships between users and items respectively based on multi-graphs.

[0218] First, calculate the joint embedding c of users with similar preferences u ,Right now:

[0219]

[0220] Where u n Indicates the nth user. u (n，l) Indicates that the user u n The lth user has similar preferences. ⊙ represents the dot product.

[0221] 2) Establish a joint embedding matrix Right now:

[0222]

[0223] Where, Memory Matrix.

[0224] 3) Calculate the attention weights of similar users Right now:

[0225]

[0226] Where K represents the attention key matrix of similar users in different latent attribute spaces.

[0227] 4) Normalize to get the attention weight α of similar users j ,Right now:

[0228]

[0229] 5) Calculate the correlation vector Right now:

[0230]

[0231] In the formula, the correlation vector Used to represent user u n The lth user with similar preferences to user u n The influence of preferences.

[0232] 6) Establish a deep neural network for determining the weight of the association vector, namely:

[0233]

[0234] Where, Represents the association vector The weight of .

[0235] The input of a deep neural network is a correlation vector, and the output is a correlation vector weight.

[0236] Relevance vector weight As shown below:

[0237]

[0238] Where W is the model parameter.

[0239] 7) Calculate the influence weight of each user with similar preferences, that is:

[0240]

[0241] Where, L n Represents all users with n Users with similar preferences.

[0242] 8) Generate implicit relationship information v u ,Right now:

[0243]

[0244] The data fusion module fuses the user multi-attribute embedding information, the item multi-attribute embedding information, and the implicit relationship information to obtain the final embedding representation of the user and the item. The final embedding representation is transmitted to the recommendation module.

[0245] The data fusion module and the recommendation module store an MLP network.

[0246] The input of the MLP network is user association information U u =[z u ||v u ]·A u Project related information i =[z i ||v i ]·A i, the output is user u’s preference level r’ for item i ui Intermediate parameter A u =softmax(σ(W a1 ·z u +W a2 ·v u )), A i =softmax(σ(W a1 ·z i +W a2 ·v i )).

[0247] The MLP network outputs user u’s preference level r’ for item i ui The process is as follows:

[0248] g1=[U u ||I i ],

[0249] g2=σ(W2·g1+b2),

[0250] …

[0251] g l =σ(W l ·g l-1 +b l ),

[0252] r′ ui =w T ·g l , (20)

[0253] Where l is the index of the hidden layer. g l represents the hidden layer. w is the weight and b is the bias.

[0254] Objective function of the MLP network As shown below:

[0255]

[0256] Where, the training parameters θ = {W, Q}. λ and θ represent regularization weights.

[0257] Among them, the function equation As shown below:

[0258]

[0259] Where, represents the observed preference level. r ui represents the true preference level of user u for item i.

[0260] The recommendation module calculates the user's preference rating for the item based on the final embedding representation of the user and the item.

[0261] The method for the recommendation module to send recommended item information to the user based on the user's preference level for the item is: sending items with a preference level greater than a threshold ε to the user.

[0262] A collaborative filtering recommendation system based on graph neural network also includes a database for storing user-item bipartite graph generation module, multi-attribute embedding information generation module, implicit relationship construction module, data fusion module and recommendation module data.

[0263] Example 2:

[0264] See also Figure 4 and Figure 5 , a verification experiment of collaborative filtering recommendation system based on graph neural network, the content is as follows:

[0265] This example conducts experiments on three real-world datasets to evaluate its model. Furthermore, this example conducts extensive ablation studies for each proposed component. The experiments also explore the impact of different information fusion methods on the results.

[0266] Datasets and evaluation metrics

[0267] This paper conducts extensive experiments on three real-world datasets: MovieLens, Amazon, and Yelp, which are publicly accessible and vary in domain, size, and sparsity.

[0268] -MovieLens-100K: A widely used benchmark dataset for movie recommendation, containing 100,000 ratings of 1,682 movies by 943 users.

[0269] Amazon: A widely used product recommendation dataset containing 65,170 ratings from 1,000 users to 1,000 items.

[0270] Yelp: Business review dataset, containing 30,838 ratings from 1,286 users on 2,614 items.

[0271] For each data set, this embodiment randomly selects 80% of the historical ratings as the training set and the rest as the test set.

[0272] For all experiments, this embodiment evaluates the model of this embodiment and the baseline according to two widely used evaluation protocols: root mean square error (RMSE) and mean absolute error (MAE) as evaluation metrics.

[0273] Baseline

[0274] This example studies the performance of the following models: Matrix factorization methods: PMF

[18] , BiasMF

[19] , and LLORMA-Local

[20] . Autoencoder-based methods: AUTOREC

[21] and CF-NADE

[22] . Graph Convolutional Network-based collaborative filtering model: GC-MC

[23] . In addition, this example uses a project-based setting for I-AUTOREC and I-CF-NADE, which achieves better performance than a user-based setting.

[0275] Parameter settings

[0276] In this embodiment, the model parameters are randomly initialized with Gaussian distribution, and then Adam is used as the optimizer. The batch size and learning rate are selected from {64, 128, 256} and {0.0005, 0.001, 0.002} respectively. Except for the multi-attribute extraction part, the dropout method is applied, and its probability is selected from {0.3, 0.4, 0.5} for testing. The parameters of L0 regularization are set according to the literature

[25] . In this embodiment, the number of attribute spaces is changed in the range of {1, 2, 3, 4}. For the attention-based memory module, the number of memory slices in M ​​is set to 4 for Yelp and 8 for Amazon and Movielens. For the neural network, this embodiment adopts two layers for all neural parts based on experience, and the activation function is ReLU. The model is implemented in Pytorch, and the embedding dimension is selected from {16, 32, 64, 128, 256, 512} for different experiments. All baselines are initialized as the corresponding papers, and in terms of neural network models, this embodiment uses the same embedding dimension for fair comparison. Then carefully tune it for optimal performance.

[0277] Comparison with baseline

[0278] Table 2 reports the overall performance compared to the baseline. Each result is the average performance of 5 runs with random initialization. Based on these results, this example draws the following conclusions:

[0279] The model of this embodiment consistently outperforms all baselines, demonstrating its effectiveness in recommendation. More specifically, the model of this embodiment improves the best RMSE by 9.32%, 1.70%, and 1.28%, respectively. For MAE, the model of this embodiment outperforms the best by 8.94%, 1.51%, and 1.15%, respectively. This demonstrates that by leveraging multiple graphs and multiple attention mechanisms to mine latent information, the model of this embodiment can better predict ratings.

[0280] The Yelp dataset has high sparsity, but the performance of the model in this example on the Yelp dataset has been significantly improved. This fully demonstrates that by adding multiple graphs, we can better obtain information, thereby more effectively alleviating the sparsity problem when using collaborative filtering.

[0281] It can be observed that I-AUTOREC, I-CFENDO, GCMC, and MCCF generally outperform PMF, BiasMF, and LLORMA-Local, which shows the power of neural network models. At the same time, among these baselines, the overall performance of neural network-based models is better than other models, which means that neural networks have a strong role in the performance of graph data.

[0282] Ablation studies

[0283] This example performs ablation analysis on the Yelp and Amazon datasets, starting with only the Bipar-GCN layer, then adding the IRM layer without the storage module, and finally adding the full IRM layer. Table 3 shows the contribution of each component. The embedding dimension for all ablation experiments is 128. For the best-performing baseline, this example sets its embedding dimension to 64, as this yields better results than the 128-degree embedding.

[0284] This embodiment proposes a novel recommendation system model MI-CF, which aims to model multi-attribute and implicit relational factors for collaborative filtering recommendation systems. The idea is to explicitly explore the two factors that influence user purchase motivations in order to reveal the fine-grained factors behind the interaction. First, the user-item bipartite graph is used to model the multi-attributes of users and items, and the latent semantics of specific user-item pairs are encoded and represented as attribute spaces. Secondly, the user-user graph and item-item graph are explicitly modeled, and the memory attention network and relational attention are used to model the nodes at a fine-grained level. Three embedded available data are constructed from two perspectives, which greatly enriches the representation capability and reflects fine-grained user preferences. Extensive experiments on three real datasets demonstrate the effectiveness of the method of this embodiment, and ablation studies quantitatively verify the important contribution made by each component.

Claims

1. A collaborative filtering recommendation system based on graph neural network, characterized by: It includes user-item bipartite graph generation module, multi-attribute embedding information generation module, implicit relationship construction module, data fusion module and recommendation module; The user-item bipartite graph generation module obtains the user's ratings of several recommended items and generates a user-item bipartite graph And transmit it to the multi-attribute embedding information generation module and implicit relationship construction module; The multi-attribute embedding information generation module generates the user-item bipartite graph Processing is performed to generate user multi-attribute embedding information and item multi-attribute embedding information, and transmitted to the data fusion module; The implicit relationship construction module uses the user-item bipartite graph to construct user-user and item-item graphs to model implicit relationship information, and transmits it to the data fusion module; The data fusion module fuses the user multi-attribute embedding information, the project multi-attribute embedding information, and the implicit relationship information to obtain the final embedding representation of the user and the project; The final embedding representation is transmitted to the recommendation module; The recommendation module calculates the user's preference rating for the item based on the final embedding representation of the user and the item; The steps of generating user and item embedding information by the multi-attribute embedding information generation module include: a1) Extract M latent attribute spaces from the user-item bipartite graph; where the mth latent attribute space affects the mth interaction probability in the user-item interaction; a2) Extracting user potential attribute space and the project potential attribute space Among them, the mth potential attribute space of user u is and the mth latent attribute space of item i They are as follows: Where W={W1,W2,…,W M } is the user potential attribute space transformation matrix; Q = {Q1, Q2, ..., Q M } is the transformation matrix of the project potential attribute space; m=1,…,M; u u 、p i are the characteristics of user u and item i; a3) Calculate the likelihood of user u interacting with item i in the mth latent attribute space Right now: Where, att node represents a node-level attention neural network; a4) Use the softmax function to calculate the weight coefficient Right now: Where σ represents the activation function; a m represents the node-level attention vector in the mth latent attribute space; For the project set; a5) Calculate the potential attribute space set of user u Among them, the mth potential attribute feature of user u is As shown below: Calculate the weight of each potential attribute space, that is: Where, Represents the weight of the Mth potential attribute space, att spac Represents attribute space-level attention neural network; a6) Calculate the joint embedding of user u’s own latent attribute features and its latent attribute space Right now: Where C m represents the weight matrix; b m represents the bias vector; Calculate the importance of the mth latent attribute space Right now: Where q represents the attention vector; b represents the bias; Normalized calculation to obtain the weight of the mth potential attribute space Right now: a7) Generate user embedding information z of the bipartite graph module u and project embedding information z i ,Right now: Where, is the weight of the mth potential attribute space of item i; is the mth potential attribute feature of item i; The step of constructing implicit relationship information by the implicit relationship construction module includes: b1) Using the user-item bipartite graph, we construct a user-user graph and an item-item graph based on cosine similarity to represent similar users and similar items, and model the implicit relationships between users and items using multiple graphs. First, calculate the joint embedding c of users with similar preferences u ,Right now: Where u n represents the nth user; u (n，l) Indicates that the user u n The lth user with similar preferences; ⊙ represents the dot product; b2) Build a joint embedding matrix Right now: Where, is the memory matrix; j = 1,…, N; N is the number of similar users in different latent attribute spaces; b3) Calculate the attention weights of similar users Right now: Where K represents the attention key matrix of similar users in different latent attribute spaces; Kj is the element of matrix K; b4) Normalize to get the attention weight α of similar users j ,Right now: b5) Calculate the correlation vector Right now: In the formula, the correlation vector Used to represent user u n The lth user with similar preferences to user u n The influence of preferences; b6) Establishing a deep neural network for determining the weights of the association vectors, namely: Where, Represents the association vector The weight of The input of the deep neural network is the association vector, and the output is the association vector weight; Relevance vector weight As shown below: Where W is the model parameter; b7) Calculate the influence weight of each user with similar preferences, that is: Where, L n Represents all users with n Users with similar preferences; b8) Generate implicit relationship information v u ,Right now: The data fusion module and the recommendation module store an MLP network; The input of the MLP network is user association information U u =[z u ||u u ]·A u Project related information i =[z i ||v i ]·A i , the output is user u’s preference level r’ for item i ui ;Intermediate parameter A u =softmax(σ(W a1 ·z u +W a2 ·v u )), A i =softmax(σ(W a1 ·z i +W a2 ·v i ));v u 、v i are the implicit relationship information of users and items respectively; W a1 、W a2 is the calculation coefficient; The MLP network outputs user u’s preference rating r′x for item i i The process is as follows: g1=[U u ||I i ], g2=σ(W2·g1+b2), … 9 l =σ(W l ·g l-1 +b l ), r′ ui =w T ·g l , (20) Where l is the index of the hidden layer; g l represents the hidden layer; w is the weight, b l For bias.

2. The collaborative filtering recommendation system based on graph neural network according to claim 1, characterized in that: Modeling user-item ratings as a user-item bipartite graph in, and Represent the user set and the set of items to be recommended respectively; and Represents the feature matrices of the user set and the item set to be recommended respectively; N u is the total number of users; N i is the total number of items to be recommended; d represents the feature dimension; the scoring set Contains the user's rating of each interacted item; rating set R is the highest rating level; ε represents the edge set; any edge e=(u, i, r)∈ε in the set ε indicates that user u has a clear rating level r for item i.

3. The collaborative filtering recommendation system based on graph neural network according to claim 1, characterized in that: Objective function of the MLP network As shown below: Where, training parameters θ = {W, Q}; λ and θ represent regularization weights; Among them, the function equation As shown below: Where, represents the observed preference level; r ui represents the true preference level of user u for item i.

4. The collaborative filtering recommendation system based on graph neural network according to claim 1, characterized in that: The method for the recommendation module to send recommended item information to the user based on the user's preference level for the item is: sending items with a preference level greater than a threshold ε to the user.

5. The collaborative filtering recommendation system based on graph neural network according to claim 1, characterized in that: It also includes a database for storing user-item bipartite graph generation module, multi-attribute embedding information generation module, implicit relationship construction module, data fusion module and recommendation module data.

Images