A Graph Neural Network Session Recommendation Method Incorporating Correlation Information

Abstract

The present invention discloses a graph neural network session recommendation method integrating correlation information, belonging to the technical field of session recommendation. The present invention converts all items in all sequences into an undirected graph and then imports it into the relevant information embedding module to model the correlation information; at the same time, each sequence is converted into a directed graph, and the transfer information between items is captured based on the transfer relationship, and the transfer information embedding module is used to model the transfer information. After merging the correlation information and the transfer information, it is passed to the session representation module to obtain the final representation of the current user interaction sequence. The encoder is used to process the interaction timestamp sequence, and the decoder is used to process the interaction item sequence, so as to integrate the specific timing information of each interaction behavior to capture a more fine-grained user behavior pattern. The present invention overcomes the defect that the existing method does not consider the correlation between items, weakens the influence brought by the noise in the sequence, and improves the recommendation effect.

Description

Technical Field

[0001] The present invention belongs to the technical field of conversational recommendation, and particularly relates to a graph neural network conversational recommendation method that integrates relevance information. Background Art

[0002] Existing conversational recommendation methods mainly model the user (possibly anonymous) historical interaction behavior sequence through common sequence modeling algorithms (such as Markov chains, recurrent neural networks, graph neural network models) to capture effective user behavior patterns, so as to predict the items that the user may interact with in the future. However, the algorithms of existing conversational recommendation models still have deficiencies: existing conversational recommendation methods only model the transfer information between items in the sequence, while ignoring the relevance information between items. This relevance information is also extremely important for modeling the user interaction sequence. Only using transfer information makes the sequence modeling not only vulnerable to noise in the sequence, but also difficult to deeply and effectively model complex user behavior patterns. Summary of the Invention

[0003] The present invention provides a graph neural network conversational recommendation method that integrates relevance information, which can be used to improve the recommendation effect of items.

[0004] The technical solution adopted by the present invention is as follows:

[0005] A graph neural network conversational recommendation method that integrates relevance information, the method includes:

[0006] Step 1, construct and train a recommendation model;

[0007] The recommendation model includes a relevance information module, a transfer information module, and a user representation module;

[0008] Among them, the input of the recommendation model is the user-item interaction sequence [s1, s2,..., s l …, s L , s l represents the l-th interaction sequence, where l = 1, 2,..., L, and L represents the number of interaction sequences;

[0009] Construct all items in all interaction sequences into an undirected graph of items. In the undirected graph of items, if two items are adjacent in the sequence, they are connected, and define A as the adjacency matrix of the undirected graph of items,

[0010] Define a learnable item embedding matrix E, and the embedding information of any item i is represented by e i ;

[0011] Add the identity matrix I N with dimension N×N to the adjacency matrix A Among them, N represents the number of items in the item undirected graph;

[0012] The relevance information module uses a multi-layer graph convolutional neural network to obtain the relevance information embedding representation of items. The input of the relevance information module (the first layer of the graph convolutional neural network) includes: the matrix and the initialized item embedding matrix E. Each layer of the graph convolutional neural network updates the item embedding matrix E according to the relevance information embedding representation output by the previous layer. The specific update formula is: Among them, represents the degree matrix of the item undirected graph, respectively represent the relevance information embedding representations updated by the k-th layer and the (k + 1)-th layer of the graph convolutional neural network;

[0013] The final relevance embedding representation E of all items output by the relevance information module r is the weighted sum of the embedding representations obtained in each layer of the graph convolutional network: Among them, β k represents the weight value of each layer of the graph convolutional neural network, and E (0) represents the initialized item embedding matrix E;

[0014] Convert all items in each user-item interaction sequence into a directed graph. In the directed graph, if two items are adjacent in the current sequence, they are connected;

[0015] Use the directed graph of each user-item interaction sequence and the E output by the relevance information module r as the input of the transferability information module. The transferability information module is based on the attention mechanism and calculates the attention coefficient θ between any two adjacent items i and j in the current user-item interaction sequence according to , where Att() represents the attention function, ij , and represents the weight parameter representing the transfer relationship between items i and j, and e i , e j represent the embedding information of items i and j, which is determined based on the E output by the relevance information module r ;

[0016] The transferability information module then converts the weight parameter into a probability form through the softmax function: Among them, represents the set of items that are the neighbor nodes of item i in the directed graph formed by the current sequence s, k represents the item number in the set , and θ ik represents the attention coefficient between items i and k;

[0017] The metastatic information embedding representation of item i finally output by the metastatic information module in the current sequence s is expressed as the weighted sum of its own embedding representation and the embedding representations of its neighbor nodes: Among them, represents the embedding information of item j in the current sequence s, determined based on the E output by the relevance information module r Determine;

[0018] For each item i in the current sequence, the user representation module adds the metastatic information embedding representation output by the metastatic information module and the relevance embedding representation e ri output by the relevance information module to obtain the embedding representation of each item and obtains the global embedding representation s of the current sequence through average pooling operation g : Among them, n represents the number of items in the current sequence;

[0019] The user representation module obtains the local embedding representation s of the previous sequence based on the position embedding l :

[0020] For each interaction sequence s, there is a corresponding position embedding matrix P = [p1, p2, …, p i …, p t , where p i represents the position embedding vector of the i-th item, and the feature vector of item i in the interaction sequence s is calculated according to and the user's short-term interest representation of item i is obtained according to and then the local embedding representation s is obtained according to where W1, W2, and W3 represent the corresponding weights, learnable parameters, b represents the bias term of the feature vector, and c represents the bias term of the user's short-term interest representation, all of which are learnable parameters; l The user representation module concatenates the global embedding representation and the local embedding representation to obtain the embedding representation S of the current sequence = [s

[0021] || s l || s g ;

[0022] And calculate the preference score of the current sequence for each item by taking the inner product of the embedding representation S of the current sequence and the embedding representation of each item :

[0023] ​Perform deep learning training on the learnable parameters in the relevance information module, transferability information module, and user representation module based on the set training dataset, and stop when the preset training end condition (loss convergence or reaching the maximum number of training times) is met to obtain a trained recommendation model;

[0024] Step 2, obtain a recommendation list based on the trained recommendation model:

[0025] Use the trained recommendation model to make predictions on the dataset to be processed. For each user-item interaction sequence in the dataset to be processed, calculate its preference score for each item through the recommendation model, and recommend the top K (K is greater than or equal to 1) items with the largest preference scores as the recommendation list and push them to the user corresponding to the current interaction sequence. Further, LeakyRelu is used as the activation function to obtain α ij :

[0026]

[0027] Among them, ⊙ represents element-wise multiplication in the vector.

[0028] The technical solution provided by the present invention at least brings the following beneficial effects:

[0029] The present invention proposes a model based on a graph neural network, which simultaneously learns two types of information between items: uses the attention mechanism to learn the transferability information of items in the sequence in the directed graph composed of the sequence, so as to better capture the long-term and short-term interests of users; at the same time, the present invention uses a graph convolutional neural network to learn the relevance information between items in the undirected graph composed of items, overcomes the defect that the existing method does not consider the relevance between items, weakens the influence brought by the noise in the sequence, and improves the recommendation effect. Description of the Drawings

[0030] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following will briefly introduce the drawings required for the description of the embodiments. Obviously, the following drawings are only some embodiments of the present invention. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative efforts.

[0031] Figure 1 It is a schematic structural diagram of the recommendation model adopted in the embodiment of the present invention. Detailed Embodiments

[0032] To make the purpose, technical solutions, and advantages of the present invention clearer, the following will further describe the embodiments of the present invention in detail with reference to the drawings.

[0033] In view of the drawback that existing methods ignore the correlation information between items, the present invention proposes a unique model based on graph neural network, which fully utilizes two types of information, namely the correlation information between items and the transition information between items. More specifically, on the one hand, in order to obtain the item-to-item correlation information, the present invention converts all items in all sequences into a large undirected graph. If two items are adjacent in a certain sequence, they are connected, and the weight value of the connected edge is the number of times these two items appear adjacent. Then the graph is imported into the correlation information embedding module proposed by the present invention to model the correlation information. On the other hand, each sequence is converted into a directed graph. Based on predefined transition relationships (for example, in the same sequence, only item i transfers to item j, only item j transfers to item i, and there are mutual transfers between item i and item j, which are three different transition relationships), the transition information between items is captured, and then the transition information embedding module is used to model the transition information. In addition, in order to learn the long-term and short-term interests of users, after merging the obtained correlation information and transition information, it is passed to a session representation module to obtain the global representation (representing long-term interest) and local representation (representing short-term interest) of the user interaction sequence respectively. Finally, the global representation and the local representation are fused to obtain the final representation of the current user interaction sequence.

[0034] In view of the drawback that the temporal information cannot be used to model the user's historical interaction behavior, the present invention proposes a unique method with an encoder-decoder structure. The encoder is used to process the interaction timestamp sequence, and the decoder is used to process the interaction item sequence, so as to fuse the specific temporal information of each interaction behavior to capture a more fine-grained user behavior pattern. At the same time, in view of the continuity of the temporal information, the present invention uses a window function-based encoding method, which can better convert the timestamp into an embedded code. Combined with the self-attention mechanism, the redundant parts in the temporal information and item information are removed, and the key information is utilized to make a more accurate prediction of the user's recent interaction behavior, overcoming the defect that the existing methods do not consider the specific temporal information and improving the recommendation effect.

[0035] As Figure 1 shown, the session recommendation model adopted by the present invention includes: a module for capturing item correlation information, a module for capturing item transfer information, and a user representation module. The specific implementation of each module is as follows:

[0036] (1) Construction of the correlation information module.

[0037] The input of the entire model is the user-item interaction sequence [s1, s2, …, s L, and the input of the relevance information module has two parts in total: First, all items in all sequences are constructed into a large undirected graph, and two items are connected if they are adjacent in the sequence. Then, the adjacency matrix A of the undirected graph of all items belongs to R N×N As the first part of the input of the relevance information module, where N represents the number of items. For the item sequence, a learnable item embedding matrix E belongs to R N ×d , d represents the dimension of the item embedding information, and the embedding information of each item i is represented by e i , and this item embedding matrix is the second part of the input of the relevance information module. In this module, first, the identity matrix I is added to the adjacency matrix N to strengthen its self-interaction information:

[0038]

[0039] A multi-layer graph convolutional neural network is used in the module. In each layer of the graph convolutional neural network, the relevance information embedding representation of all items is updated according to the result of the previous layer. The specific update formula is as follows:

[0040]

[0041] Among them, is the degree matrix of the undirected graph, is equivalent to performing symmetric normalization on the adjacency matrix , represents the relevance information embedding representation updated by the k-th layer of the graph convolutional neural network. For each item, its relevance information update formula is:

[0042]

[0043] Among them, N i represents the number of neighbor nodes of item i in the undirected graph, N j represents the number of neighbor nodes of item j in the undirected graph, represents the k-th update of the relevance information of item i. Each update is equivalent to performing an aggregation operation on the neighbor nodes around item i. Finally, the relevance embedding representation of all items is the weighted sum of the embedding representations obtained in each layer of the graph convolutional network, that is:

[0044]

[0045] Among them, β k is the weight value of each layer and is a learnable hyperparameter. E (0) is the item embedding matrix input at the beginning and can be randomly initialized.

[0046] (2) Construction of the transfer information module.

[0047] As Figure 1 shown, the input of the transfer information module also consists of two parts. First, each sequence is converted into a directed graph, and all the directed graphs are used as the first part of the input of this module. The second part is the same as the input of the relevance information module, which is the embedding matrix E ∈ R N×d . Since the transfer relationships between different items in each directed graph are different, the attention mechanism is used to distinguish the importance of different transfer relationships for the construction of the transfer information embedding. In the first stage, for the embedding vectors e i and e j of items i and j whose corresponding items are adjacent in the same session, the attention coefficient θ ij is calculated, representing the importance of item j for i:

[0048]

[0049] ij where Att represents the attention function, and r represents the transfer relationship between items i and j. Therefore,

[0050]

[0051] represents the neighbor nodes of item i in the directed graph formed by the current sequence s. For Att, that is, the choice of the attention activation function is diverse. In the session recommendation model of the present invention, in order to better combine the information between the current node and the neighbor nodes, the method of multiplying the corresponding elements in the embedding vectors is used to combine the transfer information between items, and LeakyRelu is used as the activation function:

[0052]

[0053]

[0054]

[0055] (3) Construction of the user representation module.

[0056] By Figure 1As shown, after calculating the transfer information embedding of each item in the current sequence and the correlation information embedding of each item, it is necessary to combine the two types of information to further obtain the embedding representation of the entire sequence. First, for each item i in the current sequence s, its final embedding representation is the element sum of the two types of information:

[0057]

[0058] After that, in order to capture the long-term interest and short-term interest of the user respectively, a sequence global embedding is constructed to represent the long-term interest of the user, and a local embedding of the sequence is used to represent the short-term interest of the user. For the global embedding representation, an average pooling strategy is used to combine the information of all items in the current sequence:

[0059]

[0060] Among them, n represents the number of items in the current sequence.

[0061] For the local embedding representation of the user, the present invention believes that the later an item is clicked in the user interaction sequence, the more it can represent the short-term interest of the user. Therefore, the concept of position embedding is introduced. For each sequence s, there is a corresponding position embedding matrix P = [p1, p2,..., p i …, p t , where p i ∈R d is a learnable embedding vector. In order to reflect that the later an item is clicked, the more it can represent the short-term interest of the user, the item embedding vector and the position embedding vector are concatenated in reverse order:

[0062]

[0063] Among them, tanh is the activation function, and both W1 and b are learnable parameters. After that, a soft attention mechanism is used to obtain the weight of each item in each sequence for the short-term interest of the user, and finally a weighted sum is used to obtain the final short-term interest representation of the user.

[0064]

[0065]

[0066] Among them, W2 and W3 represent the corresponding weights, and c represents the bias term of the soft attention mechanism, all of which are learnable parameters.

[0067] Finally, the embedding representation of the current sequence is the concatenation of the global embedding representation and the local embedding representation of the sequence:

[0068] S = [s l ||s g ​

[0069] Finally, the preference score of each item in the sequence is calculated by taking the inner product of the embedding representation of the current sequence and the embedding representation of each item:

[0070]

[0071]

[0072] Among them, the final calculation result y i This means that the next possible interactive item in the current sequence is v i The preference scores of the first K (K is greater than or equal to 1) items with the largest preference scores are recommended as a recommendation list and pushed to the user corresponding to the current interaction sequence.

[0073] In order to further verify the session recommendation performance of the present invention, the present invention evaluates the model proposed by the present invention on three public datasets from display scenarios. These datasets have different sparsity, size, and time span.

[0074] The Diginetica dataset comes from the CIKM Cup and contains typical transaction data; the Nowplaying dataset consists of users' music listening behaviors; the Tmall dataset comes from the IJCAI competition and consists of user-item interactions on the Tmall online shopping platform.

[0075] Table 1 Dataset characteristics (after preprocessing)

[0076] dataset total number of interactions number of training sequences number of test sequences total number of items average length of sequences Diginetica 982,961 719,470 60,858 43,097 5.12 Nowplaying 1,367,963 825,304 89,824 60,417 7.42 Tmall 818,479 351,268 25,898 40,728 6.69

[0077] The present invention uses the following two evaluation indicators that are widely used in the industry:

[0078] 1. Hit Ratio (HR): HR mainly measures the accuracy of recommendations.

[0079] 3. Mean Reciprocal Rank (MRR): MRR measures the recommendation effect of the model by measuring the position of the correctly recommended items in the recommendation list.

[0080] In this embodiment, the method for comparison with the method of the present invention is as follows:

[0081] Pop: The simplest baseline, recommending items based on their popularity.

[0082] Item-KNN: Recommend items similar to the items in the current sequence based on cosine similarity

[0083] FMPC: A hybrid model combining Markov chain and matrix factorization, used to capture user preferences and sequential information.

[0084] GRU4Rec: A model that uses RNN to model users' historical interaction behaviors, and proposes a "session parallel" method to accelerate.

[0085] NARM: This model combines RNN with an attention mechanism to capture users' sequential behavior characteristics and main purposes.

[0086] STAMP: Adopts a self-attention mechanism, not only simulates users' long-term behaviors by capturing general interests, but also focuses on enhancing the influence of users' short-term interests based on the last click of users in the session.

[0087] SR-GNN: Uses a graph neural network for session-based recommendation. It captures information between items by converting the sequence into a directed graph. And like STAMP, it uses a self-attention mechanism to learn the final representation of the sequence.

[0088] FGNN: Learns the item embedding by designing a weighted attention layer, thus considering the potential order of the sequence.

[0089] GCE-GNN: Establishes two different sequence graphs to obtain global information and local information.

[0090] The comparison results of the recommendation results of the present invention and the above-mentioned existing methods are shown in Table 2:

[0091] Table 2

[0092]

[0093] As can be seen from Table 2, the RA-GNN proposed by the present invention has obtained the best results in various metrics on three datasets. As shown in Table 2, the bold font is the best result under the current metric.

[0094] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that they can still modify the technical solutions recorded in the foregoing embodiments, or perform equivalent replacements for some of the technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

[0095] The above are only some embodiments of the present invention. For those of ordinary skill in the art, without departing from the creative concept of the present invention, several deformations and improvements can still be made, and these all belong to the protection scope of the present invention.

Claims

1. A graph neural network session recommendation method that fuses correlation information, characterized in that, It includes the following steps: Step 1, construct and train a recommendation model; The recommendation model includes a correlation information module, a transfer information module, and a user representation module; Among them, the input of the recommendation model is the user-item interaction sequence [s1, s2, …, s l …, s L , where s l represents the l-th interaction sequence, where l = 1, 2, …, L, and L represents the number of interaction sequences; Construct all items in all interaction sequences into an undirected graph of items. In the undirected graph of items, if two items are adjacent in the sequence, they are connected, and define A as the adjacency matrix of the undirected graph of items. Define a learnable item embedding matrix E, and represent the embedding information of any item i with e i ; Add the adjacency matrix A to the identity matrix I of dimension N×N N , to obtain the matrix where N represents the number of items in the undirected graph of items; The relevance information module uses a multi-layer graph convolutional neural network to obtain the relevance information embedding representation of items. The inputs of the relevance information module include: matrix and the initialized item embedding matrix E. Each layer of the graph convolutional neural network updates the item embedding matrix E according to the relevance information embedding representation output by the previous layer. The specific update formula is: where represents the degree matrix of the item undirected graph, respectively represent the relevance information embedding representations updated by the k-th and (k + 1)-th layers of the graph convolutional neural network; The final relevance embedding representation E of all items output by the relevance information module r Is the weighted sum of the embedding representations obtained in each layer of the graph convolutional network: Where, β k Represents the weight value of each layer of the graph convolutional neural network, which is a learnable parameter, and E (0) Represents the initialized item embedding matrix E; Convert all items in each user-item interaction sequence into a directed graph. In the directed graph, if two items are adjacent in the current sequence, they are connected; Take the directed graph of each user-item interaction sequence and the E output by the relevance information module r as the input of the transfer information module. The transfer information module, based on the attention mechanism, according to calculate the attention coefficient θ between any two adjacent items i and j in the current user-item interaction sequence ij , where Att() represents the attention function, represents the weight parameter indicating the transfer relationship between items i and j, which is a learnable parameter, e i , e j represents the embedding information of items i and j, which is determined based on the E output by the relevance information module r ; The transfer information module then converts the weight parameters into a probability form through the softmax function: Among them, represents the set of items that are the neighbor nodes of item i in the directed graph formed by the current sequence s, and k represents the item number in the set θ ik represents the attention coefficient between item i and k; The metastatic information embedding representation of item i finally output by the metastatic information module in the current sequence s is expressed as the weighted sum of its own embedding representation and the embedding representations of its neighbor nodes: where represents the embedding information of item j in the current sequence s, determined based on the E output by the correlation information module r determine; For each item i in the current sequence, the user representation module embeds the transfer information output by the transfer information module and adds it to the relevance embedding representation e output by the relevance information module ri to obtain the embedding representation of each item and obtains the global embedding representation s of the current sequence through an average pooling operation g : where n represents the number of items in the current sequence; The user representation module obtains the local embedding representation s of the previous sequence based on the positional embedding l : For each interaction sequence s, there is a corresponding position embedding matrix P = [p1, p2, …, p i …, p t , where p i represents the position embedding vector of the i-th item. According to , the feature vector f i s of item i in the interaction sequence s is calculated, and according to , the short-term user interest representation of item i is obtained. Then, according to , the local embedding representation s l is obtained, where W1, W2, and W3 represent the corresponding weights, which are learnable parameters, b represents the bias term of the feature vector, c represents the bias term of the short-term user interest representation, and both b and c are learnable parameters; The user representation module splices the global embedding representation and the local embedding representation to obtain the embedding representation S of the current sequence = [s l ||s g ; And by using the embedding representation S of the current sequence and the embedding representation of each item calculate the preference score of the current sequence for each item by taking the inner product: Perform deep learning training on the learnable parameters in the correlation information module, the transfer information module, and the user representation module based on the set training data set, and stop when the preset training end condition is met to obtain a trained recommendation model; Step 2, obtain a recommendation list based on the trained recommendation model: Use the trained recommendation model to make predictions on the data set to be processed. For each user-item interaction sequence in the data set to be processed, calculate its preference score for each item through the recommendation model, and then use the top K items with the largest preference scores as the recommendation list and push them to the user corresponding to the current interaction sequence, where K is greater than or equal to 1.

2. The method according to claim 1, characterized in that LeakyRelu is used as an activation function to obtain α ij : Among them, ⊙ represents element-wise multiplication in the vector, and LeakyRelu() represents the output of the LeakyRelu activation function.

Images