Session recommendation method based on graph attention and multi-interest extraction

Abstract

The application provides a conversation recommendation method based on graph attention and multi-interest extraction, and the implementation steps are as follows: obtaining a training sample set and a test sample set; constructing a conversation recommendation model based on graph attention and multi-interest extraction; defining a total loss function of the conversation recommendation model; iteratively training the conversation recommendation model; and obtaining a conversation recommendation result. The graph attention module in the application can suppress noise in interactive features, the multi-interest extraction module performs bilinear mapping on the interactive embedding vectors containing interactive features output by the graph attention module through a bilinear mapping network, and performs gated weighted fusion on the mapped interest embedding vectors, so that multiple different interests of a user can be obtained, and the accuracy of recommendation is effectively improved.

Description

TECHNICAL FIELD

[0001] The present application belongs to the field of information technology, and relates to a session recommendation method, in particular to a session recommendation method based on graph attention and multi-interest extraction, which can be used for movie, book, commodity and service recommendation. BACKGROUND

[0002] With the rapid development of e-commerce and social media based on Internet technology, the user scale and the explosive growth of item categories, the recommendation system has become an indispensable tool to improve the convenience of life and improve enterprise profitability. Generally speaking, most recommendation methods assume that long-term behavior records of users are included. However, this assumption does not hold in many real-world scenarios, such as scenarios where some users do not log in or scenarios where user privacy needs to be protected. At this time, it is often difficult to collect the user's login record information. Therefore, session recommendation is used to solve this problem. Specifically, session recommendation mainly predicts the next item that the user will interact with through the user's anonymous historical session, that is, it uses the user's session to recommend items. Session recommendation algorithms can analyze the user's interest preferences from the user's historical session and recommend items or content that the user may be interested in among a large amount of information.

[0003] Session recommendation methods include methods based on matrix decomposition and recurrent neural networks, etc. These methods are difficult to capture high-order relationships between different items. Since the graph structure data in the graph neural network connects multiple items in the form of edges, the graph neural network can directly capture the high-order relationship between different items, becoming a more attractive choice for recommendation systems. In the session recommendation method, after the user browses the items, the user will be recommended items that they may like, not only saving the user from the annoyance of constantly searching for similar items, but also allowing the user to compare prices. In the session recommendation method, the evaluation indicators are usually precision and mean reciprocal rank (MRR) for comparison simulation. Precision focuses on how many of the recommended results are truly relevant, that is, the proportion of items that the user likes or clicks in the recommended list. The higher the precision, the higher the value, and vice versa. The higher the indicator, the higher the accuracy of the session recommendation. Mean reciprocal rank (MRR) can measure the position of the user's test item in the list of possible interactive items generated for the user. The higher the position, the higher the value, and vice versa. The higher the indicator, the higher the ranking of the accurately recommended item.

[0004] The session recommendation method based on a deep learning model is a typical session recommendation method. In recent years, with the rapid development of graph neural networks, and the fact that user sessions can be naturally used to construct graph structure data, applying graph neural networks to session recommendation has become a new solution. For example, Northwest Normal University discloses a graph neural network session recommendation method based on double-channel information fusion in a patent application with the application number CN202211064385.9 and the publication number CN115470406A and the name "a graph neural network session recommendation method based on double-channel information fusion". The implementation scheme is as follows: step 1, constructing a global information enhanced session representation learning layer; step 2, constructing a transduction information enhanced session representation learning layer; step 3, fusing time information into the item representation in the session sequence for neighbor embedding learning. This method weakens the influence of irrelevant items introduced when using other session information on the performance of the model, and fuses time information when aggregating neighbors, thereby improving the performance of the recommendation system. However, this invention has the following disadvantages: the graph neural network directly uses simple graph convolution to obtain interaction features, which results in a large amount of noise in the interaction features, and the double-channel method cannot model multiple interests of users, resulting in low rating accuracy and poor accuracy of the recommendation system. SUMMARY

[0005] The present application aims to overcome the above-mentioned defects of the prior art and proposes a session recommendation method based on graph attention and multi-interest extraction to solve the technical problem of low recommendation accuracy in the prior art.

[0006] To achieve the above-mentioned purpose, the technical solution adopted by the present application includes the following steps:

[0007] (1) Obtain a training sample set and a test sample set:

[0008] Construct four kinds of session graphs represented by self-connection matrix m and interaction information of D user interested items from M sessions Input matrix Output matrix and input-output matrix Then, more than half of the interaction information corresponding to the session graphs form a training sample set, and the remaining interaction information corresponding to the session graphs form a test sample, where M≥500, each session S m and the interaction information of the dth user interested item is w m,d , represents the real number field;

[0009] (2) Construct a session recommendation model H based on graph attention and multi-interest extraction:

[0010] A session recommendation model H including a cascaded graph attention module, a readout function module and a prediction module, and a multi-interest extraction module is constructed; the graph attention module includes a cascaded input layer and L graph attention layers; the multi-interest extraction module includes L interest extraction modules, and the input end of each interest extraction module is connected with each graph attention layer respectively, and the output end of the multi-interest extraction module is connected with the prediction module; the interest extraction module includes C bilinear mapping networks B arranged in parallel and a gating network and an interest aggregation layer cascaded therewith, and L interest aggregation layers are connected with each other, wherein L≥2 and C≥2;

[0011] (3) defining a total loss function of the session recommendation model H

[0012]

[0013]

[0014]

[0015] σ1=sigmoid

[0016] wherein, is a main loss function, sigmoid is a sigmoid nonlinear activation function, and alpha is a weight, is a self-supervised loss function, y Sm,d , are respectively an interest anchor vector and a session vector of each session S m for the dth interaction information, a predicted score, are respectively an interest anchor vector and a session vector of the Lth layer of the session S m ;

[0017] (4) iteratively training the session recommendation model H:

[0018] forward propagating the training sample set as the input of the session recommendation model H to obtain the trained session recommendation model H * ;

[0019] (5) obtaining a session recommendation result:

[0020] forward propagating the test sample set as the input of the trained session recommendation model H * to obtain a predicted score of a session vector for each interaction information, and selecting TopK predicted scores corresponding items as items recommended to a user performing a session, wherein TopK≥1.

[0021] Compared with the prior art, the present application has the following advantages:

[0022] (1) The application can effectively improve the recommendation accuracy compared with the prior art in the process of training the conversation recommendation model and obtaining the recommended items to the user, by the multi-interest extraction module performing bilinear mapping on the interaction embedding vectors containing interaction features output by the graph attention module through a bilinear mapping network, and performing gated weighted fusion on the mapped interest embedding vectors, so as to obtain multiple different interests of the user.

[0023] (2) The application can further improve the recommendation accuracy in the process of training the conversation recommendation model and obtaining the recommended items to the user, by the graph attention module performing graph attention convolution of each graph attention layer to obtain multiple interaction embedding vectors after noise suppression of the interaction features. BRIEF DESCRIPTION OF DRAWINGS

[0024] Figure 1 is the implementation flowchart of the application.

[0025] Figure 2 is the structure diagram of the conversation recommendation model constructed in the application. DETAILED DESCRIPTION

[0026] The application will be described in further detail below in combination with the drawings and specific embodiments.

[0027] Reference Figure 1 The present example includes the following steps.

[0028] Step 1) Obtain a training sample set and a test sample set:

[0029] Obtain M conversations from a real data set, and obtain the interaction information of each conversation S m and D user interested items to construct a self-connection matrix input matrix output matrix and input-output matrix representing four kinds of conversation graphs, then group the conversation graphs corresponding to more than half of the interaction information to form a training sample set, and group the remaining interaction information corresponding to the conversation graphs to form a test sample, wherein M≥500, each conversation S m and the interaction information of the dth user interested item is w m,d , representing the real number field; in the present example, the real data set is Diginetica, by mining the transfer pattern between the user's interest and the item in the conversation, the next item to be interacted by the user is predicted, wherein M=982,961, D=43,097;

[0030] Step 1a) The method for obtaining the interaction information of each conversation S m and D user interested items is:

[0031] For each session S m Duplicate items are removed from the list, resulting in D items that the user is interested in. The interaction information between the user and the items in each session is W. m , where w m,d For S m The interaction information with the d-th item; if the user has interacted with the d-th item in the session, then the interaction information w is used. m,d =1, otherwise w m,d =0, d'=d-1, d”=d+1;

[0032] Step 1b) Self-connection matrix Input matrix Output matrix and input-output matrix The construction method is as follows:

[0033] Step 1b1) Construct for each session S m Interaction information w with the d-th item m,d For array element, w m,d =1, a self-connected matrix of dimension 1×D with all other elements being 0.

[0034] Step 1b2) Construct for each session S m Interaction information w with the d'th item m,d' Interaction information w with the d-th item m,d For array element, w m,d' =w m,d =1, an input matrix of dimension 1×D with all other elements being 0.

[0035] Step 1b3) Construct S for each session m Interaction information w with the d-th item m,d Interaction information w with the "d"th item m,d” For array element, w m,d =w m,d” =1, an output matrix with dimensions 1×D and all other elements being 0.

[0036] Step 1b4) Construct S for each session m Interaction information w with the d'th item m,d' Interaction information w with the "d"th item m,d” For array element, w m,d' =w m,d” =1, an input-output matrix of dimension 1×D with all other elements being 0.

[0037] The self-connection matrix with size 1x43,097, a total of 982,961 are constructed; input matrix with size 1x43,097, a total of 982,961 are constructed; output matrix with size 1x43,097, a total of 982,961 are constructed; input-output matrix with size 1x43,097, a total of 982,961 are constructed;

[0038] Step 2) Construct a session recommendation model H based on graph attention and multi-interest extraction:

[0039] Referring to Figure 2 , the implementation of this step includes the following:

[0040] Construct a session recommendation model H including a cascaded graph attention module, a readout function module and a prediction module, and a multi-interest extraction module; the graph attention module includes a cascaded input layer and L graph attention layers; the multi-interest extraction module includes L interest extraction modules, and the input end of each interest extraction module is connected with each graph attention layer respectively, and the output end of the multi-interest extraction module is connected with the prediction module; the interest extraction module includes C bilinear mapping networks B arranged in parallel and a gating network and an interest aggregation layer cascaded therewith, and L interest aggregation layers are connected with each other, wherein L≥2, C≥2; in this example, L=3 and C=4;

[0041] Step 3) Define the total loss function of the session recommendation model H

[0042]

[0043]

[0044]

[0045] σ1=sigmoid

[0046] wherein, is the main loss function, sigmoid is a sigmoid nonlinear activation function, and α is a weight, is a self-supervised loss function, respectively, the interest anchor vector and the session vector of the Lth layer of the session S m The score, the predicted score, m,d respectively, of the dth interaction information w respectively, the interest anchor vector and the session vector of the Lth layer of the session S m

[0047] ​Through self-supervised learning, the interest anchor vector provides effective user interest information for the session vector output by the readout function module, thereby obtaining a higher quality session vector;

[0048] Step 4) Iteratively train the conversation recommendation model H:

[0049] The training sample set is used as input to the conversation recommendation model H for forward propagation to obtain the trained conversation recommendation model H. * ;

[0050] Step 4a) Initialize the number of iterations to t, the maximum number of iterations to T, T≥1000, and the current session recommendation model H t The trainable parameters are Θ t And let t = 1, in this example T = 2000;

[0051] Step 4b) The input layer in the graph attention module for each session S m Interactive information w m,d Vectorize to obtain the interactive embedding vector. The output of the l-th graph attention layer is sequentially convolved with the graph attention of the (l-1)-th graph attention layer on the four types of conversation graphs to obtain the interaction embedding vector containing interaction features in the l-th layer.

[0052] The formula for calculating the graph attention layer is as follows:

[0053]

[0054]

[0055] σ2=LeakyReLU

[0056] Where exp represents the exponential function, LeakyReLU is the LeakyReLU nonlinear activation function, ⊙ represents the element-wise product, and q a Let ( be the weight vector) Τ Indicates the transpose operation, r dd' For interactive information w m,d For interactive information w m,d' Attention weights;

[0057] To obtain higher quality interaction features, the graph attention convolution evaluates the importance of neighbor nodes through an attention mechanism on four types of conversation graphs, reduces noise in neighbor information, and thus improves the quality of the extracted interaction features.

[0058] Step 4c) Read out the interaction embedding vector containing interaction features from the Lth layer output by the function module to the graph attention module. Attention fusion is performed to obtain the session vector.

[0059] The aforementioned session vector The calculation formula is:

[0060]

[0061]

[0062] Among them, (W1) t Transform the session weights, (z d ) t For soft attention coefficients, in this example (W1) t The dimension is 100×1;

[0063] Step 4d) The C bilinear mapping networks B arranged in parallel in the l-th interest extraction module contain the interaction embedding vectors of interaction features. Perform a bilinear mapping to obtain C interest embedding vectors. The l-th gated network pair Gated weighted fusion is performed to obtain the layer interest embedding vector. The l-th interest aggregation layer Extract effective user interest information to obtain interest anchor vectors.

[0064] Step 4d1) The C bilinear mapping networks B arranged in parallel in the l-th interest extraction module contain the interaction embedding vectors of interaction features. Perform a bilinear mapping to obtain C interest embedding vectors.

[0065] The aforementioned interest embedding vector The calculation formula is:

[0066]

[0067]

[0068]

[0069] in, For the bilinear mapping network of the l-th layer and c-th interest in the t-th iteration, For the interaction information w in the l-th layer of the t-th iteration m,d The mapping attention weights for the c-th interest. For the interaction information w in the l-th layer of the t-th iteration m,d Normalized mapping of attention weights for the c-th interest;

[0070] The bilinear mapping network makes the model understand the importance of different interaction embedding vectors by mapping attention weight, so that the quality of interest embedding vector is higher and different interests of the user are modeled more finely;

[0071] Step 4d2) The l-th gating network performs gating weighted fusion on to obtain the layer interest embedding vector

[0072] The calculation formula of the layer interest embedding vector is as follows:

[0073]

[0074]

[0075] wherein, (W2) t is a layer interest transformation weight, is a gating weight of the c-th interest of the l-th layer in the t-th iteration, and in the present example, the dimension of (W2) t is 100x1;

[0076] The gating network makes the model select different user interests by calculating the gating weight, mines deeper user interest information, obtains a layer interest embedding vector with higher quality, and improves the expression ability of the model;

[0077] Step 4d3) The l-th interest aggregation layer performs effective user interest information extraction on to obtain the interest anchor point vector

[0078] The calculation formula of the interest anchor point vector is as follows:

[0079]

[0080]

[0081] wherein, (W3) t is an interest anchor point transformation weight, is an anchor point attention vector of the l-th layer in the t-th iteration, and in the present example, the dimension of (W3) t is 100x1;

[0082] The interest aggregation layer generates an anchor point attention vector by an attention mechanism to extract effective user interest information, removes noise in the user interest information, obtains an interest anchor point vector with higher quality, and can effectively improve the accuracy of item recommendation;

[0083] Step 4e) The prediction module outputs the session vector output by the readout function module The prediction score for the d-th interaction information is

[0084] The aforementioned predicted score The calculation formula is:

[0085]

[0086] Step 4f) uses the loss function of the conversation recommendation model H defined in step (3) and predicts the rating. Calculate the main loss function Using the interest anchor vector of the Lth layer and session vector Calculate the self-supervised loss function pass and Calculate the loss value

[0087] Step 4g) Through loss value Calculate H t parameter gradient Using gradient descent, through The weight parameters Θ of the conversation recommendation model t The session recommendation model H is updated to obtain the current iteration's model. t The weight parameter Θ t The update formula is:

[0088]

[0089] Where λ is the learning rate, and in this example, the value of λ is 0.001;

[0090] Step 4h) Determine if t≥T holds true. If so, obtain the trained conversation recommendation model H. * Otherwise, t = t + 1, H t =H, and execute step (4b);

[0091] Step 5) Obtain the conversation recommendation results:

[0092] The test sample set is used as the trained conversation recommendation model H. * The input is forward propagated to obtain the predicted score of each interaction information for the session vector, and the TopK items corresponding to the predicted scores are selected as the recommended items for the user who performed the session, where TopK≥1, and in this example, TopK=20;

[0093] The technical effects of the present invention will be further explained below with reference to simulation experiments:

[0094] 1. Simulation conditions and content:

[0095] The running environment of the simulation experiment is: Ubuntu 18.04.5 Server operating system, CPU is Intel(R) Xeon(R) CPU E5-2630 v4@2.20GHz, memory is 128GB, graphics card is Nvidia TITAN RTX, and virtual environment uses pytorch 1.9 and python 3.7.

[0096] The simulation comparison of the accuracy precision and the mean reciprocal rank MRR of the present application and the existing double-channel information fusion-based graph neural network session recommendation method is shown in Table 1.

[0097] Table 1

[0098]

[0099] 2. Simulation result analysis:

[0100] As can be seen from Table 1, the accuracy and the mean reciprocal rank of the present application on the Diginetica data set are improved in different ranges compared with the prior art, which proves that the present application has better performance in the session recommendation task; in the present application, the graph attention module obtains multiple interactive embedding vectors after noise suppression of interactive features through the graph attention layer, so that the accuracy and the mean reciprocal rank are improved; secondly, the multi-interest extraction module obtains multiple different interests of the user through the bilinear mapping network, the gating network and the interest aggregation layer, so that the accuracy and the mean reciprocal rank are further improved.

[0101] The above description is only one specific example of the present application and does not constitute any limitation on the present application. Obviously, for those skilled in the art, after understanding the content and principles of the present application, various modifications and changes in form and details can be made without departing from the principles and structures of the present application, but these modifications and changes based on the idea of the present application are still within the protection scope of the claims of the present application.

Claims

1. A conversation recommendation method based on graph attention and multi-interest extraction, characterized in that, The method comprises the following steps: (1) obtaining a training sample set and a test sample set: pass Each session in each session and A self-connected matrix constructed from interaction information of items of interest to each user. Input matrix Output matrix and input-output matrix Four types of conversation graphs are represented. The conversation graphs corresponding to more than half of the interaction information are used to form the training sample set, and the conversation graphs corresponding to the remaining interaction information are used to form the test sample set. Each session With the Interaction information for items that a user is interested in , Represents the real number field; is the interaction information of each session with the first item is an array element, and the rest of the array elements are 0, the dimension of the matrix is ​ Based on each session With the Interaction information of an item and the Interaction information of an item For the array element, The dimensions of the remaining array elements that are 0 are Matrix; Based on each session With the Interaction information of an item and the Interaction information of an item For the array element, The dimensions of the remaining array elements that are 0 are Matrix; Based on each session With the Interaction information of an item and the Interaction information of an item For the array element, The dimensions of the remaining array elements that are 0 are Matrix; (2) Construct a conversation recommendation model based on graph attention and multi-interest extraction : A session recommendation model is constructed, which comprises a cascaded graph attention module, a readout function module and a prediction module, and a multi-interest extraction module The graph attention module comprises a cascaded input layer and graph attention layers; the multi-interest extraction module comprises interest extraction modules, and the input end of each interest extraction module is connected with each graph attention layer respectively, and the output end of the multi-interest extraction module is connected with the prediction module; the interest extraction module comprises bilinear mapping networks arranged in parallel and a gating network and an interest aggregation layer connected therewith, and the interest aggregation layers are connected with each other, wherein , ; (3) defining a session recommendation model total loss function : ； ； ； ； wherein, is a main loss function, is a sigmoid nonlinear activation function, is a weight, is a self-supervised loss function, , are scores, predicted scores, respectively, for each session of the first interaction information , , are interest anchor vectors, session vectors, respectively, for the session of the first layer. (4) Conversation recommendation model Iterative training: Use the training sample set as the conversation recommendation model The input is forward propagated to obtain the trained conversation recommendation model. ; (5) obtaining a session recommendation result: Forward-propagating the test sample set as input of the trained session recommendation model to obtain a predicted score of a session vector to each interaction information, and selecting an item corresponding to the predicted score as an item recommended to a user performing the session, wherein .​​ 2. The method of claim 1, wherein, the session recommendation model in step (4) iterative training is performed, and the implementation steps are as follows: (3a) initialize the number of iterations as , the maximum number of iterations as , , the trainable parameters of the current session recommendation model as , and let ; (3b) the input layer in the graph attention module vectorizes the interaction information in each session to obtain an interaction embedding vector , the i-th graph attention layer The output of the i-th graph attention layer is sequentially subjected to graph attention convolution on the four session graphs to obtain an interaction embedding vector containing interaction features of the i-th layer ;​​ (3c) read out function module outputs the first interaction embedding vector of the layer containing the interaction feature perform attention fusion to obtain a session vector ; (3d) the parallel arrangement of the ​​​​​​​​​​​​ (3e) the prediction module predicts the session vector output by the readout function module The prediction score for the first ​​ (3f) the session recommendation model defined in step (3) loss function, and by predicting scores computing the main loss function , using the interest anchor vectors and the session vector of the first computing the self-supervised loss function , by and computing the loss value ; (3g) through loss value calculate parameter gradient Using gradient descent, through Weight parameters of the conversation recommendation model The session recommendation model is updated to obtain the current iteration's model. Among them, the weight parameters The update formula is: ； wherein, is the learning rate; (3h) judging whether it is true, if yes, obtaining the trained conversation recommendation model , otherwise , , and performing step (3b).

3. The method of claim 2, wherein, the interaction embedding vector described in step (3b) The formula for calculating is: ； ； ； where, denotes the exponential function, is the LeakyReLU nonlinear activation function, denotes the element-wise product, is the weight vector, denotes the transpose operation, is the mutual information is the attention weight on the mutual information .

4. The method of claim 2, wherein, the session vector described in step (3c) The calculation formula is: ； ； wherein, is a session transform weight, is a soft attention coefficient.

5. The method of claim 2, wherein, the interest embedding vector described in step (3d) The calculation formula is: ； ； ； wherein, is the th iteration of the th layer of the bilinear mapping network for the th interest, is the th iteration of the th layer of the interaction information mapping attention weight for the th interest, is the th iteration of the th layer of the interaction information normalized mapping attention weight for the th interest.

6. The method of claim 2, wherein, the layer interest embedding vector described in step (3d) The calculation formula is: ； ； wherein, is a layer interest transform weight, is a first iteration first layer first interest gating weight.

7. The method of claim 2, wherein, The interest anchor vector described in step (3d) The calculation formula is: ； ； wherein, interest anchor transformation weights, for the first iteration the anchor attention vector of the layer.

8. The method of claim 2, wherein, the predicted score described in step (3e) The formula for the calculation is: 。

Images