A sequential recommendation method based on graph neural network to mine users' long-term and short-term interests

Abstract

The present invention proposes a sequence recommendation method based on graph neural network mining of user long-term and short-term interests, obtains user personal information and user interaction sequence data sets, preprocesses the data sets and divides them into training sets and test sets; constructs a sequence recommendation model based on graph neural network mining of user long-term and short-term interests; trains the sequence recommendation model based on graph neural network mining of user long-term and short-term interests; inputs the personal information and interaction sequence of the user to be recommended into the trained sequence recommendation model based on graph neural network mining of user long-term and short-term interests, calculates the recommendation score of the item to be recommended relative to the user, and recommends the item to the user according to the recommendation score; the present invention solves the problem that there is no way to effectively capture the long-term and short-term interests of users and it is difficult to distinguish noise in the sequence recommendation scenario.

Description

Technical Field

[0001] The present invention belongs to the technical field of sequence recommendation, and in particular, relates to a sequence recommendation method based on graph neural network to mine users' long-term and short-term interests. Background Art

[0002] With the continuous growth of Internet services and mobile device usage, Internet users can easily access a large number of online products and services. Although this growth provides users with more available choices, it is also difficult for users to select their favorite products from a large number of candidate products. In order to reduce information overload and meet the diverse needs of users, personalized recommendation systems have emerged and are playing an increasingly important role in modern society. These systems can provide personalized experiences and serve users' personal needs. The specific benefits are: (1) helping users easily discover products that they are interested in; (2) creating opportunities for product providers to increase revenue.

[0003] In various Internet services, users visit items in chronological order, where the items a user interacts with are closely related to those he just visited. This property helps to accomplish an important recommendation task, sequential recommendation, which treats user behavior history as a sequence of interactions ordered by interaction timestamps.

[0004] The earliest work on sequential recommendation used Markov chains (MCs) to model first-order transition relationships between items, or used Markov decision processes (MDPs) to handle long-term effects. Later, factorized personalized MCs were proposed and extended to higher-order versions, inspired by and combined with methods based on general matrix factorization (MF). For example, FPMC (Factorizing Personalized Markov Chains) decomposes personalized Markov chains in order to capture long-term interests and short-term preferences. Fossil combines similarity-based models with high-order Markov chains. TransRec (Translation-basedRecommendation) proposed a translation-based sequential recommendation method. Recently, inspired by the advantages of sequence learning in natural language processing, researchers have proposed (deep) neural network-based methods to learn sequences. For example, Caser (Convolutional Sequence Embedding Recommendation) applies convolutional neural networks to process item embedding sequences. Methods based on recurrent neural networks (RNNs), especially those based on gated recurrent units (GRUs), have been used for sequence-based recommendation tasks. The self-attention mechanism has a good performance in sequence learning and has begun to be used for sequence recommendation. SASRec (Self-Attentive Sequential Recommendation) uses self-attention to adaptively consider the interactions between items.

[0005] Current problems with deep learning-based sequential recommendation methods include: (1) user behaviors in their rich historical sequences are often implicit and noisy preference signals that cannot fully reflect the user's actual preferences; (2) there are still deficiencies in modeling users' long-term interests and short-term preferences. Summary of the invention

[0006] In view of the shortcomings of the existing technology, the present invention uses technologies such as graph neural networks to mine users' long-term interests and short-term preferences, and proposes a sequence recommendation method based on graph neural networks to mine users' long-term and short-term interests.

[0007] The present invention is achieved through the following technical solutions:

[0008] A sequence recommendation method based on graph neural network to mine users' long-term and short-term interests, the method specifically comprising the following steps:

[0009] Step 1: Obtain user personal information and user interaction sequence datasets, preprocess the datasets and divide them into training sets and test sets;

[0010] Step 2: Build a sequential recommendation model based on graph neural network to mine users’ long-term and short-term interests;

[0011] Step 3: Train the sequence recommendation model based on graph neural network to mine users' long-term and short-term interests in step 2;

[0012] Step 4: Input the personal information and interaction sequence of the user to be recommended into the sequence recommendation model based on graph neural network to mine the user's long-term and short-term interests trained in step 3, calculate the recommendation score of the recommended item relative to the user, and recommend the item to the user based on the recommendation score.

[0013] Furthermore, in step 1,

[0014] The data set includes the user's click items and comment data;

[0015] The step 1 is specifically as follows:

[0016] Process the data in the dataset: First, remove users whose items of user interaction are less than 10, and remove items whose item occurrence times are less than 5;

[0017] Read user comments on items: delete items without comments, and reorder them according to the timestamps of interactions for each user;

[0018] Finally, the PKL file is generated: After the data is processed, the series of user interactions is represented as I represents the item that user U interacts with.

[0019] Furthermore, in step 2, the following steps are specifically included:

[0020] Step 2.1: Divide the dataset into long-term and short-term sequences of user interactions;

[0021] The five most recently interacted items are considered short-term sequences, and the remaining interacted items are considered long-term sequences;

[0022] Step 2.2: Obtain the user personal information embedding representation vector and the user interaction sequence embedding representation vector;

[0023] H u,l represents the long-term sequence of user u’s interactions, L u,l represents a short-term sequence of user u’s interactions, u i represents the embedding of user i, e i represents the embedding of item i;

[0024] Step 2.3: Build a compact item-item graph on the long-term sequence, use user embedding to distinguish noise, use graph convolutional neural network to update node information in the graph, use graph pooling technology and aggregation function to aggregate information in the graph, and obtain the user's long-term interest;

[0025] Step 2.4: Use a two-layer gating network to capture item features on the short-term sequence, and use an aggregation function to obtain the user's short-term preferences;

[0026] Step 2.5: Adopt a gating approach to adaptively combine the user's long-term interests and short-term preferences; the formula is as follows:

[0027]

[0028]

[0029] where g u,l is the learnable gating parameter, σ is the activation function, W g5 , W g6 , W g7 is the learnable weight parameter, u i Embed for users, For the long-term interests of users, It is the sequence representation after the fusion of long-term interests and short-term preferences. For users' short-term preferences;

[0030] Step 2.6: Use a bilinear function on the short-term sequence to capture the co-occurrence pattern between items and obtain the result vector

[0031] The specific formula is as follows:

[0032]

[0033] where e i is the item embedding, W r is a learnable weight parameter.

[0034] Furthermore, the step 2.3 is specifically as follows:

[0035] (1) By calculating the similarity between items, a compact item-item graph is constructed. The specific formula is as follows:

[0036] M i，j =cos(w⊙e i , j )

[0037]

[0038] Among them A i,jrepresents the adjacency matrix, M i,j represents the similarity score between item i and item j, cos represents the cosine calculation, It means the εnth 2 A large number, ε is used to control the sparsity of the graph, n represents the number of nodes in the graph, and ⊙ represents the Hadamard product;

[0039] (2) Using user embedding to distinguish noise, the graph convolutional neural network updates the node information in the graph; the formula is as follows:

[0040] e i =cos(e i ,u i )·W g3 ·e i

[0041]

[0042]

[0043]

[0044] In the formula, W g3 is a learnable weight parameter; σ is the activation function, I is the identity matrix, and the input of the 0th layer is the user's long-term interaction sequence H u，l , after going through l layers, a new node embedding is obtained;

[0045] (3) Graph pooling technology and aggregation function AUGRU are used to aggregate information in the graph to obtain the user's long-term interest. The formula is as follows:

[0046] S (l) =softmax(GNN l，pool (A (l) , H (l) ))

[0047]

[0048]

[0049]

[0050] Where S (l) represents the allocation matrix;

[0051] The graph is coarsened by graph pooling while maintaining its original structure; finally, long-term user representation is obtained by GRU plus attention.

[0052] Furthermore, in step 2.4:

[0053] The two layers of gating modules are: feature gating module and instance gating module; user-specific features captured by the feature gating module Will be passed to the instance gating module; in the instance gating module, important items reflecting the user's current short-term preferences It will be distinguished and used to predict the items that will be interacted with in the future. Finally, the aggregation function AUGRU is used to obtain the user's short-term interest. The specific formula is as follows:

[0054]

[0055]

[0056]

[0057] in the formula represents the final output of two layers of gating, It represents the user's short-term preference.

[0058] Furthermore, in step 3, the following steps are specifically included:

[0059] Step 3.1: Input the user personal information embedding representation vector and the user interaction sequence embedding representation vector obtained in step 2.2, the long-term and short-term interest representation vector obtained in step 2.5, and the result vector obtained in step 2.6 into the prediction module;

[0060] The specific formula is as follows:

[0061]

[0062] is the score of the item to be predicted. Indicates user embedding, It represents the sequence representation after the fusion of long-term interest and short-term preference, q j Indicates the target item;

[0063] Step 3.2: Update the model parameters through the Bayesian personalized ranking loss function, and train the optimal sequence recommendation model based on graph neural network to mine users' long-term and short-term interests;

[0064] In step 3.2, the loss function formula is as follows:

[0065]

[0066] represents the negative instance score, Θ represents the trainable parameter set, and λ is the regularization parameter, which represents the penalty intensity.

[0067] Furthermore, in step 4, the project recommendation scores are sorted, and the top k projects with the highest scores are recommended to the user.

[0068] An electronic device comprises a memory and a processor, wherein the memory stores a computer program, and the processor implements the steps of any one of the above methods when executing the computer program.

[0069] A computer-readable storage medium is used to store computer instructions, and when the computer instructions are executed by a processor, the steps of any of the above methods are implemented.

[0070] Beneficial effects of the present invention

[0071] The present invention provides a sequence recommendation method for mining users' long-term and short-term interests based on graph neural networks, which solves the problem that there is no way to effectively capture users' long-term and short-term interests and it is difficult to distinguish noise in sequence recommendation scenarios. BRIEF DESCRIPTION OF THE DRAWINGS

[0072] Figure 1 This is a flow chart of a sequence recommendation method for mining users' long-term and short-term interests based on graph neural networks in the present invention;

[0073] Figure 2 This is a framework diagram of a sequence recommendation model based on graph neural network to mine users' long-term and short-term interests;

[0074] Figure 3 A schematic diagram of a project-project diagram for constructing the present invention;

[0075] Figure 4 This is the long-term and short-term interest gating combination graph of the present invention. DETAILED DESCRIPTION

[0076] The technical solutions in the embodiments of the present invention will be described clearly and completely below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0077] Combination Figures 1 to 4 .

[0078] A sequence recommendation method based on graph neural network to mine users' long-term and short-term interests, the method specifically comprising the following steps:

[0079] Step 1: Obtain user personal information and user interaction sequence datasets, preprocess the datasets and divide them into training sets and test sets;

[0080] Step 2: Build a sequential recommendation model based on graph neural network to mine users’ long-term and short-term interests;

[0081] Step 3: Train the sequence recommendation model based on graph neural network to mine users' long-term and short-term interests in step 2;

[0082] Step 4: Input the personal information and interaction sequence of the user to be recommended into the sequence recommendation model based on graph neural network to mine the user's long-term and short-term interests trained in step 3, calculate the recommendation score of the recommended item relative to the user, and recommend the item to the user based on the recommendation score.

[0083] In step 1,

[0084] The data set includes data such as user click items and comments; three data sets are selected in the present invention: Amazon-CDs, Goodreads-Children, and MovieLens-20M; the Amazon-CDss data set is used as an example for introduction below;

[0085] The step 1 is specifically as follows:

[0086] Process the data in the dataset: First, remove users whose items of user interaction are less than 10, and remove items whose item occurrence times are less than 5;

[0087] Read user comments on items: delete items without comments, and reorder them according to the timestamps of interactions for each user;

[0088] Finally, the PKL file is generated: After the data is processed, the series of user interactions is represented as I represents the item that user U interacts with.

[0089] In step 2, the following steps are specifically included:

[0090] Step 2.1: Divide the dataset into long-term sequences of user interactions and short-term sequences of recent interactions;

[0091] The five most recently interacted items are considered short-term sequences, and the remaining interacted items are considered long-term sequences;

[0092] Step 2.2: Obtain the user personal information embedding representation vector and the user interaction sequence embedding representation vector;

[0093] H u,l represents the long-term sequence of user u’s interactions, L u,l represents a short-term sequence of user u’s interactions, u i represents the embedding of user i, e i represents the embedding of item i;

[0094] Step 2.3: Build a compact item-item graph on the long-term sequence, use user embedding to distinguish noise, use graph convolutional neural network to update node information in the graph, use graph pooling technology and aggregation function to aggregate information in the graph, and obtain the user's long-term interest;

[0095] The specific steps in step 2.3 are:

[0096] (1) By calculating the similarity between items, a compact item-item graph is constructed. The specific formula is as follows:

[0097] M i，j =cos(w⊙e i , j )

[0098]

[0099] Among them A i,j represents the adjacency matrix, M i,j represents the similarity score between item i and item j, cos represents the cosine calculation, It means the εnth 2 A large number, ε is used to control the sparsity of the graph, n represents the number of nodes in the graph, and ⊙ represents the Hadamard product;

[0100] (2) Using user embedding to distinguish noise, the graph convolutional neural network updates the node information in the graph; the formula is as follows:

[0101] e i =cos(e i ,u i )·W g3 ·e i

[0102]

[0103]

[0104]

[0105] In the formula, W g3 is a learnable weight parameter; σ is the activation function, I is the identity matrix, and the input of the 0th layer is the user's long-term interaction sequence H u，l , after going through l layers, a new node embedding is obtained;

[0106] The similarity calculation is performed by using user embedding and node embedding, so that the weight of noisy items in capturing the overall interests of users is reduced;

[0107] (3) Graph pooling technology and aggregation function AUGRU are used to aggregate information in the graph to obtain the user's long-term interest. The formula is as follows:

[0108] S (l) =softmax(GNN l，pool (A (l) , H (l) ))

[0109]

[0110]

[0111]

[0112] Where S (l) Represents the allocation matrix, which can be trained in model optimization;

[0113] The graph is coarsened by graph pooling while maintaining its original structure; finally, long-term user representation is obtained by GRU plus attention.

[0114] Step 2.4: Use a two-layer gating network to capture item features on the short-term sequence, and use an aggregation function to obtain the user's short-term preferences;

[0115] In step 2.4:

[0116] The two gating modules are: feature gating module and instance gating module. The feature gating module can adaptively capture the potential features that really attract users in the project based on user preferences, where the captured user-specific features Will be passed to the instance gating module; in the instance gating module, important items reflecting the user's current short-term preferences It will be distinguished and used to predict the items that will be interacted with in the future. Finally, the aggregation function AUGRU is used to obtain the user's short-term interest. The specific formula is as follows:

[0117]

[0118]

[0119]

[0120] in the formula represents the final output of two layers of gating, It represents the user's short-term preference.

[0121] Step 2.5: Adopt a gating approach to adaptively combine users’ long-term interests and short-term preferences;

[0122] In step 2.5, a gating method is then used to combine long-term interests and short-term interests to control the contribution of the two interests; the method used is similar to GRU; a learnable gating is used to control the contribution of the representation of long-term interests and the representation of short-term interests to the final prediction item; the formula is as follows:

[0123]

[0124]

[0125] where g u,l is the learnable gating parameter, σ is the activation function, W g5 , W g6 , W g7 is the learnable weight parameter, u i Embed for users, For the long-term interests of users, It is the sequence representation after the fusion of long-term interests and short-term preferences. For users' short-term preferences;

[0126] Step 2.6: Use a bilinear function on the short-term sequence to capture the co-occurrence pattern between items and obtain the result vector

[0127] The specific formula is as follows:

[0128]

[0129] where e i is the item embedding, W r is a learnable weight parameter.

[0130] In step 3, the following steps are specifically included:

[0131] Step 3.1: Input the user personal information embedding representation vector and the user interaction sequence embedding representation vector obtained in step 2.2, the long-term and short-term interest representation vector obtained in step 2.5, and the result vector obtained in step 2.6 into the prediction module;

[0132] The specific formula is as follows:

[0133]

[0134] is the score of the item to be predicted. Indicates user embedding, It represents the sequence representation after the fusion of long-term interest and short-term preference, q j Indicates the target item;

[0135] Step 3.2: Update the model parameters through the Bayesian personalized ranking loss function, and train the optimal sequence recommendation model based on graph neural network to mine users' long-term and short-term interests;

[0136] In step 3.2, the loss function formula is as follows:

[0137]

[0138] represents the negative instance score, Θ represents the trainable parameter set, and λ is the regularization parameter, which represents the penalty intensity.

[0139] In step 4, the project recommendation scores are sorted, and the top k projects with the highest scores are recommended to the user.

[0140] An electronic device comprises a memory and a processor, wherein the memory stores a computer program, and the processor implements the steps of any one of the above methods when executing the computer program.

[0141] A computer-readable storage medium is used to store computer instructions, and when the computer instructions are executed by a processor, the steps of any of the above methods are implemented.

[0142] The above is a detailed introduction to the sequence recommendation method for mining users' long-term and short-term interests based on graph neural networks proposed in the present invention, and the principles and implementation methods of the present invention are explained. The description of the above embodiments is only used to help understand the method of the present invention and its core idea; at the same time, for those skilled in the art, according to the ideas of the present invention, there will be changes in the specific implementation methods and application scopes. In summary, the content of this specification should not be understood as a limitation on the present invention.

Claims

1. A sequence recommendation method based on graph neural network to mine users' long-term and short-term interests, characterized by: The method specifically comprises the following steps: Step 1: Obtain user personal information and user interaction sequence datasets, preprocess the datasets and divide them into training sets and test sets; Step 2: Build a sequential recommendation model based on graph neural network to mine users’ long-term and short-term interests; Step 2.1: Divide the dataset into long-term and short-term sequences of user interactions; The five most recently interacted items are considered short-term sequences, and the remaining interacted items are considered long-term sequences; Step 2.2: Obtain the user personal information embedding representation vector and the user interaction sequence embedding representation vector; H u,l represents the long-term sequence of user u’s interactions, L u,l represents a short-term sequence of user u’s interactions, u i represents the embedding of user i, e i represents the embedding of item i; Step 2.3: Build a compact item-item graph on the long-term sequence, use user embedding to distinguish noise, use graph convolutional neural network to update node information in the graph, use graph pooling technology and aggregation function to aggregate information in the graph, and obtain the user's long-term interest; Step 2.4: Use a two-layer gating network to capture item features on the short-term sequence, and use an aggregation function to obtain the user's short-term preferences; Step 2.5: Adopt gating to adaptively combine users’ long-term interests and short-term preferences; The formula is as follows: where g u,l is the learnable gating parameter, σ is the activation function, W g5 , W g6 , W g7 is the learnable weight parameter, u i Embed for users, For the long-term interests of users, It is the sequence representation after the fusion of long-term interests and short-term preferences. For users' short-term preferences; Step 2.6: Use a bilinear function on the short-term sequence to capture the co-occurrence pattern between items and obtain the result vector The specific formula is as follows: where e i is the item embedding, W r is a learnable weight parameter; Step 3: Train the sequence recommendation model based on graph neural network to mine users' long-term and short-term interests in step 2; Step 4: Input the personal information and interaction sequence of the user to be recommended into the sequence recommendation model based on graph neural network to mine the user's long-term and short-term interests trained in step 3, calculate the recommendation score of the recommended item relative to the user, and recommend the item to the user based on the recommendation score.

2. The method according to claim 1, characterized in that: In step 1, The data set includes the user's click items and comment data; The step 1 is specifically as follows: Process the data in the dataset: First, remove users whose items of user interaction are less than 10, and remove items whose item occurrence times are less than 5; Read user comments on items: delete items without comments, and reorder them according to the timestamps of interactions for each user; Finally, the PKL file is generated: After the data is processed, the series of user interactions is represented as I represents the item that user U interacts with.

3. The method according to claim 2, characterized in that: The specific steps in step 2.3 are: (1) By calculating the similarity between items, a compact item-item graph is constructed. The specific formula is as follows: M i,j =cos(w⊙e i ,w⊙e j ) Among them A i,j represents the adjacency matrix, M i,j represents the similarity score between item i and item j, cos represents the cosine calculation, It means the εnth 2 A large number, ε is used to control the sparsity of the graph, n represents the number of nodes in the graph, and ⊙ represents the Hadamard product; (2) Using user embedding to distinguish noise, the graph convolutional neural network updates the node information in the graph; the formula is as follows: it is i =basket(s) i ,u i )·W g3 ·e i In the formula, W g3 is a learnable weight parameter; σ is the activation function, I is the identity matrix, and the input of the 0th layer is the user's long-term interaction sequence H u,l , after going through l layers, a new node embedding is obtained; (3) Graph pooling technology and aggregation function AUGRU are used to aggregate information in the graph to obtain the user's long-term interest. The formula is as follows: S (l) =softmax(GNN l,pool (A (l) ,H (l) )) Where S (l) represents the allocation matrix; The graph is coarsened by graph pooling while maintaining its original structure; finally, long-term user representation is obtained by GRU plus attention.

4. The method according to claim 3, characterized in that: In step 2.4: The two layers of gating modules are: feature gating module and instance gating module; user-specific features captured by the feature gating module Will be passed to the instance gating module; In the instance gating module, important items that reflect the user's current short-term preferences It will be distinguished and used to predict the items that will be interacted with in the future. Finally, the aggregation function AUGRU is used to obtain the user's short-term interest. The specific formula is as follows: in the formula represents the final output of two layers of gating, It represents the user's short-term preference.

5. The method according to claim 4, characterized in that: In step 3, the following steps are specifically included: Step 3.1: Input the user personal information embedding representation vector and the user interaction sequence embedding representation vector obtained in step 2.2, the long-term and short-term interest representation vector obtained in step 2.5, and the result vector obtained in step 2.6 into the prediction module; The specific formula is as follows: is the score of the item to be predicted. Indicates user embedding, It represents the sequence representation after the fusion of long-term interest and short-term preference, q j Indicates the target item; Step 3.2: Update the model parameters through the Bayesian personalized ranking loss function, and train the optimal sequence recommendation model based on graph neural network to mine users' long-term and short-term interests; In step 3.2, the loss function formula is as follows: represents the negative instance score, Θ represents the trainable parameter set, and λ is the regularization parameter, which represents the penalty intensity.

6. The method according to claim 5, characterized in that: In step 4, the project recommendation scores are sorted, and the top k projects with the highest scores are recommended to the user.

7. An electronic device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 6 are implemented.

8. A computer-readable storage medium for storing computer instructions, characterized in that: When the computer instructions are executed by a processor, the steps of the method according to any one of claims 1 to 6 are implemented.

Images