A next item recommendation method based on knowledge-enhanced attention graph neural network

Abstract

The present invention discloses a next item recommendation method based on a knowledge-enhanced attention graph neural network, comprising the following steps: representing each conversation in a conversation dataset as a sequence of conversation items, a sequence of target behaviors in the conversation, and a sequence of auxiliary behaviors in the conversation, and dividing the conversation dataset into a training set, a test set, and a validation set; using the training set to construct an initial model of a next item recommendation method based on a knowledge-enhanced attention graph neural network, NR-KAGNN; adjusting the parameters of the initial NR-KAGNN model using the training set and the validation set to obtain an optimized NR-KAGNN model, and evaluating the recommendation accuracy of the optimized model using the test set; and using the obtained optimized NR-KAGNN model to provide next item recommendations for users in the conversation dataset or actual conversation data. The next item recommendation method provided by the present invention has the characteristics of high recommendation accuracy and has broad application prospects in many fields such as music, movies, online news, and e-commerce.

Description

Technical Field

[0001] The present invention belongs to the technical field of conversation-based recommendation systems, and relates to a next item recommendation method based on deep neural networks, and in particular to a next item recommendation method based on knowledge-enhanced attention graph neural networks. Background Art

[0002] In the era of information overload and digital economy, recommender systems (RS) play an increasingly important role in consumption, services, and decision-making. Session-based recommender systems (SBRS), also known as conversational recommender systems, are a new paradigm of recommendation systems that has emerged in recent years (see: Shoujin Wang, Longbing Cao, Yan Wang, Quan Z. Sheng, Mehmet A. Organ, Defu Lian: A Survey on Session-based Recommender Systems. ACM Comput. Surv. 54(7): 154: 1-154: 38 (2022). https: / / doi.org / 10.1145 / 3465401). Conversational recommender systems mainly capture users' short-term, dynamic preferences based on their session information, so as to provide users with more timely and accurate recommendations. In a conversational recommender system, a session is a non-empty, bounded list of user-item interactions initiated by a user over a continuous period of time; user-item interactions are often referred to as interactions or behaviors. Given a user, each interaction can be represented as a two-tuple: (item, corresponding behavior type), where the behavior type can be a target behavior or an auxiliary behavior (see: Wen Wang, Wei Zhang, Shukai Liu, Qi Liu, Bo Zhang, Leyu Lin, Hongyuan Zha: Beyond Clicks: Modeling Multi-Relational Item Graph for Session-Based Target Behavior Prediction. WWW 2020: 3056-3062. https: / / doi.org / 10.1145 / 3366423.3380077). The interaction data involved in a recommendation system in a certain period of time usually includes users, items, user-item interactions and interaction timestamps; a certain amount of interaction data constitutes an interaction dataset.In a conversational recommendation system, the interaction dataset needs to be converted into a session dataset. The conversion method and steps are as follows: first, the interaction data in an interaction dataset is grouped by user, and then the interaction data of each user is processed as follows: the interaction data is sorted in ascending order of the interaction timestamp, and the sorted interaction data is divided into several sessions at a certain time interval to form the session data of the user; the session data of all users constitute a session dataset.

[0003] Next-item recommendation (or next interaction recommendation) is the primary task of conversational recommendation. Next-item recommendation aims to recommend the next possible user interaction (e.g., clicking a song, purchasing a product) in the current conversation based on a given conversational context (i.e., a list of past interactions). In practical application scenarios of conversational recommendation, a conversation may contain multiple types of user-item interactions (e.g., clicks, purchases). Such conversations are called multi-type-action sessions, also known as multiple types of behaviors sessions. The corresponding recommendation problem (task) is called multi-type behavior session recommendation or multi-behavior recommendation. In multi-behavior (session) recommendation scenarios, dependencies between items exist not only within interactions of the same type but also across different types of interactions. Furthermore, the dependencies between items vary across different types of interactions. For example, items clicked together in a conversation may be similar or competitive, while items purchased together in a conversation may be complementary. Therefore, conversational recommendation faces the challenge of how to effectively and accurately learn the dependencies between items under the same type of behavior and under different types of behavior.

[0004] The technical problem addressed by the present invention relates to multi-type behavioral conversational recommendation in next-item (interactive) recommendations. Multi-type behavioral conversational recommendation methods and technologies (sometimes collectively referred to as models) have broad application prospects in numerous fields, including music, movies, online news, and e-commerce. In these fields, users struggle to find items of interest (music, movies, news, and products, etc.) amidst vast amounts of information. Improving the accuracy of next-item (interactive) recommendations would help increase product sales and profits, improve user experience, and enhance user satisfaction.

[0005] A conversational recommendation method (method or approach) is also called a conversational recommendation model (model). Currently, most conversational recommendation methods / models are designed for single-type behavior conversations and therefore fail to leverage multi-type user behavior information to capture user preferences. Although a few conversational recommendation models are designed for multi-type behavior conversations, these models still have some shortcomings. For example, the Multi-Relational Graph Neural Network model for Session-based Target Behavior Prediction (MGNN-Spred) (see: Wen Wang, Wei Zhang, Shukai Liu, Qi Liu, Bo Zhang, Leyu Lin, Hongyuan Zha: Beyond Clicks: Modeling Multi-Relational Item Graph for Session-Based Target Behavior Prediction. WWW 2020: 3056-3062. https: / / doi.org / 10.1145 / 3366423.3380077) simply takes multi-type behavior sequences as input, ignoring the sequential dependencies between multi-type interactions of users in the original session data, and is unable to utilize the knowledge about items in the recommendation system (referred to as item knowledge) in external knowledge sources such as knowledge graphs (KG) or knowledge bases (KB) to capture dependencies within the session.Another example is the model that integrates user micro-behaviors and item knowledge into multi-task learning for session-based recommendation (MKM-SR) (see: Wenjing Meng, Deqing Yang, Yanghua Xiao: Incorporating User Micro-behaviors and Item Knowledge into Multi-task Learning for Session-based Recommendation. SIGIR 2020: 1091-1100. https: / / doi.org / 10.1145 / 3397271.3401098). It only encodes the behavior sequence independently, ignoring the contribution of the items corresponding to the behavior and failing to capture the high-order dependencies between entities encoded in the knowledge base. In short, these multi-type behavior session recommendation models generally fail to fully utilize the user's multi-type behavior information and knowledge from the knowledge graph or knowledge base to model dependencies within the session.

[0006] In order to overcome the above-mentioned deficiencies of the existing technology in the relevant technical field, the present invention intends to use a knowledge-enhanced attention graph neural network to construct a next-item recommendation model. Such a model can not only make full use of the user's multi-type behavior information and the item knowledge in the knowledge graph to enhance the multi-type behavior session embedding, but also improve the accuracy of the next-item recommendation.

[0007] The knowledge graph (KG) (see: Language and Knowledge Computing Committee of the Chinese Society for Information Processing. Knowledge Graph Development Report (2018), August 2018. http: / / cips-upload.bj.bcebos.com / KGDevReport2018.pdf) is one of the core technologies of the new generation of knowledge engineering. It is a heterogeneous graph composed of entities (as nodes in the graph) and relationships between entities (as edges in the graph). The knowledge graph contains a wealth of structured knowledge and semantic information. Therefore, it can serve as an auxiliary knowledge source to provide additional information or knowledge for items in (conversational) recommendation systems, thereby facilitating a more comprehensive and accurate capture of user preferences.

[0008] Deep neural networks (DNNs) have been widely used in fields such as speech recognition, computer vision, and recommender systems due to their powerful learning and representation capabilities, achieving tremendous success. Graph neural networks (GNNs) are a typical example of DNNs. GraphSAGE (see: William L. Hamilton, Zhitao Ying, Jure Leskovec: Inductive Representation Learning on Large Graphs. NIPS 2017: 1024-1034. http: / / papers.nips.cc / paper / 6703-inductive-representation-learning-on-large-graphs) is a typical GNN that uses a uniform sampling strategy to sample a fixed number of neighbors. It then uses an aggregation function to obtain a representation of the neighbor features. Finally, the aggregated neighbor features are concatenated with the features of the target node in the previous layer. The attention mechanism in deep neural networks (see: Dichao Hu. An Introductory Survey on Attention Mechanisms in NLP Problems. In: Proceedings of the 2019 Intelligent Systems Conference, IntelliSys 2019, pp. 432-448. https: / / doi.org / 10.1007 / 978-3-030-29513-4_31) is similar to the selective visual attention mechanism of humans. It can ignore unimportant information from a large amount of important information, filter out a small amount of important information, and focus on this important information. Depending on the specific needs of the technical problem to be solved, the attention mechanism can be organically combined with graph neural networks (such as GraphSAGE) to form an attentional graph neural network (Attentional Graph Neural Network). Furthermore, such a graph neural network can also be organically combined with knowledge graphs to form a knowledge-enhanced attentional graph neural network (Knowledge-Enhanced Attentional Graph Neural Network). The core idea of ​​the method of the present invention is to use knowledge-enhanced attention graph neural network to realize multi-type behavior conversation recommendation in next item (interaction) recommendation and improve recommendation accuracy.

[0009] The following is a brief description of several prior arts adopted in the technical solution of the present invention. These prior arts are merely several individual technical features in the technical solution of the present invention. They constitute the technical solution of the present invention only after being organically combined with other technical features of the present invention in a non-obvious manner.

[0010] Knowledge Graph Attention Network (KGAT) (see: Xiang Wang, Xiangnan He, Yixin Cao, Meng Liu, Tat-Seng Chua: KGAT: Knowledge Graph Attention Network for Recommendation. KDD 2019: 950-958. https: / / doi.org / 10.1145 / 3292500.3330989) combines the user-item graph with the knowledge graph to form a collaborative knowledge graph and uses the Graph Attention Network (GAT) (see: Petar Velickovic, Guillem Cucurull, Arantxa Casanova, Adriana Romero, Pietro Liò, Yoshua Bengio: Graph Attention Networks. In: Proceedings of 6th International Conference on Learning Representations, ICLR (Poster) 2018. https: / / openreview.net / pdf?id=rJXMpikCZ) to learn representations of users and items and use them for recommendations.

[0011] The Adam optimizer (see: Diederik P. Kingma, Jimmy Ba: Adam: A Method for Stochastic Optimization. 3rd International Conference on Learning Representations, ICLR 2015, May 7-9, 2015, Conference Track Proceedings. https: / / arxiv.org / abs / 1412.6980) is used to optimize the parameters of the next recommendation model.

[0012] Average pooling is used to average the item features in the target behavior sequence graph or auxiliary behavior sequence graph of the session to obtain the representative item features of the graph.

[0013] Softmax is an activation function that is used to map the next item probability (initial value) of all items in the conversation dataset to an open interval (0, 1).

[0014] The present invention evaluates the accuracy of a recommendation model using three commonly used performance evaluation metrics in the art: hit rate (HR@N), mean reciprocal ranking (MRR@N), and normalized discounted cumulative gain (NDCG@N) (see: Charu C. Aggarwal. Evaluating Recommender Systems. In: Rcommender Systems: The Textbook. Springer 2016, pp. 225-254. https: / / doi.org / 10.1007 / 978-3-319-29659-3_7), which are defined as follows:

[0015] (1) Hit rate: It is defined as the proportion of items in the recommendation list that are truly relevant to the user. This indicator is defined by the following formula:

[0016]

[0017] Among them, N represents the number of recommended items, Number of Hits@N represents the number of hit items in the recommendation list, and GT represents the set of all target items in the test set. The higher this indicator is, the higher the recommendation accuracy is.

[0018] (2) Average reciprocal ranking: It is defined as the reciprocal ranking of the items that are truly relevant to the user in the recommendation list, and then the average of the reciprocal rankings. This indicator is defined by the following formula:

[0019]

[0020] Among them, N represents the number of recommended items, m represents the number of users, and GT u Represents the set of all items that are truly relevant to the user, rank v represents the ranking of item v in the recommendation list of user u.

[0021] (3) Normalized cumulative loss gain: defined as the ratio of the cumulative loss gain to its ideal value. This indicator is defined by the following formula:

[0022]

[0023] Where N is the number of recommended items, IDCG@N is the maximum discount-cumulative gain under ideal conditions, and DCG@N is the discount-cumulative gain, which is defined by the following formula:

[0024]

[0025] Among them, m represents the number of users, GT u Represents the set of all items that are truly relevant to the user, rank v represents the ranking of item v in the recommendation list of user u.

[0026] In summary, in order to overcome the above-mentioned shortcomings in the existing technology, namely: the conversation recommendation method / model based on deep neural network does not fully utilize multi-type behavior information and item knowledge from the knowledge graph or knowledge base to effectively model the dependencies within the conversation, resulting in low recommendation accuracy, the present invention creatively combines the knowledge graph, attention mechanism and graph neural network to form a novel knowledge-enhanced attention graph neural network. This network can fully utilize multi-type behavior information and item knowledge in the knowledge graph to enhance conversation embedding, thereby improving the accuracy of the next item recommendation. In short, the present invention provides a next-item recommendation method based on knowledge-enhanced attention graph neural network (NR-KAGNN for short). Summary of the Invention

[0027] The technical problem to be solved by the present invention is to provide a next item recommendation method with high recommendation accuracy.

[0028] To solve the above technical problems, the present invention is achieved through the following technical solutions:

[0029] The present invention provides a next item recommendation method based on knowledge-enhanced attention graph neural network, comprising the following steps:

[0030] Step S1: Represent each session in a session dataset as a sequence of items in the session, a sequence of target behaviors in the session, and a sequence of auxiliary behaviors in the session, and divide the session dataset into a training set, a test set, and a validation set;

[0031] Step S2: using the training set to construct an initial model of the next item recommendation method NR-KAGNN based on knowledge-enhanced attention graph neural network;

[0032] Step S3: Parameter adjustment is performed on the initial model of the NR-KAGNN using the training set and the validation set to obtain an optimized model of the NR-KAGNN, and the recommendation accuracy of the optimized model is evaluated using the test set;

[0033] Step S4: Using the obtained NR-KAGNN optimization model, provide the next recommendation for the user in the conversation dataset or the actual conversation data.

[0034] In this method, step S2 further comprises:

[0035] Step S21: constructing a knowledge-enhanced multi-type behavior conversation graph generation layer in the initial model of the next recommendation method NR-KAGNN, and using the layer to obtain a knowledge-enhanced multi-type behavior conversation graph;

[0036] Step S22: constructing an attention aggregation layer on the knowledge-enhanced multi-type behavior conversation graph in the initial model of the next recommendation method NR-KAGNN, and using the layer to obtain a target behavior sequence graph and an auxiliary behavior sequence graph;

[0037] Step S23: constructing an embedding generation layer for multi-type behavior sessions in the initial model of the next recommendation method NR-KAGNN, and using the layer to generate an embedding for the multi-type behavior session;

[0038] Step S24: Construct a next item probability prediction layer in the initial model of the next item recommendation method NR-KAGNN, and use this layer to generate a next item probability vector for all items in the conversation dataset, where each component represents the probability that the corresponding item will become the user's next interaction item.

[0039] In this method, step S21 further includes:

[0040] Step S211: Represent each item in the conversation dataset as a d-dimensional vector, where the hyperparameter d∈{64, 128, 256}, thereby obtaining the initial embedding of all items in the conversation dataset;

[0041] Step S212: construct a multi-type behavior conversation graph using the conversation item sequence, the conversation target behavior sequence, and the conversation auxiliary behavior sequence. The multi-type behavior conversation graph is defined as follows:

[0042] Each node in the session graph represents an item in the item sequence of the session. Each directed edge in the session graph represents a relationship between a pair of items. The relationship is represented as a triple: (head item, tail item, edge type). There are three types of edge types: target behavior, auxiliary behavior, and item sequence. A target behavior edge indicates that in the target behavior sequence of the session, the user interacts with a head item and then interacts with a tail item. An auxiliary behavior edge indicates that in the auxiliary behavior sequence of the session, the user interacts with a head item and then interacts with a tail item. An item sequence edge indicates that in the item sequence of the session, the user interacts with a head item and then interacts with a tail item.

[0043] Step S213: aligning all items in the item sequence in the session with entities in a knowledge graph to obtain an item entity set, and forming a multi-hop neighborhood of each entity in the item entity set in the knowledge graph;

[0044] Step S214: Generate initial embeddings for each entity and each relationship in the knowledge graph using the knowledge graph embedding model TransR, and use the knowledge graph attention network to perform multi-hop attention embedding propagation to generate embeddings for all entities in the project entity set. The multi-hop attention embedding propagation specifically includes:

[0045] The four steps of information propagation, information aggregation, high-order propagation, and full connection are performed in sequence to generate the embeddings of all entities in the project entity set, as follows:

[0046] The information propagation step calculates the relationship coefficient between each entity in the project entity set and each direct neighbor of the entity in the knowledge graph through a relational attention mechanism, and obtains the neighbor information of the entity after performing weighted aggregation on all direct neighbors of the entity based on these relationship coefficients;

[0047] The information aggregation step uses a bidirectional interaction aggregator to aggregate the initial embedding of each entity and the neighbor information of the entity to form the first-hop embedding of the entity;

[0048] The high-order propagation step collects information from its higher-hop neighbors for each entity by performing the above information propagation step and information aggregation step multiple times in sequence. Furthermore, the H-th hop embedding of an entity is generated by aggregating the entity's H-1-th hop embedding and the entity's H-1-th hop neighbor information, where the hyperparameter H∈{1, 2, 3, 4, 5};

[0049] The fully connected step concatenates the 1st, 2nd, ..., Hth hop embeddings of each entity after multi-hop attention embedding propagation into a vector, and then performs a linear transformation on the vector to obtain the embedding of the entity;

[0050] Step S215: Using a fusion function, the initial embeddings of the items in the multi-type behavior conversation graph are fused with the embeddings of the entities in the item entity set to obtain a knowledge-enhanced multi-type behavior conversation graph. The fusion function is defined by the following formula:

[0051]

[0052] Where || is the vector concatenation operation, is the embedding of an entity in the project entity set, v i is the initial embedding of an item in the multi-type behavior session graph, i∈{1, 2, ..., n}, n is the total number of nodes in the multi-type behavior session graph, is a weight matrix, the hyperparameter d∈{64, 128, 256} is the vector dimension of the initial embedding of the item, and the calculation result x i is the embedding of an item in the knowledge-enhanced multi-type behavior conversation graph.

[0053] In this method, step S22 further includes:

[0054] Step S221: The attention aggregation layer on the knowledge-enhanced multi-type behavior conversation graph directly uses a K-layer attention graph neural network to perform multiple rounds of updates on the embeddings of all items in the knowledge-enhanced multi-type behavior conversation graph to form an aggregated knowledge-enhanced multi-type behavior conversation graph, where the hyperparameter K∈{0, 1, 2, 3, 4}, and decomposes the multi-type behavior conversation graph into a target behavior sequence graph and an auxiliary behavior sequence graph, as follows:

[0055] First, based on the type and direction of the directed edges between any item and its direct neighbors in the knowledge-enhanced multi-type behavior conversation graph, the K-layer attention graph neural network divides the item's direct neighbors into six types: target behavior forward neighbors, target behavior backward neighbors, auxiliary behavior forward neighbors, auxiliary behavior backward neighbors, item sequence forward neighbors, and item sequence backward neighbors. Thus, all neighbors of the same type for each item in the knowledge-enhanced multi-type behavior conversation graph constitute the item's neighbor set of that type, forming six types of neighbor sets for the item: target behavior forward neighbor set, target behavior backward neighbor set, auxiliary behavior forward neighbor set, auxiliary behavior backward neighbor set, item sequence forward neighbor set, and item sequence backward neighbor set.

[0056] Then, the layer number k of the K-layer attention graph neural network is used as the loop variable of the counting control loop, with its initial value being 1, its termination value being K, and the incremental step being 1, and the following steps (1) to (3) are repeated K times:

[0057] (1) The six types of neighbor sets of each item v in the knowledge-enhanced multi-type behavior conversation graph are average pooled to obtain the k-th layer embedding of the six types of neighbor sets of item v, which are the embeddings of the target forward neighbor set respectively. Embedding of target backward neighbor set Embedding of auxiliary forward neighbor sets Embedding of auxiliary backward neighbor sets Embedding of forward neighbor sets of item sequences Backward Neighborhood Embedding of Item Sequences

[0058] (2) Use the attention mechanism to aggregate the k-th layer embeddings of the six types of neighbor sets of item v, thereby obtaining the k-th layer embeddings of the direct neighbors of item v The calculation process is defined by the following formula:

[0059]

[0060]

[0061]

[0062]

[0063] Where, and They are all weight parameters, and their values ​​are determined by parameter learning. The hyperparameter d∈{64, 128, 256} is the vector dimension of the initial embedding of the project, and tanh is an activation function. is the matrix transpose operation;

[0064] (3) Embed the k-1th layer of item v and the k-th layer embedding of the direct neighbors of item v After adding, we get the k-th layer embedding of item v Among them, when k=1, the k-1=0th layer embedding of item v is the embedding of item v calculated in step S215;

[0065] Then, after K repetitions, the embedding of item v is updated to the K-th layer embedding Then, the embeddings of all items 1, 2, ..., n in the knowledge-enhanced multi-type behavior conversation graph are updated to the K-th layer embedding Each embedding The hyperparameter d∈{64, 128, 256} is the vector dimension of the initial embedding of the items, thus forming an aggregated knowledge-enhanced multi-type behavior conversation graph;

[0066] Finally, the K-layer attention graph neural network decomposes the aggregated knowledge-enhanced multi-type behavior conversation graph into a target behavior sequence graph and an auxiliary behavior sequence graph according to the target behavior sequence of the conversation and the auxiliary behavior sequence of the conversation.

[0067] In this method, step S23 further includes:

[0068] Step S231: Perform average pooling on the target behavior sequence diagram and the auxiliary behavior sequence diagram respectively to obtain the embedding of the target behavior sequence and the embedding of the auxiliary behavior sequence, and use a gated fusion mechanism to calculate the relative importance weights of the embeddings of the two sequences. Finally, perform weighted summation on the embeddings of the two sequences according to the relative importance weights to generate an embedding for a multi-type behavior session.

[0069] In this method, step S24 further includes:

[0070] Step S241: performing a vector dot product between the embedding of the multi-type behavior session and the initial embedding of all items in the session dataset to obtain a score vector;

[0071] Step S242: Use a softmax function to normalize the score vector and generate a next item probability vector for all items in the conversation dataset, where each component represents the probability that the corresponding item is the user's next interaction item.

[0072] In this method, step S3 further comprises:

[0073] Step S31: Using the Adam optimizer to learn the parameters of the initial model or the tuned model of the NR-KAGNN on the training set, and calculating the recommendation accuracy of the model on the validation set, specifically including:

[0074] First, the grid search method is used to set the values ​​of the following five hyperparameters for the model: the vector dimension d of the initial embedding of the item, the number of hops H of the multi-hop attention embedding propagation, the number of layers K of the attention graph neural network, the learning rate η, and the L2 regularization coefficient λ. The search range of these hyperparameters, i.e., the domain, is as follows: the domain of d is the set {64, 128, 256}, the domain of H is the set {1, 2, 3, 4, 5}, the domain of K is the set {0, 1, 2, 3, 4}, the domain of η is the set {0.001, 0.01, 0.1}, and the domain of λ is the set {10 -7 , 10 -6 , 10 -5 , 10 -4 , 10 -3};

[0075] Then, the Adam optimizer is used on the training set to learn the parameter values ​​of the model to obtain the parameter-adjusted model of NR-KAGNN;

[0076] Finally, the recommendation accuracy of the model is calculated on the validation set;

[0077] Step S32: Determine whether the recommendation accuracy of the NR-KAGNN model after parameter adjustment reaches a predetermined threshold, where the threshold value range is a closed interval [0.6, 0.7]. If the recommendation accuracy of the model has not reached the predetermined threshold, continue to adjust the parameters by repeating steps S31 and S32 in sequence. Otherwise, terminate the parameter adjustment process to obtain the optimized model of NR-KAGNN.

[0078] Step S33: Use the test set and three performance evaluation indicators, namely, hit rate, average reciprocal rank, and normalized discounted cumulative gain, to evaluate the recommendation accuracy of the optimized model of the NR-KAGNN.

[0079] The beneficial technical effects of the present invention mainly include three aspects: (1) The present invention provides a next item recommendation method with high recommendation accuracy, which has scientific significance for the development of next item recommendation technology; (2) The next item recommendation method NR-KAGNN of the present invention has higher recommendation accuracy than the typical multi-type behavior conversation recommendation models MGNN-Spred and MKM-SR. The reason is that the technical solution of the present invention fully utilizes the user's multi-type behavior information and the item knowledge in the knowledge graph by organically combining means such as the knowledge graph attention network and the attention graph neural network to enhance the embedding of multi-type behavior conversations; (3) The present invention has broad application prospects in many fields such as music, movies, online news, and e-commerce.

[0080] The following further describes the specific embodiments of the present invention in conjunction with the accompanying drawings. Additional aspects and advantages of the present invention will be partially given in the following description, which will become obvious from the following description or be learned through practice of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0081] Figure 1 A schematic flow chart of a next item recommendation method based on a knowledge-enhanced attention graph neural network according to the technical solution of the present invention is shown;

[0082] Figure 2 A model framework diagram of a next item recommendation method based on a knowledge-enhanced attention graph neural network according to the technical solution of the present invention is shown;

[0083] Figure 3 A flowchart for constructing an initial model in a next-item recommendation method based on a knowledge-enhanced attention graph neural network according to the technical solution of the present invention is shown. DETAILED DESCRIPTION

[0084] The embodiments of the present invention are described in detail below, and examples of the embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar concepts, objects, elements, etc., or concepts, objects, elements, etc. having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain the present invention, and are not to be construed as limiting the present invention.

[0085] It will be understood by those skilled in the art that, unless otherwise defined, all terms (including technical and scientific terms) used herein have the same meaning as commonly understood by those skilled in the art in the art and related fields to which the present invention belongs. It should also be understood that terms such as those defined in common dictionaries should be understood to have meanings consistent with their meanings in the context of the prior art and, unless defined as such herein, will not be interpreted in an idealized or overly formal sense.

[0086] In order to solve the above technical problems, the present invention is achieved through the following technical solutions:

[0087] The present invention provides a next item recommendation method based on knowledge-enhanced attention graph neural network, such as Figure 1 As shown, the following steps S1 to S4 are included:

[0088] Step S1: Represent each session in a session dataset as a sequence of items in the session, a sequence of target behaviors in the session, and a sequence of auxiliary behaviors in the session, and divide the session dataset into a training set, a test set, and a validation set.

[0089] Step S2: Using the training set, construct an initial model of the next recommendation method NR-KAGNN based on knowledge-enhanced attention graph neural network; Figure 2 and Figure 3 As shown, the step S2 further includes:

[0090] Step S21: constructing a knowledge-enhanced multi-type behavior conversation graph generation layer in the initial model of the next recommendation method NR-KAGNN, and using the layer to obtain a knowledge-enhanced multi-type behavior conversation graph; Step S21 further includes:

[0091] Step S211: Represent each item in the conversation dataset as a d-dimensional vector, where the hyperparameter d∈{64, 128, 256}, thereby obtaining the initial embedding of all items in the conversation dataset;

[0092] Step S212: construct a multi-type behavior conversation graph using the conversation item sequence, the conversation target behavior sequence, and the conversation auxiliary behavior sequence. The multi-type behavior conversation graph is defined as follows:

[0093] Each node in the session graph represents an item in the item sequence of the session. Each directed edge in the session graph represents a relationship between a pair of items. The relationship is represented as a triple: (head item, tail item, edge type). There are three types of edge types: target behavior, auxiliary behavior, and item sequence. A target behavior edge indicates that in the target behavior sequence of the session, the user interacts with a head item and then interacts with a tail item. An auxiliary behavior edge indicates that in the auxiliary behavior sequence of the session, the user interacts with a head item and then interacts with a tail item. An item sequence edge indicates that in the item sequence of the session, the user interacts with a head item and then interacts with a tail item.

[0094] Step S213: aligning all items in the item sequence in the session with entities in a knowledge graph to obtain an item entity set, and forming a multi-hop neighborhood of each entity in the item entity set in the knowledge graph;

[0095] Step S214: Generate initial embeddings for each entity and each relationship in the knowledge graph using the knowledge graph embedding model TransR, and use the knowledge graph attention network to perform multi-hop attention embedding propagation to generate embeddings for all entities in the project entity set. The multi-hop attention embedding propagation specifically includes:

[0096] The four steps of information propagation, information aggregation, high-order propagation, and full connection are performed in sequence to generate the embeddings of all entities in the project entity set, as follows:

[0097] The information propagation step calculates the relationship coefficient between each entity in the project entity set and each direct neighbor of the entity in the knowledge graph through a relational attention mechanism, and obtains the neighbor information of the entity after performing weighted aggregation on all direct neighbors of the entity based on these relationship coefficients;

[0098] The information aggregation step uses a bidirectional interaction aggregator to aggregate the initial embedding of each entity and the neighbor information of the entity to form the first-hop embedding of the entity;

[0099] The high-order propagation step collects information from its higher-hop neighbors for each entity by performing the above information propagation step and information aggregation step multiple times in sequence. Furthermore, the H-th hop embedding of an entity is generated by aggregating the entity's H-1-th hop embedding and the entity's H-1-th hop neighbor information, where the hyperparameter H∈{1, 2, 3, 4, 5};

[0100] The fully connected step concatenates the 1st, 2nd, ..., Hth hop embeddings of each entity after multi-hop attention embedding propagation into a vector, and then performs a linear transformation on the vector to obtain the embedding of the entity;

[0101] Step S215: Using a fusion function, the initial embeddings of the items in the multi-type behavior conversation graph are fused with the embeddings of the entities in the item entity set to obtain a knowledge-enhanced multi-type behavior conversation graph. The fusion function is defined by the following formula:

[0102]

[0103] Where || is the vector concatenation operation, is the embedding of an entity in the project entity set, v i is the initial embedding of an item in the multi-type behavior session graph, i∈{1, 2, ..., n}, n is the total number of nodes in the multi-type behavior session graph, is a weight matrix, the hyperparameter d∈{64, 128, 256} is the vector dimension of the initial embedding of the item, and the calculation result x i is the embedding of an item in the knowledge-enhanced multi-type behavior conversation graph;

[0104] Step S22: constructing an attention aggregation layer on the knowledge-enhanced multi-type behavior conversation graph in the initial model of the next recommendation method NR-KAGNN, and using the layer to obtain a target behavior sequence diagram and an auxiliary behavior sequence diagram; Step S22 further includes:

[0105] Step S221: The attention aggregation layer on the knowledge-enhanced multi-type behavior conversation graph directly uses a K-layer attention graph neural network to perform multiple rounds of updates on the embeddings of all items in the knowledge-enhanced multi-type behavior conversation graph to form an aggregated knowledge-enhanced multi-type behavior conversation graph, where the hyperparameter K∈{0, 1, 2, 3, 4}, and decomposes the multi-type behavior conversation graph into a target behavior sequence graph and an auxiliary behavior sequence graph, as follows:

[0106] First, based on the type and direction of the directed edges between any item and its direct neighbors in the knowledge-enhanced multi-type behavior conversation graph, the K-layer attention graph neural network divides the item's direct neighbors into six types: target behavior forward neighbors, target behavior backward neighbors, auxiliary behavior forward neighbors, auxiliary behavior backward neighbors, item sequence forward neighbors, and item sequence backward neighbors. Thus, all neighbors of the same type for each item in the knowledge-enhanced multi-type behavior conversation graph constitute the item's neighbor set of that type, forming six types of neighbor sets for the item: target behavior forward neighbor set, target behavior backward neighbor set, auxiliary behavior forward neighbor set, auxiliary behavior backward neighbor set, item sequence forward neighbor set, and item sequence backward neighbor set.

[0107] Then, the layer number k of the K-layer attention graph neural network is used as the loop variable of the counting control loop, with its initial value being 1, its termination value being K, and the incremental step being 1, and the following steps (1) to (3) are repeated K times:

[0108] (1) The six types of neighbor sets of each item v in the knowledge-enhanced multi-type behavior conversation graph are average pooled to obtain the k-th layer embedding of the six types of neighbor sets of item v, which are the embeddings of the target forward neighbor set respectively. Embedding of target backward neighbor set Embedding of auxiliary forward neighbor sets Embedding of auxiliary backward neighbor sets Embedding of forward neighbor sets of item sequences Backward Neighborhood Embedding of Item Sequences

[0109] (2) Use the attention mechanism to aggregate the k-th layer embeddings of the six types of neighbor sets of item v, thereby obtaining the k-th layer embeddings of the direct neighbors of item v The calculation process is defined by the following formula:

[0110]

[0111]

[0112]

[0113]

[0114] Where, and They are all weight parameters, and their values ​​are determined by parameter learning. The hyperparameter d∈{64, 128, 256} is the vector dimension of the initial embedding of the project, and tanh is an activation function. is the matrix transpose operation;

[0115] (3) Embed the k-1th layer of item v and the k-th layer embedding of the direct neighbors of item v After adding, we get the k-th layer embedding of item v Among them, when k=1, the k-1=0th layer embedding of item v is the embedding of item v calculated in step S215;

[0116] Then, after K repetitions, the embedding of item v is updated to the K-th layer embedding Then, the embeddings of all items 1, 2, ..., n in the knowledge-enhanced multi-type behavior conversation graph are updated to the K-th layer embedding Each embedding The hyperparameter d∈{64, 128, 256} is the vector dimension of the initial embedding of the items, thus forming an aggregated knowledge-enhanced multi-type behavior conversation graph;

[0117] Finally, the K-layer attention graph neural network decomposes the aggregated knowledge-enhanced multi-type behavior conversation graph into a target behavior sequence graph and an auxiliary behavior sequence graph according to the target behavior sequence and the auxiliary behavior sequence of the conversation;

[0118] Step S23: constructing an embedding generation layer for multi-type behavior sessions in the initial model of the next recommendation method NR-KAGNN, and using the layer to generate an embedding for the multi-type behavior session; step S23 further includes:

[0119] Step S231: performing average pooling on the target behavior sequence graph and the auxiliary behavior sequence graph respectively to obtain the embedding of the target behavior sequence and the embedding of the auxiliary behavior sequence, and using a gated fusion mechanism to calculate the relative importance weights of the embeddings of the two sequences. Finally, performing a weighted summation of the embeddings of the two sequences according to the relative importance weights to generate an embedding of a multi-type behavior session;

[0120] Step S24: constructing a next item probability prediction layer in the initial model of the next item recommendation method NR-KAGNN, and using the layer to generate a next item probability vector for all items in the conversation dataset, wherein each component represents the probability that the corresponding item will become the next interactive item of the user; Step S24 further includes:

[0121] Step S241: performing a vector dot product between the embedding of the multi-type behavior session and the initial embedding of all items in the session dataset to obtain a score vector;

[0122] Step S242: Use a softmax function to normalize the score vector and generate a next item probability vector for all items in the conversation dataset, where each component represents the probability that the corresponding item is the user's next interaction item.

[0123] Step S3: Parameter adjustment is performed on the initial model of the NR-KAGNN using the training set and the validation set to obtain an optimized model of the NR-KAGNN, and the recommendation accuracy of the optimized model is evaluated using the test set; Step S3 further includes:

[0124] Step S31: Using the Adam optimizer to learn the parameters of the initial model or the tuned model of the NR-KAGNN on the training set, and calculating the recommendation accuracy of the model on the validation set, specifically including:

[0125] First, the grid search method is used to set the values ​​of the following five hyperparameters for the model: the vector dimension d of the initial embedding of the item, the number of hops H of the multi-hop attention embedding propagation, the number of layers K of the attention graph neural network, the learning rate η, and the L2 regularization coefficient λ. The search range of these hyperparameters, i.e., the domain, is as follows: the domain of d is the set {64, 128, 256}, the domain of H is the set {1, 2, 3, 4, 5}, the domain of K is the set {0, 1, 2, 3, 4}, the domain of η is the set {0.001, 0.01, 0.1}, and the domain of λ is the set {10 -7 , 10 -6 , 10 -5 , 10 -4 , 10 -3};

[0126] Then, the Adam optimizer is used on the training set to learn the parameter values ​​of the model to obtain the parameter-adjusted model of NR-KAGNN;

[0127] Finally, the recommendation accuracy of the model is calculated on the validation set;

[0128] Step S32: Determine whether the recommendation accuracy of the NR-KAGNN model after parameter adjustment reaches a predetermined threshold, where the threshold value range is a closed interval [0.6, 0.7]. If the recommendation accuracy of the model has not reached the predetermined threshold, continue to adjust the parameters by repeating steps S31 and S32 in sequence. Otherwise, terminate the parameter adjustment process to obtain the optimized model of NR-KAGNN.

[0129] Step S33: Use the test set and three performance evaluation indicators, namely, hit rate, average reciprocal rank, and normalized discounted cumulative gain, to evaluate the recommendation accuracy of the optimized model of the NR-KAGNN.

[0130] Step S4: Using the obtained NR-KAGNN optimization model, provide the next recommendation for the user in the conversation dataset or the actual conversation data.

[0131] The following is a specific example to further illustrate the specific implementation of the technical solution of the present invention. The technical solution of the present invention (i.e., NR-KAGNN) is experimentally compared with two of the most advanced multi-type behavior session recommendation models: the MGNN-Spred model (see: Wen Wang, Wei Zhang, Shukai Liu, Qi Liu, Bo Zhang, Leyu Lin, Hongyuan Zha: Beyond Clicks: Modeling Multi-Relational Item Graph for Session-Based Target Behavior Prediction. WWW 2020: 3056-3062. https: / / doi.org / 10.1145 / 3366423.3380077) and the MKM-SR model (see: Wenjing Meng, Deqing Yang, Yanghua Xiao: Incorporating User Micro-behaviors and Item Knowledge into Multi-task Learning for Session-based Recommendation. SIGIR 2020: 1091-1100. https: / / doi.org / 10.1145 / 3397271.3401098) performed a performance comparison of recommendation accuracy using three performance evaluation metrics: hit rate @ Nominal (HR@N), mean reciprocal rank (MRR@N), and normalized discounted cumulative gain (NDCG@N).

[0132] The specific embodiment uses the MovieLens 10M dataset in the film field (see: F. Maxwell Harper, Joseph A. Konstan: The MovieLens Datasets: History and Context. ACM Trans. Interact. Intell. Syst. 5(4): 19: 1-19: 19 (2016) https: / / doi.org / 10.1145 / 2827872). The MovieLens 10M dataset is an interaction dataset that contains approximately 10 million ratings (1 to 5) from 71,567 users on 10,681 movies. The method and steps for converting the MovieLens 10M interaction dataset into a session dataset are as follows: first, the interaction data in the dataset is grouped by user, and then the interaction data of each user is processed as follows: the interaction data is sorted in ascending order of the interaction timestamp, and the sorted interaction data is divided into several sessions at a certain time interval to form the session data of the user; the session data of all users constitute a session dataset.

[0133] Table 1 gives a small part of the data in the MovieLens 10M dataset, and Table 2 gives a small part of the data in the conversation dataset converted from MovieLens 10M.

[0134] Table 1. A small portion of the data in the MovieLens 10M dataset

[0135] User ID Movie ID User Rating Timestamp (unit: seconds) 1 122 5 1112486027 1 185 3.5 1112484676 1 231 3.5 1112484676 ... ... ... 3676 5816 3 1282559851 3676 5888 4 1409090042 3676 5902 3.5 1364639781 3676 5903 5 1282559745 ... ... ... 71567 260 5 912577968 71567 261 2 912578016 71567 266 2 912578173

[0136] Table 2. A small portion of the conversational dataset converted from MovieLens 10M

[0137] Session ID User ID Movie ID User Rating Timestamp (unit: seconds) 1 1 924 4 1094785598 1 1 919 4 1094785921 1 1 2683 5 1094786190 2 1 422 3.5 1094786223 2 1 5436 3 1094786402 2 1 2423 3.5 1094786532 2 1 6547 5 1094786732 2 1 32 3 1094786821 ... ... ... ... ... 42 1 463 3 1094811389 43 2 232 4 1228632147 43 2 678 3.5 1228632362 43 2 345 5 1228632589 43 2 5468 5 1228632739 ... ... ... ... ... 2218251 67788 2338 5 1175543476 2218251 67788 2384 2 1175543454

[0138] Furthermore, in order to compare the recommendation accuracy of the technical solution of the present invention (i.e., the NR-KAGNN model) with the MGNN-Spred model and the MKM-SR model using the above dataset, the inventors managed to implement these three models using the Pycharm software development platform and Python programming language. The specific implementation techniques are as follows:

[0139] MGNN-SPred model implementation technology: The implementation tools are Python 3.6 and TensorFlow 1.12.0. The model code is directly obtained from the MGNN-SPred GitHub repository (see: https: / / github.com / Autumn945 / MGNN-SPred).

[0140] MKM-SR model implementation technology: The implementation tools are Python 3.6 and PyTorch 1.5.0. The model code is directly obtained from the MKM-SR GitHub repository (see: https: / / github.com / ciecus / MKM-SR).

[0141] NR-KAGNN model implementation techniques: Python 3.6.9 and TensorFlow 1.12.0 are used. The implementation techniques are as follows: To build the knowledge-enhanced multi-type behavior conversation graph generation layer, download the code from GitHub (see: https: / / github.com / xiangwang1223 / knowledge_graph_attention_network) and make necessary modifications. To build the attention aggregation layer on the knowledge-enhanced multi-type behavior conversation graph, download the code from GitHub (see: https: / / github.com / Autumn945 / MGNN-SPred) and make necessary modifications. To build the embedding generation layer for multi-type behavior conversations, download the code from GitHub (see: https: / / github.com / Autumn945 / MGNN-SPred) and make necessary modifications. To build the next-item probability prediction layer, use the nn class and the dense class of the layer module in the deep learning framework TensorFlow (see: https: / / tensorflow.google.cn / ).

[0142] The software operating environment of the experiment is as follows: (1) Hardware platform: Intel(R) Xeon(R) Silver 4116CPU@2.10GHz×48, 2 processors, each with 12 cores, 96GB memory, and GPU model: GEFORCE RTX 2080Ti; (2) Operating system: Ubuntu 18.04.5 LTS; (3) Software platform: Pycharm.

[0143] The experimental process is as follows:

[0144] Experimental process of the MGNN-SPred model: (1) Set the values ​​of hyperparameters according to the recommendations given in the original publication of the MGNN-SPred model (see: Wen Wang, Wei Zhang, Shukai Liu, Qi Liu, Bo Zhang, Leyu Lin, Hongyuan Zha: Beyond Clicks: Modeling Multi-Relational Item Graph for Session-Based Target Behavior Prediction. WWW 2020: 3056-3062. https: / / doi.org / 10.1145 / 3366423.3380077); (2) Calculate the recommendation accuracy of the MGNN-SPred model on the test set of the conversational dataset converted from MovieLens 10M.

[0145] Experimental process of the MKM-SR model: (1) Set the values ​​of hyperparameters according to the recommendations given in the original publication of the MKM-SR model (see: Wenjing Meng, Deqing Yang, Yanghua Xiao: Incorporating User Micro-behaviors and ItemKnowledge into Multi-task Learning for Session-based Recommendation. SIGIR2020: 1091-1100. https: / / doi.org / 10.1145 / 3397271.3401098); (2) Calculate the recommendation accuracy of the MKM-SR model on the test set of the conversational dataset converted from MovieLens 10M.

[0146] The experimental process of the NR-KAGNN model is as follows: (1) each session in the session dataset converted from MovieLens 10M is represented as a sequence of items in the session, a sequence of target behaviors in the session, and a sequence of auxiliary behaviors in the session, and the session dataset is divided into a training set, a test set, and a validation set; (2) the training set is used to construct an initial model of the next item recommendation method NR-KAGNN based on the knowledge-enhanced attention graph neural network; (3) the parameters of the initial model of NR-KAGNN are adjusted using the training set and the validation set to obtain an optimized model of NR-KAGNN, and the test set is used to evaluate the recommendation accuracy of the optimized model.

[0147] With respect to the specific embodiment, the specific implementation of the technical solution of the present invention is as follows:

[0148] like Figure 1 As shown, a next item recommendation method based on knowledge-enhanced attention graph neural network includes the following steps S1 to S4:

[0149] Step S1: Represent each session in a session dataset as a sequence of items in the session, a sequence of target behaviors in the session, and a sequence of auxiliary behaviors in the session, and divide the session dataset into a training set, a test set, and a validation set; in this embodiment: the session dataset is a MovieLens The total number of sessions converted from the 10M data set is 2218251. According to the common practice in this technical field, the session data set is first divided into a training set and a test set at a ratio of 8:2, and then 10% of the training set is used as a validation set. Two points need to be further explained: (1) If the total number of sessions in some session data sets cannot be strictly divided into training set, test set and validation set according to the above ratio, then the number of sessions contained in the training set, test set and validation set needs to be fine-tuned during the division process to maximize the approximation to the ratio. The session data set in this embodiment does encounter such a situation. Therefore, after the said fine-tuning in the experiment, the number of sessions contained in the training set, test set and validation set are 1597141, 443650 and 177460 respectively; (2) According to the common practice in this technical field, a session data set can also be divided into a training set, a test set and a validation set in sequence according to other division ratios, and the said division ratios include but are not limited to 8:1:1 and 7:1:2.

[0150] Step S2: Using the training set, construct an initial model of the next recommendation method NR-KAGNN based on knowledge-enhanced attention graph neural network; Figure 2 and Figure 3 As shown, the step S2 further includes:

[0151] Step S21: constructing a knowledge-enhanced multi-type behavior conversation graph generation layer in the initial model of the next recommendation method NR-KAGNN, and using the layer to obtain a knowledge-enhanced multi-type behavior conversation graph; Step S21 further includes:

[0152] Step S211: Represent each item in the conversation dataset as a d-dimensional vector, where the hyperparameter d∈{64, 128, 256}, thereby obtaining the initial embedding of all items in the conversation dataset; in this embodiment: the value of d is 64;

[0153] Step S212: construct a multi-type behavior conversation graph using the conversation item sequence, the conversation target behavior sequence, and the conversation auxiliary behavior sequence. The multi-type behavior conversation graph is defined as follows:

[0154] Each node in the session graph represents an item in the item sequence of the session. Each directed edge in the session graph represents a relationship between a pair of items. The relationship is represented as a triple: (head item, tail item, edge type). There are three types of edge types: target behavior, auxiliary behavior, and item sequence. A target behavior edge indicates that in the target behavior sequence of the session, the user interacts with a head item and then interacts with a tail item. An auxiliary behavior edge indicates that in the auxiliary behavior sequence of the session, the user interacts with a head item and then interacts with a tail item. An item sequence edge indicates that in the item sequence of the session, the user interacts with a head item and then interacts with a tail item.

[0155] Step S213: aligning all items in the item sequence in the session with entities in a knowledge graph to obtain an item entity set, and forming a multi-hop neighborhood of each entity in the item entity set in the knowledge graph;

[0156] Step S214: Generate initial embeddings for each entity and each relationship in the knowledge graph using the knowledge graph embedding model TransR, and use the knowledge graph attention network to perform multi-hop attention embedding propagation to generate embeddings for all entities in the project entity set. The multi-hop attention embedding propagation specifically includes:

[0157] The four steps of information propagation, information aggregation, high-order propagation, and full connection are performed in sequence to generate the embeddings of all entities in the project entity set, as follows:

[0158] The information propagation step calculates the relationship coefficient between each entity in the project entity set and each direct neighbor of the entity in the knowledge graph through a relational attention mechanism, and obtains the neighbor information of the entity after performing weighted aggregation on all direct neighbors of the entity based on these relationship coefficients;

[0159] The information aggregation step uses a bidirectional interaction aggregator to aggregate the initial embedding of each entity and the neighbor information of the entity to form the first-hop embedding of the entity;

[0160] The high-order propagation step collects information from its higher-hop neighbors for each entity by sequentially executing the above-mentioned information propagation step and information aggregation step multiple times. Furthermore, the H-th hop embedding of an entity is generated by aggregating the entity's H-1-th hop embedding and the entity's H-1-th hop neighbor information, where the hyperparameter H∈{1, 2, 3, 4, 5}; in this embodiment: the value of H is 2;

[0161] The fully connected step concatenates the 1st, 2nd, ..., Hth hop embeddings of each entity after multi-hop attention embedding propagation into a vector, and then performs a linear transformation on the vector to obtain the embedding of the entity;

[0162] Step S215: Using a fusion function, the initial embeddings of the items in the multi-type behavior conversation graph are fused with the embeddings of the entities in the item entity set to obtain a knowledge-enhanced multi-type behavior conversation graph. The fusion function is defined by the following formula:

[0163]

[0164] Where || is the vector concatenation operation, is the embedding of an entity in the project entity set, v i is the initial embedding of an item in the multi-type behavior session graph, i∈{1, 2, ..., n}, n is the total number of nodes in the multi-type behavior session graph, is a weight matrix, the hyperparameter d∈{64, 128, 256} is the vector dimension of the initial embedding of the item, and the calculation result x i is the embedding of an item in the knowledge-enhanced multi-type behavior conversation graph; in this embodiment: the value of d is 64;

[0165] Step S22: constructing an attention aggregation layer on the knowledge-enhanced multi-type behavior conversation graph in the initial model of the next recommendation method NR-KAGNN, and using the layer to obtain a target behavior sequence diagram and an auxiliary behavior sequence diagram; Step S22 further includes:

[0166] Step S221: The attention aggregation layer on the knowledge-enhanced multi-type behavior conversation graph directly uses a K-layer attention graph neural network to perform multiple rounds of updates on the embeddings of all items in the knowledge-enhanced multi-type behavior conversation graph to form an aggregated knowledge-enhanced multi-type behavior conversation graph, where the hyperparameter K∈{0, 1, 2, 3, 4}, and decomposes the multi-type behavior conversation graph into a target behavior sequence graph and an auxiliary behavior sequence graph, as follows:

[0167] First, based on the type and direction of the directed edges between any item and its direct neighbors in the knowledge-enhanced multi-type behavior conversation graph, the K-layer attention graph neural network divides the item's direct neighbors into six types: target behavior forward neighbors, target behavior backward neighbors, auxiliary behavior forward neighbors, auxiliary behavior backward neighbors, item sequence forward neighbors, and item sequence backward neighbors. Thus, all neighbors of the same type for each item in the knowledge-enhanced multi-type behavior conversation graph constitute the item's neighbor set of that type, forming six types of neighbor sets for the item: target behavior forward neighbor set, target behavior backward neighbor set, auxiliary behavior forward neighbor set, auxiliary behavior backward neighbor set, item sequence forward neighbor set, and item sequence backward neighbor set.

[0168] Then, the layer number k of the K-layer attention graph neural network is used as the loop variable of the counting control loop, with its initial value being 1, its termination value being K, and the incremental step being 1, and the following steps (1) to (3) are repeated K times:

[0169] (1) The six types of neighbor sets of each item v in the knowledge-enhanced multi-type behavior conversation graph are average pooled to obtain the k-th layer embedding of the six types of neighbor sets of item v, which are the embeddings of the target forward neighbor set respectively. Embedding of target backward neighbor set Embedding of auxiliary forward neighbor sets Embedding of auxiliary backward neighbor sets Embedding of forward neighbor sets of item sequences Backward Neighborhood Embedding of Item Sequences

[0170] (2) Use the attention mechanism to aggregate the k-th layer embeddings of the six types of neighbor sets of item v, thereby obtaining the k-th layer embeddings of the direct neighbors of item v The calculation process is defined by the following formula:

[0171]

[0172]

[0173]

[0174]

[0175] Where, and They are all weight parameters, and their values ​​are determined by parameter learning. The hyperparameter d∈{64, 128, 256} is the vector dimension of the initial embedding of the project, and tanh is an activation function. is the matrix transpose operation;

[0176] (3) Embed the k-1th layer of item v and the k-th layer embedding of the direct neighbors of item v After adding, we get the k-th layer embedding of item v Among them, when k=1, the k-1=0th layer embedding of item v is the embedding of item v calculated in step S215;

[0177] Then, after K repetitions, the embedding of item v is updated to the K-th layer embedding Then, the embeddings of all items 1, 2, ..., n in the knowledge-enhanced multi-type behavior conversation graph are updated to the K-th layer embedding Each embedding The hyperparameter d∈{64, 128, 256} is the vector dimension of the initial embedding of the items, thus forming an aggregated knowledge-enhanced multi-type behavior conversation graph;

[0178] Finally, the K-layer attention graph neural network decomposes the aggregated knowledge-enhanced multi-type behavior conversation graph into a target behavior sequence graph and an auxiliary behavior sequence graph based on the target behavior sequence and the auxiliary behavior sequence of the conversation. In this embodiment, the value of K is 1.

[0179] Step S23: constructing an embedding generation layer for multi-type behavior sessions in the initial model of the next recommendation method NR-KAGNN, and using the layer to generate an embedding for the multi-type behavior session; step S23 further includes:

[0180] Step S231: performing average pooling on the target behavior sequence graph and the auxiliary behavior sequence graph respectively to obtain the embedding of the target behavior sequence and the embedding of the auxiliary behavior sequence, and using a gated fusion mechanism to calculate the relative importance weights of the embeddings of the two sequences. Finally, performing a weighted summation of the embeddings of the two sequences according to the relative importance weights to generate an embedding of a multi-type behavior session;

[0181] Step S24: constructing a next item probability prediction layer in the initial model of the next item recommendation method NR-KAGNN, and using the layer to generate a next item probability vector for all items in the conversation dataset, wherein each component represents the probability that the corresponding item will become the next interactive item of the user; Step S24 further includes:

[0182] Step S241: performing a vector dot product between the embedding of the multi-type behavior session and the initial embedding of all items in the session dataset to obtain a score vector;

[0183] Step S242: Use a softmax function to normalize the score vector and generate a next item probability vector for all items in the conversation dataset, where each component represents the probability that the corresponding item is the user's next interaction item.

[0184] Step S3: Parameter adjustment is performed on the initial model of the NR-KAGNN using the training set and the validation set to obtain an optimized model of the NR-KAGNN, and the recommendation accuracy of the optimized model is evaluated using the test set; Step S3 further includes:

[0185] Step S31: Using the Adam optimizer to learn the parameters of the initial model or the tuned model of the NR-KAGNN on the training set, and calculating the recommendation accuracy of the model on the validation set, specifically including:

[0186] First, the grid search method is used to set the values ​​of the following five hyperparameters for the model: the vector dimension d of the initial embedding of the item, the number of hops H of the multi-hop attention embedding propagation, the number of layers K of the attention graph neural network, the learning rate η, and the L2 regularization coefficient λ. The search range of these hyperparameters, i.e., the domain, is as follows: the domain of d is the set {64, 128, 256}, the domain of H is the set {1, 2, 3, 4, 5}, the domain of K is the set {0, 1, 2, 3, 4}, the domain of η is the set {0.001, 0.01, 0.1}, and the domain of λ is the set {10 -7 , 10 -6 , 10 -5 , 10 -4 , 10 -3 In this embodiment, the value of d is 64, the value of H is 2, the value of K is 1, the value of η is 0.01, and the value of λ is 10. -5 ;

[0187] Then, the Adam optimizer is used on the training set to learn the parameter values ​​of the model to obtain the parameter-adjusted model of NR-KAGNN;

[0188] Finally, the recommendation accuracy of the model is calculated on the validation set;

[0189] Step S32: Determine whether the recommendation accuracy of the NR-KAGNN model after parameter adjustment reaches a predetermined threshold, where the value range of the threshold is a closed interval [0.6, 0.7]. If the recommendation accuracy of the model has not reached the predetermined threshold, continue to adjust the parameters by repeating steps S31 and S32 in sequence. Otherwise, terminate the parameter adjustment process to obtain the optimized model of NR-KAGNN. In this embodiment, the predetermined threshold is set to HR@20=0.7.

[0190] Step S33: Use the test set and the three performance evaluation indicators of hit rate, average reciprocal ranking and normalized cumulative loss gain to evaluate the recommendation accuracy of the obtained NR-KAGNN optimization model; in this embodiment: the recommendation accuracy results of the NR-KAGNN optimization model are shown in Table 3, which also lists the recommendation accuracy results of the MGNN-Spred model and the MKM-SR model. The recommendation accuracy given in the table specifically has three evaluation indicators: hit rate (HR@20), average reciprocal ranking (MRR@20) and normalized cumulative loss gain (NDCG@20). The larger the indicator value, the better the performance.

[0191] Table 3. Next item recommendation accuracy results of three models on the conversational dataset converted from MovieLens 10M

[0192] Model HR@20 MRR@20 NDCG@20 NR-KAGNN 0.714 0.357 0.401 MGNN-SPred 0.643 0.308 0.361 MKM-SR 0.681 0.338 0.387

[0193] Step S4: Using the obtained NR-KAGNN optimization model to provide the next recommendation for the users in the conversation dataset or the actual conversation data; in this embodiment: using the obtained NR-KAGNN optimization model to provide the next recommendation for the users in the test set.

[0194] The above experimental results show that compared with the MGNN-SPred model and the MKM-SR model, the recommendation accuracy of the next item recommendation method NR-KAGNN based on the knowledge enhanced attention graph neural network provided by the present invention is the highest among the three models.

[0195] In general, it can be understood from the above technical solutions and specific implementation methods of the present invention that the beneficial technical effects of the present invention mainly include three aspects: (1) The present invention provides a next item recommendation method with high recommendation accuracy, which has scientific significance for the development of next item recommendation technology; (2) The next item recommendation method NR-KAGNN of the present invention has higher recommendation accuracy than the typical multi-type behavior conversation recommendation models MGNN-Spred and MKM-SR. The reason is that the technical solution of the present invention fully utilizes the user's multi-type behavior information and the item knowledge in the knowledge graph by organically combining means such as the knowledge graph attention network and the attention graph neural network to enhance the embedding of multi-type behavior conversations; (3) The present invention has broad application prospects in many fields such as music, movies, online news, and e-commerce.

[0196] The foregoing describes only some embodiments of the present invention. It should be noted that, except for the technical features described in the specification, all other aspects are known to those skilled in the art. Those skilled in the art may make improvements and modifications without departing from the principles of the present invention, and such improvements and modifications should also be considered within the scope of protection of the present invention.

Claims

1. A next item recommendation method based on knowledge-enhanced attention graph neural network, comprising the following steps: Step S1: Represent each session in a session dataset as a sequence of items in the session, a sequence of target behaviors in the session, and a sequence of auxiliary behaviors in the session, and divide the session dataset into a training set, a test set, and a validation set; Step S2: using the training set to construct an initial model of the next item recommendation method NR-KAGNN based on knowledge-enhanced attention graph neural network; Step S3: Parameter adjustment is performed on the initial model of the NR-KAGNN using the training set and the validation set to obtain an optimized model of the NR-KAGNN, and the recommendation accuracy of the optimized model is evaluated using the test set; Step S4: Using the obtained NR-KAGNN optimization model, provide the next recommendation for the user in the conversation dataset or the actual conversation data; The step S2 further comprises the following steps: Step S21: Constructing a knowledge-enhanced multi-type behavior conversation graph generation layer in the initial model of the next recommendation method NR-KAGNN. The processing steps of the conversation graph generation layer are as follows: Step S211: Represent each item in the conversation dataset as a d-dimensional vector, where the hyperparameter d∈{64, 128, 256}, and obtain the initial embedding of all items in the conversation dataset; Step S212: construct a multi-type behavior conversation graph using the conversation item sequence, the conversation target behavior sequence, and the conversation auxiliary behavior sequence. The multi-type behavior conversation graph is defined as follows: Each node in the session graph represents an item in the sequence of items in the session. Each directed edge in the session graph represents a relationship between a pair of items. The relationship is represented as a triple: (head item, tail item, edge type), where There are three types of edges: target behavior, auxiliary behavior, and item sequence. A target behavior edge indicates that in the target behavior sequence of a session, the user interacts with a head item and then interacts with a tail item. An auxiliary behavior edge indicates that in the auxiliary behavior sequence of a session, the user interacts with a head item and then interacts with a tail item. An item sequence edge indicates that in the item sequence of the session, the user interacts with a head item and then interacts with a tail item. Step S213: Align all items in the item sequence in the session with entities in a knowledge graph to obtain an item entity set, and form a multi-hop neighborhood for each entity in the item entity set in the knowledge graph; Step S214: Generate the initial embedding of each entity and each relationship in the knowledge graph through the knowledge graph embedding model TransR, and use the knowledge graph attention network to perform multi-hop attention embedding propagation to generate the embedding of all entities in the project entity set; Step S215: using a fusion function to fuse the initial embedding of the item in the multi-type behavior conversation graph with the embedding of the entity in the item entity set to obtain a knowledge-enhanced multi-type behavior conversation graph; Step S22: Constructing an attention aggregation layer on the knowledge-enhanced multi-type behavior conversation graph in the initial model of the next item recommendation method NR-KAGNN. The attention aggregation layer directly uses a K-layer attention graph neural network to perform multiple rounds of updates on the embeddings of all items in the knowledge-enhanced multi-type behavior conversation graph to form an aggregated knowledge-enhanced multi-type behavior conversation graph, where the hyperparameter K∈{0, 1, 2, 3, 4}, and decomposes the multi-type behavior conversation graph into a target behavior sequence graph and an auxiliary behavior sequence graph. Step S23: constructing an embedding generation layer for multi-type behavior sessions in the initial model of the next recommendation method NR-KAGNN, and using the layer to generate an embedding for the multi-type behavior session; Step S24: Construct a next item probability prediction layer in the initial model of the next item recommendation method NR-KAGNN, and use this layer to generate a next item probability vector for all items in the conversation dataset, where each component represents the probability that the corresponding item will become the user's next interaction item.

2. The method according to claim 1, characterized in that The step S21 further includes: The multi-hop attention embedding propagation specifically includes: The four steps of information propagation, information aggregation, high-order propagation, and full connection are performed in sequence to generate the embeddings of all entities in the project entity set, as follows: The information propagation step calculates the relationship coefficient between each entity in the project entity set and each direct neighbor of the entity in the knowledge graph through a relational attention mechanism, and obtains the neighbor information of the entity after performing weighted aggregation on all direct neighbors of the entity based on these relationship coefficients; The information aggregation step uses a bidirectional interaction aggregator to aggregate the initial embedding of each entity and the neighbor information of the entity to form the first-hop embedding of the entity; The high-order propagation step collects information from its higher-hop neighbors for each entity by performing the above information propagation step and information aggregation step multiple times in sequence. Furthermore, the H-th hop embedding of an entity is generated by aggregating the entity's H-1-th hop embedding and the entity's H-1-th hop neighbor information, where the hyperparameter H∈{1, 2, 3, 4, 5}; The fully connected step concatenates the 1st, 2nd, ..., Hth hop embeddings of each entity after multi-hop attention embedding propagation into a vector, and then performs a linear transformation on the vector to obtain the embedding of the entity; The fusion function is defined by the following formula: Where || is the vector concatenation operation, is the embedding of an entity in the project entity set, v i is the initial embedding of an item in the multi-type behavior session graph, i∈{1, 2, ..., n}, n is the total number of nodes in the multi-type behavior session graph, is a weight matrix, the hyperparameter d∈{64, 128, 256} is the vector dimension of the initial embedding of the item, and the calculation result x i is the embedding of an item in the knowledge-enhanced multi-type behavior conversation graph.

3. The method according to claim 2, characterized in that The step S22 further includes the following steps: First, based on the type and direction of the directed edges between any item and its direct neighbors in the knowledge-enhanced multi-type behavior conversation graph, the K-layer attention graph neural network divides the item's direct neighbors into six types: target behavior forward neighbors, target behavior backward neighbors, auxiliary behavior forward neighbors, auxiliary behavior backward neighbors, item sequence forward neighbors, and item sequence backward neighbors. Thus, all neighbors of the same type for each item in the knowledge-enhanced multi-type behavior conversation graph constitute the item's neighbor set of that type, forming six types of neighbor sets for the item: target behavior forward neighbor set, target behavior backward neighbor set, auxiliary behavior forward neighbor set, auxiliary behavior backward neighbor set, item sequence forward neighbor set, and item sequence backward neighbor set. Then, the layer number k of the K-layer attention graph neural network is used as the loop variable of the counting control loop, with its initial value being 1, its termination value being K, and the incremental step being 1, and the following steps (1) to (3) are repeated K times: (1) The six types of neighbor sets of each item v in the knowledge-enhanced multi-type behavior conversation graph are average pooled to obtain the k-th layer embedding of the six types of neighbor sets of item v, which are the embeddings of the target forward neighbor set respectively. Embedding of target backward neighbor set Embedding of auxiliary forward neighbor sets Embedding of auxiliary backward neighbor sets Embedding of forward neighbor sets of item sequences Backward Neighborhood Embedding of Item Sequences (2) Use the attention mechanism to aggregate the k-th layer embeddings of the six types of neighbor sets of item v, thereby obtaining the k-th layer embeddings of the direct neighbors of item v The calculation process is defined by the following formula: Where, and They are all weight parameters, and their values ​​are determined by parameter learning. The hyperparameter d∈{64, 128, 256} is the vector dimension of the initial embedding of the project, tanh is an activation function, and T is the matrix transpose operation; (3) Embed the k-1th layer of item v and the k-th layer embedding of the direct neighbors of item v After adding, we get the k-th layer embedding of item v Among them, when k=1, the k-1=0th layer embedding of item v is the embedding of item v calculated in step S215; Then, after K repetitions, the embedding of item v is updated to the K-th layer embedding Then, the embeddings of all items 1, 2, ..., n in the knowledge-enhanced multi-type behavior conversation graph are updated to the K-th layer embedding Each embedding The hyperparameter d∈{64, 128, 256} is the vector dimension of the initial embedding of the items, thus forming an aggregated knowledge-enhanced multi-type behavior conversation graph; Finally, the K-layer attention graph neural network decomposes the aggregated knowledge-enhanced multi-type behavior conversation graph into a target behavior sequence graph and an auxiliary behavior sequence graph according to the target behavior sequence of the conversation and the auxiliary behavior sequence of the conversation.

4. The method according to claim 3, characterized in that The step S23 further includes the following steps: Step S231: Perform average pooling on the target behavior sequence diagram and the auxiliary behavior sequence diagram respectively to obtain the embedding of the target behavior sequence and the embedding of the auxiliary behavior sequence, and use a gated fusion mechanism to calculate the relative importance weights of the embeddings of the two sequences. Finally, perform weighted summation on the embeddings of the two sequences according to the relative importance weights to generate an embedding for a multi-type behavior session.

5. The method according to claim 4, characterized in that The step S24 further includes the following steps: Step S241: performing a vector dot product between the embedding of the multi-type behavior session and the initial embedding of all items in the session dataset to obtain a score vector; Step S242: Use a softmax function to normalize the score vector and generate a next item probability vector for all items in the conversation dataset, where each component represents the probability that the corresponding item is the user's next interaction item.

6. The method according to claim 5, characterized in that The step S3 further comprises the following steps: Step S31: Using the Adam optimizer to learn the parameters of the initial model or the tuned model of the NR-KAGNN on the training set, and calculating the recommendation accuracy of the model on the validation set, specifically including: First, the grid search method is used to set the values ​​of the following five hyperparameters for the model: the vector dimension d of the initial embedding of the item, the number of hops H of the multi-hop attention embedding propagation, the number of layers K of the attention graph neural network, the learning rate η, and the L2 regularization coefficient λ. The search range of these hyperparameters, i.e., the domain, is as follows: the domain of d is the set {64, 128, 256}, the domain of H is the set {1, 2, 3, 4, 5}, the domain of K is the set {0, 1, 2, 3, 4}, the domain of η is the set {0.001, 0.01, 0.1}, and the domain of λ is the set {10 -7 , 10 -6 , 10 -5 , 10 -4 , 10 -3 }; Then, the Adam optimizer is used on the training set to learn the parameter values ​​of the model to obtain the parameter-adjusted model of NR-KAGNN; Finally, the recommendation accuracy of the model is calculated on the validation set; Step S32: Determine whether the recommendation accuracy of the NR-KAGNN model after parameter adjustment reaches a predetermined threshold, where the threshold value range is a closed interval [0.6, 0.7]. If the recommendation accuracy of the model has not reached the predetermined threshold, continue to adjust the parameters by repeating steps S31 and S32 in sequence. Otherwise, terminate the parameter adjustment process to obtain the optimized model of NR-KAGNN. Step S33: Use the test set and three performance evaluation indicators, namely, hit rate, average reciprocal rank, and normalized discounted cumulative gain, to evaluate the recommendation accuracy of the optimized model of the NR-KAGNN.

Images