Hypergraph contrast learning session recommendation method based on multiple behaviors
By constructing global and local hypergraphs, combining multi-layer convolution and contrast learning, the shortcomings of existing session recommendation systems in processing sparse session data and high-order correlation modeling are solved, and more accurate user intention mining and recommendation are achieved. Improved effect.
Patent Information
- Application Number
- CN202411085162.X
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-08-08
- Publication Date
- 2024-12-06
AI Technical Summary
When processing sparse session recommendation systems, it is difficult for existing session recommendation systems to fully tap user intentions, and relying solely on project ID modeling is not enough to capture the high-order correlation between the user's multiple interaction behaviors, resulting in poor recommendation results.
Using a multi-behavior-based hypergraph comparison learning method, by constructing global hypergraphs and local heterogeneous hypergraphs, combining multi-layer convolution operations and contrast learning, global project embedding and conversation representation are learned, and the relative positions of projects with different interactive behaviors are retained. Fully model high-order correlations.
It improves the accuracy of session recommendations, can extract user preferences more comprehensively, enhance project high-order correlation representation, and improve recommendation results.
Smart Images

Figure CN119089134A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of machine learning prediction, and in particular to a multi-behavior based hypergraph comparative learning conversation recommendation method. Background Art
[0002] Recommendation systems have been widely used in various real-world businesses such as e-commerce and online streaming to alleviate information overload, improve user experience, and increase business revenue. Most traditional recommendation systems assume that user profiles and long-term historical interaction records of users are available, and make recommendations based on this. However, in the real world, user identities and long-term historical interaction records of users may be unknown. In this scenario, only the interaction records in the current session can be used for recommendations. Therefore, session-based recommendation (SBR) has received increasing attention. The basic form of session recommendation is to learn a vector that can effectively represent user intent based on the current session. For sparse session data, relying only on item ID modeling is obviously not enough to fully mine user intent. Therefore, mining auxiliary information other than item ID (such as user-item interaction behavior) to enhance user intent representation has become an important branch of session recommendation research.
[0003] In e-commerce, for merchants, what can generate direct value is the user's purchase behavior. Therefore, some studies on conversational recommendation aim to predict the item of the user's next most likely target behavior (i.e., purchase behavior). There are often many types of interactive behaviors between users and items. In addition to purchase behavior, there are also clicks, adding to shopping carts, etc. However, most existing studies only use a single interactive behavior for recommendation, and only a few researchers consider the role of auxiliary behaviors other than target behaviors (such as adding to shopping carts, clicking, etc.) on recommendations. If only target behavior modeling is used, the rich user preferences hidden in auxiliary behaviors will be ignored. For example, when a user wants to buy an item (i.e., a product), he will frequently click on the item and similar items, and even add the item he is most willing to buy to the shopping cart. For the same user, different interactive behaviors with items show different intentions of the user. Different users will also have different operating habits and show different preferences for the same behavior type. For example, for users with strong shopping purposes, if the user frequently clicks on the items he wants to buy, the user's purchase intention can be learned from the items he clicks; if the user is more inclined to click on various items without purpose, it is difficult to obtain the user's purchase intention based on the items he clicks. Figure 1As shown in the figure, user 1 and user 2 have the same purchase items (i.e., laptop and mobile phone). If the items with auxiliary behaviors (clicks) are not considered, the system will recommend the same items to them. However, in reality, the two users choose different items (user 1 and 2 choose mouse and headphones respectively). Therefore, it is necessary to consider the items of auxiliary behavior interaction. In addition, the items with interactive behaviors with users are often not one-to-one related. The triggering of any interactive behavior item may be driven by multiple previous interactive behaviors. That is, there is often a high-order correlation between items (i.e., a many-to-many correlation between items). For example, the items previously clicked by a user ("Huawei headphones", "Apple headphones") and the items purchased ("Huawei mobile phone", "Huawei watch") jointly determine that the next item purchased by this user is "Huawei headphones". However, when incorporating auxiliary behaviors into conversational recommendations, there are two major challenges.
[0004] (1) Most studies only build conversation graphs on a single session, while most users can only click on a very small part of a large number of items in a session. The number of interactions is very limited. The lack of available data makes it impossible for recommendation models based on a single session to learn accurate user intent. Even if some studies use all sessions to build a global conversation graph, they only model the one-to-one correlation between items and it is difficult to fully model the complex interactions between items. Therefore, the first challenge is to use all sessions to model the high-order correlations between items within the same behavior and between items in different behaviors, so as to mine richer user intent from the perspective of all sessions.
[0005] (2) Most studies model target behaviors and auxiliary behaviors separately, which destroys the relative positions of items of different interactive behaviors in the original session in the session sequence. However, clicking on the same item before and after a purchase behavior obviously expresses different user intentions. In addition, the triggering of a purchase behavior is often the result of the combined effect of multiple previous click behaviors. In other words, the correlation between items of different behaviors is high-order. Therefore, the second challenge is to model the high-order correlation between items of different interactive behaviors (including target behaviors and auxiliary behaviors) to fully explore the user intention in the current session. Summary of the invention
[0006] The purpose of the present invention is to overcome the shortcomings of the prior art and provide a novel hypergraph comparative learning conversation recommendation method based on multiple behaviors, which realizes conversation-based recommendation of the user's next possible interaction item by using the user preferences hidden in the user's auxiliary behaviors and learning richer high-order correlations of items through other conversations.
[0007] In order to achieve the above object, the technical solution adopted by the present invention is a multi-behavior based hypergraph comparative learning conversation recommendation method, which includes the following steps:
[0008] S1. Construct a global hypergraph based on the target behavior sequence and auxiliary behavior sequence of all sessions respectively, combine the two global hypergraphs to perform multi-layer hypergraph convolution operations on the initial item embedding, learn the global item embedding based on the target behavior and the global item embedding based on the auxiliary behavior respectively, and use contrastive learning to enhance the global item embedding based on the target behavior from the global item embedding based on the auxiliary behavior, and use the enhanced global item embedding based on the target behavior as the input of the subsequent components;
[0009] S2, learn local session representation and global session representation respectively: construct a local session heterogeneous hypergraph based on the current session, perform multi-layer heterogeneous hypergraph convolution operations on the local session heterogeneous hypergraph, and then generate local session representation through the readout function. Combined with the enhanced global item embedding, and based on the target behavior sequence and auxiliary behavior sequence in all sessions, use the soft attention mechanism with time information to obtain the global session representation based on the target behavior and the global session representation based on the auxiliary behavior, and then generate the global session representation through the mean pooling operation;
[0010] S3. Construct a multi-behavior line graph for all sessions in the current batch, use the local session representation as the initial embedding of the nodes in the multi-behavior line graph, obtain the line graph session representation through multi-layer line graph convolution operations, and compare it with the global session representation as a self-supervisory signal to enhance the global session representation;
[0011] S4, respectively, the enhanced global session representation, the local session representation and the enhanced global project embedding are subjected to similarity calculation and aggregation operations through the prediction layer, so as to obtain the predicted interaction probability of each project;
[0012] S5. Recommend items based on the predicted interaction probability of each item.
[0013] The initial item embedding consists of encoding the input item ID to obtain each item v i One-hot encoding representation of Use the learnable embedding matrix W1 to convert it into a low-dimensional dense vector representation, that is, the item v i The initial embedding h i ,h i It is expressed as:
[0014]
[0015] in, Represents the transpose of W1.
[0016] The method also includes constructing a global hypergraph based on the target behavior using the target behavior sequences of all sessions when learning the global item embedding; and then performing L gLayer hypergraph convolution operation to obtain global item embedding based on target behavior;
[0017] The auxiliary behavior sequences of all sessions are used to construct a global hypergraph based on auxiliary behaviors. Then, L g Layer hypergraph convolution operation to obtain global item embedding based on auxiliary behaviors;
[0018] The two learned global item embeddings are used as contrast signals to each other, and contrastive learning is used so that they can acquire information from each other, thereby obtaining an enhanced global item embedding.
[0019] Constructing a global hypergraph based on target behavior includes: The global hypergraph based on target behavior is defined as All items in V are nodes of the global hypergraph, is a hyperedge set, a session set The number of sessions in is the number of hyperedges. One session corresponds to one hyperedge. Any hyperedge in Usually contains multiple nodes, each of which is a session s j For each target behavior item contained in (j=1,2,…,J), for the hypergraph The matrix P is used to represent its adjacency relationship, where the element in the i-th row and j-th column is p ij , if the super edge Contains Project v i , then p ij is 1 if the value is true, otherwise it is 0.
[0020] The global hypergraph convolution operation based on the target behavior includes: The global hypergraph convolution is the process of information propagation along the node-hyperedge-node. The specific convolution operation is defined as:
[0021]
[0022] in, For the lth (l=1,2,…,L g ) Layer Project v i Embedded, is the l-1th layer project v k Embedding That is, project v i The initial embedding of
[0023] Each project v i The embedding needs to be done by L g The global hypergraph convolution is performed on each layer, and the embedding obtained in each layer and the initial embedding of the item are averaged and pooled to obtain the global item embedding based on the target behavior. Here’s how:
[0024]
[0025] Construct a global hypergraph for auxiliary action sequences of all sessions Then, after the global hypergraph convolution operation, the project v is learned from the global hypergraph based on the auxiliary behavior i Global item embedding based on auxiliary behavior
[0026] In the recommendation method, contrastive learning for global item embeddings involves learning embeddings of the same item v from different hypergraphs. i Different global embeddings of and Use contrastive learning to complement each other and embed the global and Each is regarded as the true value label of the other, and the contrastive loss function is used for contrastive learning tasks. The standard binary cross entropy loss between positive and negative samples is used as the target of contrastive learning, as shown in formula (4):
[0027]
[0028] in, is a matrix The negative samples obtained by row and column transformation are Is an enhanced global project embedding for all projects The matrix is composed of i as the row index, f(·): is a discriminator function used to score the consistency of the two embeddings of the input, and σ(·) is the Sigmoid function.
[0029] The local conversation representation in the recommendation method includes:
[0030] (1) Constructing a local session heterogeneous hypergraph includes constructing a local session heterogeneous hypergraph based on the current session and defining the local session heterogeneous hypergraph as G loc =(B,V loc ,E loc ), V loc is a node set, that is, a set of items in the current session s. Each node corresponds to an item. loc is a set of hyperedges, and the number of hyperedges is J loc It is determined by the number of target behavior items in s, that is, Each target behavior item is constructed as a hyperedge with an auxiliary behavior item whose timestamp is less than that of this item and greater than that of the previous target behavior item, and is constructed as a hyperedge with an auxiliary behavior item whose timestamp is greater than that of this item and less than that of the next target behavior item. In this way, the target behavior item and its preceding and succeeding auxiliary behavior items are included in two different hyperedges, and the auxiliary behavior item and its preceding and succeeding target behavior items are also included in two different hyperedges, so as to retain the relative position between the target behavior item and the auxiliary behavior item; all target behavior items are constructed as a hyperedge, and the local session heterogeneous hypergraph is constructed with the matrix Q∈R 2×Jloc×x Indicates that if the project Belongs to the hyperedge e loc:j , that is, the τth timestamp interaction behavior in s is Project In the super edge loc:j Then the element q in the matrix Q kjτ =1, otherwise q kjτ =0; hyperedge e loc:j It is Figure G loc The first target behavior item in s and the previous auxiliary behavior item constitute the first hyperedge, and the group of auxiliary behavior items that follow it constitute the second hyperedge, and so on. All target behavior items constitute the Jth hyperedge. loc Super edge;
[0031] (2) Local item embedding aggregation: Local item embedding aggregation is to extract user preferences from the local item embeddings of session s. The local item embeddings are aggregated using a soft attention mechanism. The soft attention mechanism uses two different query vectors, one for each hyperedge e containing all target behavior items in the current session. loc:Jloc and the hyperedge e containing the last target behavior item and its subsequent auxiliary behavior items in the current session loc:Jloc-1 ; Based on these two different query vectors, different session representations are generated after the soft attention mechanism.
[0032] In the recommendation method, the global item embedding is enhanced, and after combining the position information of the items in the two types of behavior sequences of the current session, the soft attention mechanism is used to obtain the global session representation based on the target behavior sequence and the global session representation based on the auxiliary behavior sequence respectively. Then, the global session representation under the two behaviors is subjected to the mean pooling operation to fuse the preferences of the two angles to generate the global session representation; a multi-behavior line graph is constructed based on the global hypergraph, and the line graph session representation is generated through the convolution operation on the multi-behavior line graph; finally, the global session representation and the line graph session representation are used as each other's true value labels for comparative learning to maximize the similarity between the global session representation and the line graph session representation representing the same session to enhance the global session representation;
[0033] The generation of the global session representation includes: for the target behavior sequence in the current session s The enhanced global item embedding for each item in ascending order of timestamp is for Middle The enhanced global item embedding of items is then used with a learnable position embedding matrix for Add a reverse position embedding to each item in the , as follows:
[0034]
[0035] Among them, W m ∈R 2d×d is a trainable parameter matrix, for The item embedding after adding the position information of the i-th item in ;
[0036] Subsequently, the item embeddings with added position information are aggregated using a soft attention mechanism to obtain a global session representation based on the target behavior of session s.
[0037]
[0038] Among them, η i is the attention coefficient of the item at the corresponding position, obtained by formula (17).
[0039]
[0040] Where σ(·) is the Sigmoid function, and (both∈R d×d ) is a trainable parameter matrix, is the mean of all item embeddings after adding position information, that is, Query vector as soft attention mechanism;
[0041] For auxiliary behavior sequences, the corresponding auxiliary behavior-based global session representation can be generated by aggregating the enhanced global item embeddings of the items in the auxiliary behavior sequence
[0042] Finally, the global session representations based on the target behavior and auxiliary behaviors are aggregated using a mean pooling operation to generate a comprehensive global session representation z for session s. g(s) :
[0043]
[0044] Conversational contrastive learning includes:
[0045] (1) Construction of multi-behavior line graph: Multi-behavior line graph G lin It is a heterogeneous ordinary graph constructed by extracting key information from the global hypergraph. Each node in the graph represents a session, and the edge represents the connectivity information between sessions. Undirected edges are used here. lin In the example, two types of edges are set according to different types of interactive behaviors. Each type of edge is extracted from the global hypergraph of the corresponding behavior. Each hyperedge in the graph G lin If a node in If two hyperedges have common nodes, then G lin There is an edge connection between the two corresponding nodes in the graph; the multi-behavior line graph can be formally defined as G lin =(V lin ,E lin ), where the node set Edge Set in, represents the k-th type of edge in a multi-line graph, and Based on b k The set of hyperedges of the global hypergraph of the behavior and the i-th hyperedge in it, is the number of common nodes contained in the two hyperedges; for each edge Set a weight Used to indicate the degree of association between two nodes in a multi-behavior line graph, where is the total number of nodes contained in the two hyperedges;
[0046] (2) Multi-behavior graph convolution: The multi-behavior graph contains the connectivity information between sessions in the global hypergraph. If there is an edge connection between nodes, the nodes are defined as neighbors of each other. The multi-behavior graph convolution is the process of aggregating the neighbor information of the node to the node. According to the constructed multi-behavior graph G lin , define the matrix in is the number of nodes, that is, the number of sessions in the current batch during training, and its elements are the weights a of each edge in the line graph kij ; Add self-connection edges to each node in the line graph, and define the association matrix and k=1,2)=A k +I (I is the identity matrix, the purpose is to add the node's self-connection edge. A k is a matrix with index k in the first dimension of A); define the degree matrix and k=1,2) is a diagonal matrix, Elements in yes The elements in ; the multi-line graph convolution is defined as:
[0047]
[0048] Among them, θ (l) ,θ (l-1) They are the lth (l=1,2,…,L g ) layer and the multi-line graph node embedding of the l-1th layer, and Respectively represent the node's neighbor information from the target behavior type edge and the auxiliary behavior type edge aggregation to the node, and then use Normalize, (·) -1 represents the inverse operation of the matrix, μ is a hyperparameter used to control the degree to which the neighbors of the node in the line graph aggregate information from the edges of the auxiliary behavior type to the node. In the multi-behavior line graph, each node represents a session. The session corresponding to each node is taken as the current session, and the corresponding local session representation is obtained as the initial embedding of the corresponding node (i.e., θ (0) =z s ), through multi-layer convolution operations on multi-behavior line graphs, the information in the local session representation is enhanced to the line graph session representation, and further enhanced to the global session representation through contrastive learning operations;
[0049] After the above convolution process, the nodes of each layer are embedded into θ (l) Take the average to get the line graph session representation corresponding to the current session s (L g +1).
[0050] (3) Create self-supervisory signals: The global session representation and the line graph session representation obtained by learning multi-behavior line graphs are used as each other's true value labels, and the information of each other is supplemented through comparative learning; in the sessions trained in the same batch, the global session representation and the line graph session representation representing the same session are regarded as positively correlated samples, otherwise they are marked as negatively correlated samples; specifically, if the session s j In the current batch training session, use and Represent their global session representation and line graph session representation respectively, then and For positively correlated samples, perform row and column transformations on the matrix composed of the global session representation (or line graph session representation) of all sessions in the current training batch, and take session s j The vector at the original index position is obtained (or ) (or ), for which the standard binary cross entropy loss between positive and negative samples is used as the learning objective and is defined as:
[0051]
[0052] Where σ(·) is the Sigmoid function, f(·): is a discriminator function.
[0053] In the recommendation method, the prediction layer outputs the predicted probability of each interactive item.
[0054] Given the current session s, use the corresponding local session representation z s and the enhanced global session representation z g(s) Respectively with project v i Enhanced global project embedding Do the inner product, calculate the similarity between the session representation and the enhanced global item embedding, and then aggregate to get each candidate item v i Score The specific method is as follows:
[0055]
[0056] Among them, the LeakyRelu(·) function with a hyperparameter γ is used to introduce nonlinear transformation. are the corresponding transposes respectively.
[0057] After that, the softmax function is used to calculate the probability of all items becoming the next target behavior item in the current session:
[0058]
[0059] Among them, the vector
[0060] The advantages of the present invention are: the auxiliary behaviors are introduced to more comprehensively extract user preferences from both local and global perspectives, and a local session heterogeneous hypergraph is designed in the local session representation module, which fully models the high-order correlation of items while retaining the relative positions between items of different behaviors. In the global session representation module, a global hypergraph is constructed, and through the introduction of contrastive learning and the construction of multi-behavior line graphs, the information under different behaviors is better integrated, thereby effectively improving the accuracy of session recommendations. BRIEF DESCRIPTION OF THE DRAWINGS
[0061] The following is a brief description of the contents expressed in the drawings of the present invention and the symbols in the drawings:
[0062] Figure 1 A schematic diagram of the influence of different auxiliary behavior items on the next target behavior item of the present invention;
[0063] Figure 2It is the local session heterogeneous hypergraph of the present invention;
[0064] Figure 3 This is the overall architecture diagram of the model corresponding to the method recommended by the present invention. DETAILED DESCRIPTION
[0065] The specific implementation of the present invention will be further explained in detail below by describing the optimal embodiment with reference to the accompanying drawings.
[0066] 1. Session-based recommendation aims to predict the target behavior item that the user is most likely to interact with next through anonymous session sequences. Most current studies only model session sequences based on the user's target behavior, ignoring the user preferences hidden in the user's auxiliary behaviors. In addition, they use ordinary graphs to model one-to-one item correlations in the current session, and fail to learn richer high-order correlations of items from other sessions. To solve the above problems, this paper proposes a novel multi-behavior-based hypergraph contrastive learning session recommendation model. This model models all sessions as global hypergraphs according to the target behavior sequence and auxiliary behavior sequence, uses contrastive learning to obtain global item embeddings, and further aggregates to generate global session representations to capture the high-order correlations of items from all session perspectives. At the same time, a novel local heterogeneous hypergraph is designed for the current session to capture the high-order correlations between items of different behaviors in the current session, and further learn local session representations. In addition, a multi-behavior line graph is constructed using the current batch of sessions in the training set, and a novel self-supervisory signal is designed for the global session representation to enhance the global session representation. Finally, similarity is calculated and aggregated using local session representations, global session representations, and global item embeddings to obtain the predicted interaction probability of each item. Then, the next interactive item is recommended based on the predicted interaction probability of each item, and the item with the highest probability is recommended as the next interactive item.
[0067] 2. Aiming at the challenges of the prior art mentioned in the background technology, this solution constructs a global hypergraph to learn global item embedding using the target behavior sequence (i.e., the item sequence with only the target behavior) and the auxiliary behavior sequence (i.e., the item sequence with only the auxiliary behavior) in all sessions, so as to capture the high-order correlation of items within the same behavior. In order to obtain a more comprehensive global item embedding, this paper adopts contrastive learning so that the global item embeddings learned from the global hypergraph based on the target behavior and the auxiliary behavior can enhance each other. In order to capture the high-order correlation of items between different behaviors, this paper constructs a multi-behavior line graph, which contains two types of edges, which are related to the target behavior and the auxiliary behavior sequence, respectively. Contrastive learning is also used to enhance the information learned based on the multi-behavior line graph to the global session representation. At the same time, a local session heterogeneous hypergraph is constructed for each current session to capture the high-order correlation between items while retaining the relative positions between items of different interactive behaviors. And by using the soft attention mechanism of two different query vectors, the items in the current session are aggregated from the perspectives of general preference and latest preference, respectively, to accurately extract the user intention in the current session. The items of auxiliary behaviors may be noise themselves (e.g., items clicked aimlessly). Therefore, the sequence of auxiliary behavior items in the current session does not need to be in strict time order, and the collection form of hyperedges can better reflect this. Figure 2 The local session heterogeneous hypergraph shown in Figure 1 (the construction method is detailed in Section 3.2.3 (1), where {1, 2, 3, 4, 5} represents the item ID) is used as nodes in the local session heterogeneous hypergraph. The target behavior item is used as the boundary, and its hyperedges with the previous and following auxiliary behavior items are formed respectively ( Figure 2 In addition, all target behavior items form a hyperedge ( Figure 2 In this way, in the local session heterogeneous hypergraph, the relationship between nodes and hyperedges, and the relationship between hyperedges show the high-order correlation between items in different behaviors.
[0068] 3. Specific principles of the scheme
[0069] This scheme proposes a novel Multi-Behavior Hypergraph Contrastive Leaning for Session-based Recommendation (MHCL), and its main contributions are as follows:
[0070] (1) Based on all sessions, a global hypergraph is constructed for the target behavior sequence and the auxiliary behavior sequence respectively, and contrastive learning is used to better capture the high-order correlation of the items in the two types of behaviors and effectively fuse them to obtain a global session representation. In addition, a multi-behavior line graph is designed as a contrast signal to further model the high-order correlation of items between different behaviors from a global session perspective, effectively enhancing the global session representation.
[0071] (2) A local session heterogeneous hypergraph based on multiple behaviors is constructed. While retaining the relative positions between items of different behaviors in the original session, the high-order correlations between items of different behaviors in the current session are fully exploited to obtain a more accurate local session representation.
[0072] 3.1 Problem Definition
[0073] Given a set of items V = {v1,v2,…,v N}, such as the set of items in the Taobao dataset, each item represents an item, N represents the number of items, and the set of behavior types (i.e., the interaction between users and items, including target behavior and auxiliary behavior) B = {b1, b2}, b1 is the target behavior (such as purchase behavior), and b2 is the auxiliary behavior (such as click behavior). The input of this model is a set of anonymous session sets S = {s1, s2, ..., s J}, represents all sessions, and J is the number of sessions in the session set. Current session It is a sequence composed of interacting items sorted by timestamp from far to near (i.e., from small to large), that is, in is the interaction item at the τth (τ=1,2,…,x)th timestamp in s, and its corresponding behavior types are Representative Projects The interaction behavior type, x is the length of the current session. If the items in the current session under the target behavior and auxiliary behavior are arranged in ascending order by timestamp, the target behavior sequence is obtained (items containing only the target behavior), and auxiliary behavior sequences (items containing only auxiliary behaviors), use and Respectively represent the length of the target behavior sequence and auxiliary behavior sequence in the current session The goal of this model is to learn the preferences of the current session s and predict the most likely target behavior item for the next time stamp x+1. The model will eventually output the probability of all items in V becoming the next recommended item, that is, It is project v i The probability of becoming the next recommended item is calculated, and finally the top K items with the highest probability are used as the recommended candidate items for the current session s.
[0074] 3.2MHCL
[0075] The overall structure of MHCL is as follows: Figure 3 As shown in the figure, s1 is the current session s, and {s1, s2, s3} represent all sessions, forming a session set S. First, a global hypergraph is constructed based on the target behavior sequence and auxiliary behavior sequence of all sessions, and the initial item embedding is subjected to multi-layer hypergraph convolution operations by combining the two global hypergraphs. The global item embeddings based on the target behavior and the auxiliary behavior are learned respectively, and contrastive learning is used to enhance the global item embedding based on the target behavior from the global item embedding based on the auxiliary behavior. The enhanced global item embedding based on the target behavior (hereinafter referred to as "enhanced global item embedding") is used as the input of subsequent components. Then, the local session representation and the global session representation are learned respectively. On the one hand, a local session heterogeneous hypergraph is constructed based on the current session, and multi-layer heterogeneous hypergraph convolution operations are performed on the local session heterogeneous hypergraph, and then the local session representation is generated by the readout function (for the specific implementation of the readout function, see Section 3.2.3, Part (3)). On the other hand, combined with the enhanced global item embedding, and based on the target behavior sequence and auxiliary behavior sequence in all sessions, the soft attention mechanism with time information is used to obtain the global session representation based on the target behavior and the global session representation based on the auxiliary behavior, and then the global session representation is generated through the mean pooling operation. In order to enhance the global session representation, this paper constructs a multi-behavior line graph for all sessions in the current batch, uses the local session representation as the initial embedding of the nodes in the multi-behavior line graph, and obtains the line graph session representation through multi-layer line graph convolution operations. It is compared with the global session representation as a self-supervisory signal for learning to enhance the global session representation. Finally, the enhanced global session representation, local session representation and enhanced global item embedding are respectively passed through the prediction layer for similarity calculation and aggregation operations to obtain the predicted interaction probability of each item. It is project v i The probability of becoming the next recommended item is calculated, and finally the top K items with the highest probability are used as the recommended candidate items for the current session s.
[0076] 3.2.1 Initial Project Embedding
[0077] This paper uses one-hot encoding to encode the input project ID. Each project randomly corresponds to a unique ID in (1, N) as the ID of this project. i One-hot encoding representation of Then, using the learnable embedding matrix w1(∈R N×d , where d is the embedding dimension) to convert it into a low-dimensional dense vector representation, i.e., the item v i The initial embedding h i (∈R d ), the method is shown in formula (1):
[0078]
[0079] in, Represents the transpose of W1.
[0080] 3.2.2 Global Project Embedding
[0081] When learning global item embedding, we first construct a global hypergraph based on the target behavior sequences of all sessions; then we perform L g The hypergraph convolution operation is performed at the layer level to obtain the global item embedding based on the target behavior. In order to learn the useful information of the auxiliary behavior sequence for item embedding, the auxiliary behavior sequences of all sessions are used in the same way as above to obtain the global item embedding based on the auxiliary behavior. The two learned global item embeddings are used as contrast signals to each other, and contrastive learning is used so that they can obtain information from each other, thereby obtaining the enhanced global item embedding. As a result, the enhanced global item embedding incorporates the high-order correlations of items hidden in the target behavior and auxiliary behavior of all sessions, and will be used as the input of subsequent components.
[0082] Next, we take learning global item embeddings based on target behaviors as an example to illustrate the corresponding methods.
[0083] (1) Global hypergraph construction based on target behavior
[0084] The global hypergraph based on the target behavior is defined as All items in V are nodes of the global hypergraph, is a hyperedge set, a session set The number of sessions in is the number of hyperedges (i.e. J), and one session corresponds to one hyperedge. Any hyperedge in Usually contains multiple nodes, each of which is a session s j (j=1,2,…,J) contains each target behavior item. In this paper, the matrix P(∈R N×J ) to represent its adjacency relationship, the element in the i-th row and j-th column of the matrix P is p ij , if the super edge Contains Project v i , then p ij is 1 if the value is true, otherwise it is 0.
[0085] (2) Global Hypergraph Convolution Based on Target Behavior
[0086] The global hypergraph convolution can be regarded as the process of information propagation along the node-hyperedge-node. The specific convolution operation is defined as:
[0087]
[0088] in, For the lth (l=1,2,…,L g ) Layer Project v i Embedded, is the l-1th layer project v k Embedding That is, project v i Initial embedding of p ij and p kj are the elements in the i-th row and j-th column and the elements in the k-th row and j-th column in the matrix P respectively.
[0089] Each project v i The embedding needs to be done by L g The global hypergraph convolution is performed on each layer, and the embedding obtained in each layer and the initial embedding of the item are averaged and pooled to obtain the global item embedding based on the target behavior. Here’s how:
[0090]
[0091] Similarly, the global hypergraph of the target behavior is constructed in the same way as the global hypergraph of the target behavior to construct a global hypergraph for the auxiliary behavior sequences of all sessions. Then, after the same global hypergraph convolution operation as above, the project v is learned from the global hypergraph based on the auxiliary behavior i Global item embedding based on auxiliary behavior
[0092] (3) Contrastive Learning for Global Item Embedding
[0093] Learning from different hypergraphs for the same item v i Different global embeddings of and Contrastive learning is used to complement each other, and they are regarded as each other's true value labels. The contrastive loss function is used for contrastive learning tasks, which encourages the global item embedding based on the target behavior and the global item embedding based on the auxiliary behavior to be similar, thereby obtaining an enhanced global item embedding. Here, the standard binary cross entropy loss between positive and negative samples is used as the contrastive learning target, as shown in formula (4):
[0094]
[0095] in, is a matrix The negative samples obtained by row and column transformation are Is an enhanced global project embedding for all projects The matrix consists of i as the row index, f(·) is a discriminant function used to score the consistency of the two embeddings of the input, and σ(·) is the Sigmoid function. is the standard binary cross entropy loss.
[0096] 3.2.3 Local Session Representation
[0097] As mentioned above, previous work separated the target behavior sequence from the auxiliary behavior sequence, destroying the relative position information between the target behavior items and the auxiliary behavior items in the original session. This paper retains this position information in the process of constructing the local session heterogeneous hypergraph of the current session, and more effectively captures the high-order correlation between the target behavior items and the auxiliary behavior items. The process of learning the local session representation is as follows: first, use the current session s to construct a local session heterogeneous hypergraph, and then perform L s The convolution operation is performed on the heterogeneous hypergraph layer by layer, and finally the local session representation is obtained by embedding aggregation through the readout function.
[0098] (1) Construction of local session heterogeneous hypergraph
[0099] The local session heterogeneous hypergraph is constructed based on the current session. Define the local session heterogeneous hypergraph as G loc =(B,V loc ,E loc ), V loc is a node set, that is, a set of items in the current session s. Each node corresponds to an item. loc is a set of hyperedges, and the number of hyperedges is J loc It is determined by the number of target behavior items in s, that is, Each target behavior item is constructed as a hyperedge with an auxiliary behavior item whose timestamp is less than that of this item and greater than that of the previous target behavior item, and is constructed as a hyperedge with an auxiliary behavior item whose timestamp is greater than that of this item and less than that of the next target behavior item. In this way, the target behavior item and its preceding and succeeding auxiliary behavior items are included in two different hyperedges, and the auxiliary behavior item and its preceding and succeeding target behavior items are also included in two different hyperedges, so as to preserve the relative position between the target behavior item and the auxiliary behavior item. In addition, all target behavior items are constructed as one hyperedge. Matrix for local session heterogeneous hypergraph Indicates that if the project Belongs to the hyperedge e loc:j , that is, the τth timestamp interaction behavior in s is ( Target behavior or auxiliary behavior) In the super edge loc:j Then the element q in the matrix Q kjτ =1, otherwise (like is the target behavior, then k = 1, otherwise k = 2). loc:j It is Figure G loc The first target behavior item in s and the previous auxiliary behavior item constitute the first hyperedge, and the group of auxiliary behavior items that follow it constitute the second hyperedge. In this order, and so on, all target behavior items constitute the Jth hyperedge. loc Super edge.
[0100] (2) Local Session Heterogeneous Hypergraph Convolution
[0101] The information propagation process of the local session heterogeneous hypergraph still adopts the node-hyperedge-node propagation method. During the information propagation process, the items in the local session heterogeneous hypergraph share the information of other items in the same hyperedge as this item. In this way, on the one hand, the high-order correlation of the items in the hyperedge is displayed and can be effectively extracted, including the high-order correlation between the target behavior item and the auxiliary behavior item. On the other hand, the high-order correlation of the items in the adjacent hyperedges is also fully mined after multiple rounds of information propagation (one round of information from "node-hyperedge-node", that is, one layer of hypergraph convolution). When the node propagates information to the hyperedge, in order to distinguish the importance of different interactive behaviors, each interactive behavior is given a corresponding weight. Because different users have different operating habits, we cannot simply set the interactive behavior weights of all users to the same value. In order to better learn personalized behavior weights, and considering that the goal of the MHCL model is to predict the target behavior item of the next interaction, this paper obtains the behavior weight by calculating the item similarity between the auxiliary behavior and the target behavior in each session. This similarity can indicate the amount of auxiliary behavior operations performed by the user on the same or similar items before performing the target behavior. Specifically, for the current session, the enhanced global item embeddings of items under different interactive behaviors are average pooled to obtain the corresponding behavior mean vector When k = 1 or 2, it represents the target behavior and auxiliary behavior respectively. For the current session project The enhanced global item embedding is then used to calculate the similarity between the mean vectors of different behaviors and the mean vector of the target behavior, and normalized to obtain the behavior weights when the nodes in the current session aggregate to the hyperedge. Here’s how:
[0102]
[0103] in, and Represent the target behavior weight and auxiliary behavior weight respectively, represents the mean vector of the target behavior, softmax(·) is the softmax function, and the node-to-hyperedge aggregation operation can be expressed as follows:
[0104]
[0105] in, is the lth (l=1,…,L s ) layer hyperedge e loc:j Embedded, Is the project in the current session The local item embedding at level l-1, specifically, For Project Enhanced global item embedding for .
[0106] As mentioned above, different hyperedges in the local session heterogeneous hypergraph contain different relative position information between different behavior items. To distinguish the importance of different hyperedges, different relative position information can be used to distinguish. Therefore, in the process of hyperedge aggregation to nodes, this paper uses the local item embedding of each item in the previous layer as the query vector and uses the soft attention mechanism to distinguish the importance of different hyperedges. Specifically, for the nodes containing items (and the corresponding behavior is ) loc:j (j∈{1,2,…,J loc} and q kjτ =1), its importance can be obtained by the method of formula (7), that is, the attention coefficient:
[0107]
[0108] in, It is a super edge loc:j At level l, for projects The attention coefficient, and is a trainable parameter matrix, and softmax(·) is the softmax function. Use the attention coefficient to aggregate the hyperedge embedding to generate the items in the current session s The local item embedding at level l is
[0109]
[0110] In order to retain more upper-layer information and prevent the loss of upper-layer information due to the increase in the number of layers in the local heterogeneous hypergraph convolution process, we add and fuse the local item embedding of the lth layer with the local item embedding of the l-1th layer to obtain the item The local item embedding after fusion at layer l Right now Finally, the local items after fusion at each layer are embedded Perform an average pooling operation to get the project The final local item embedding h s:τ :
[0111]
[0112] in,
[0113] (3) Local Project Embedding Aggregation
[0114] The local item embedding aggregation is to extract user preferences from the local item embeddings of session s. The soft attention mechanism has been proven to be effective in previous work, so this paper uses the soft attention mechanism to aggregate local item embeddings. In addition, in order to extract user preferences from different perspectives, two different query vectors are used for the soft attention mechanism, one for each hyperedge containing all target behavior items in the current session. and a hyperedge containing the last target behavior item in the current session and its subsequent auxiliary behavior items The purpose of selecting the above two query vectors is that the former contains all target behavior items, which can reflect the user's general preferences, and the latter contains the last target behavior item and its subsequent auxiliary behavior items, which can better reflect the user's current preferences. Based on these two different query vectors, different session representations are generated after the soft attention mechanism. The method similar to formula (6) is used to aggregate nodes to hyperedges, and the hyperedges are obtained. and The query vector and
[0115] In the soft attention mechanism, the attention coefficient of each item in the current session is first calculated to obtain the importance of each item:
[0116]
[0117] Among them, β τ and τ There are two query vectors and Next Project The attention coefficient, σ(·) is the Sigmoid function, and (both∈R d×d ) is a trainable parameter matrix.
[0118] The above two attention coefficients are used as weights to aggregate the local item embeddings of different items and generate two different session representations for the current session s. and They represent the conversation representation for current preference and the conversation representation for general preference respectively, and the calculation method is as follows:
[0119]
[0120] Finally, the gating mechanism is used to aggregate the above two session representations to obtain a more comprehensive local session representation z s .
[0121]
[0122] Where α is the gating coefficient, and σ(·) is the Sigmoid function, and is a trainable parameter matrix.
[0123] 3.2.4 Global Session Representation
[0124] First, the enhanced global item embedding of the items in Section 3.2.2 is combined with the position information of the items in the two types of behavior sequences in the current session, and the soft attention mechanism is used to obtain the global session representation based on the target behavior sequence and the global session representation based on the auxiliary behavior sequence. They respectively show the user's preferences under the target behavior and the auxiliary behavior. Then, the global session representation under the two behaviors is fused with the preferences from two angles using the mean pooling operation to generate the global session representation. In order to obtain a more comprehensive global session representation, a multi-behavior line graph is constructed based on the global hypergraph, and the line graph session representation is generated through the convolution operation on the multi-behavior line graph, thereby providing a session representation under another view. Finally, the above-mentioned global session representation and line graph session representation are used as each other's true value labels for comparative learning to maximize the similarity between the global session representation and the line graph session representation representing the same session to enhance the global session representation.
[0125] (1) Global session representation generation
[0126] For the target behavior sequence in the current session s The enhanced global item embedding for each item in ascending order of timestamp is for Middle The enhanced global item embedding of items is then used with a learnable position embedding matrix for Add a reverse position embedding to each item in the , as follows:
[0127]
[0128] Among them, W m∈R 2d×d is a trainable parameter matrix, for is the item embedding after adding the position information of the i-th item in , and tanh(·) is the tanh activation function.
[0129] Subsequently, the item embeddings with added position information are aggregated using a soft attention mechanism to obtain a global session representation based on the target behavior of session s.
[0130]
[0131] Among them, η i is the attention coefficient of the item at the corresponding position, obtained by formula (17).
[0132]
[0133] Where σ(·) is the Sigmoid function, and (both∈R d×d ) is a trainable parameter matrix, is the mean of all the above-mentioned item embeddings after adding position information, that is, Query vector as soft attention mechanism.
[0134] For auxiliary behavior sequences, the enhanced global item embeddings of items in the auxiliary behavior sequence can be aggregated through similar operations as above to generate the corresponding auxiliary behavior-based global session representation The only difference is that the position embedding is removed (i.e., there is no need to perform the step (15)), because the auxiliary behavior operation is highly random, the possibility of random clicks is high, and there is no clear time dependency. The details are as follows:
[0135] For the sequence of auxiliary actions in the current session s The enhanced global item embedding for each item in ascending order of timestamp is for Middle The enhanced global item embedding of items is aggregated using the soft attention mechanism to obtain the auxiliary behavior-based global session representation of session s.
[0136]
[0137] Among them, ψ i is the attention coefficient of the item at the corresponding position, obtained by formula (19).
[0138]
[0139] Where σ(·) is the Sigmoid function, and (both∈R d×d ) is a trainable parameter matrix, for The mean value of Query vector as soft attention mechanism.
[0140] Finally, the global session representations based on the target behavior and auxiliary behaviors are aggregated using a mean pooling operation to generate a comprehensive global session representation z for session s. g(s) :
[0141]
[0142] (2) Conversational Contrastive Learning
[0143] First, we introduce auxiliary behavior information to expand it into a multi-behavior model. We use the sessions in the current training batch to construct a multi-behavior line graph based on the global hypergraph of different behaviors. Then, we use the local session representation obtained in Section 3.2.3 as the initial embedding of the nodes in the multi-behavior line graph, and perform multi-layer line graph convolution to learn and obtain the line graph session representation. Finally, we combine the above global session representation and perform comparative learning to enhance the global session representation. The details are as follows:
[0144] Multi-behavior line graph construction: Multi-behavior line graph G lin It is a heterogeneous ordinary graph constructed by extracting key information from the global hypergraph. Each node in the graph represents a session (the number of sessions in the current training batch is the number of nodes in the graph), and the edge represents the connectivity information between sessions. Undirected edges are used here. lin In , two types of edges are set according to different types of interaction behaviors, and each type of edge is extracted from the global hypergraph of the corresponding behavior. Each hyperedge in the graph G lin If a node in If two hyperedges have common nodes, then G lin There is an edge connection between the two corresponding nodes in the global hypergraph. The above situation is similar. Therefore, the multi-behavior line graph can be formally defined as G lin =(V lin ,E lin ), where the node set Edge Set in, represents the kth type of edge in a multi-line graph, and Based on b k The set of hyperedges of the global hypergraph of the behavior and the i-th hyperedge in it, is the number of common nodes contained in the two hyperedges. In addition, for each edge Set a weight Used to indicate the degree of association between two nodes in a multi-behavior line graph, where is the total number of nodes contained in the two hyperedges (duplicate nodes are counted only once).
[0145] Multi-behavior graph convolution: The multi-behavior graph contains the connectivity information between sessions in the global hypergraph. If there is an edge connection between nodes, the nodes are defined as neighbors of each other. The multi-behavior graph convolution is the process of aggregating the neighbor information of the node to the node. According to the multi-behavior graph G constructed above lin , define the matrix is the number of nodes, that is, the number of sessions in the current batch during training), and its elements are the weights a of each edge in the line graph kij In order to avoid the loss of node information during the convolution process, a self-connection edge is added to each node in the line graph, and the association matrix is defined as and k=1,2)=A k +I (I is the identity matrix, the purpose is to add the node's self-connection edge. A k is the matrix of A with index k in the first dimension). Define the degree matrix and k=1,2) is a diagonal matrix, Elements in yes The elements in . The multi-line graph convolution is defined as:
[0146]
[0147] Among them, θ (l) ,θ (l-1) They are the lth (l=1,2,…,L g ) layer and the multi-line graph node embedding of the l-1th layer, and Respectively represent the node's neighbor information from the target behavior type edge and the auxiliary behavior type edge aggregation to the node, and then use Normalize, (·) -1 represents the inverse operation of the matrix. μ is a hyperparameter used to control the degree to which the neighbors of the node in the line graph aggregate information from the edges of the auxiliary behavior type to the node. In the multi-behavior line graph, each node represents a session. Therefore, the session corresponding to each node is taken as the current session, and the corresponding local session representation is obtained using the method in Section 3.2.3 as the initial embedding of the corresponding node (i.e., θ (0) =z s), through multi-layer convolution operations on multi-behavior line graphs, the information in the local session representation can be enhanced to the line graph session representation, and further enhanced to the global session representation through subsequent contrastive learning operations.
[0148] After the above convolution process, the nodes of each layer are embedded into θ (l) Take the average to get the line graph session representation corresponding to the current session s
[0149] Create self-supervisory signals: The global session representation obtained in Section 3.2.4 and the line graph session representation obtained by learning multi-behavior line graphs are used as each other’s true value labels, and the information of each other is supplemented through comparative learning. In the sessions trained in the same batch, the global session representation and line graph session representation representing the same session are regarded as positively correlated samples, otherwise they are marked as negatively correlated samples. Specifically, if the session s j In the current batch training session, use and Represent their global session representation and line graph session representation respectively, then and For positively correlated samples, perform row and column transformations on the matrix composed of the global session representation (or line graph session representation) of all sessions in the current training batch, and take session s j The vector at the original index position is obtained (or ) (or ), for which the standard binary cross entropy loss between positive and negative samples is used as the learning objective and is defined as:
[0150]
[0151] Among them, σ(·) is the Sigmoid function, and f(·) is a discriminant function, which is similar to the discriminator function in Section 3.2.2. Under the optimization of this learning objective, the global session representation can be effectively enhanced.
[0152] 3.2.5 Prediction layer and model optimization
[0153] (1) Prediction layer
[0154] First, given the current session s, use the corresponding local session representation z s and the enhanced global session representation z g(s) Respectively with project v i Enhanced global project embedding Do the inner product, calculate the similarity between the session representation and the enhanced global item embedding, and then aggregate to get each candidate item v i The score yi The specific method is as follows:
[0155]
[0156] Among them, the LeakyRelu(·) function with a hyperparameter γ is used to introduce nonlinear transformation and prevent the gradient from disappearing. are the corresponding transposes respectively.
[0157] After that, the softmax function is used to calculate the probability of all items becoming the next target behavior item in the current session:
[0158]
[0159] Among them, the vector
[0160] Then Arrange them in descending order, and take the first k items as the recommended items for the current session s
[0161] (2) Model Optimization
[0162] Here, the learning objective is modeled as a cross entropy loss function, which is widely used in recommendation systems and is defined as:
[0163]
[0164] Among them, y(∈R N ) is the one-hot encoding vector for the true target item in the current session, y i (∈(0,1)) is the value at index i of vector y, if the real target item of the current session is item v i , then y i =1, the remaining index positions of y are 0.
[0165] Will recommend loss and contrastive learning loss Perform joint optimization and minimize through Adam High-quality conversation recommendations can be obtained.
[0166]
[0167] Among them, δ is a hyperparameter that controls the proportion of contrastive learning loss. During the learning and training process, the training model MHCL is optimized based on the cross entropy loss function, and the parameters in the model are updated by back propagation until the model reaches the optimal value. The trained model has a high accuracy in predicting the probability of outputting the next item.
[0168] 4 Experimental Verification
[0169] 4.1 Research Questions
[0170] To study the performance of MHCL, a large number of experiments are conducted in this section to answer the following questions.
[0171] Question 1: Does MHCL outperform some baseline models on real datasets?
[0172] Question 2: Are the key components in MHCL effective?
[0173] Question 3: How does the performance of MHCL change with changes in key hyperparameters?
[0174] 4.2 Experimental Setup
[0175] 4.2.1 Dataset
[0176] This paper uses three public real-world datasets (JD-Computer, JD-Application, and Taobao) to evaluate the performance of MHCL. JD-Computer and JD-Application are both from the shopping website JD.com. They are anonymous user interaction sessions containing computers and home appliances, respectively, with ten different behavior types. Taobao is a user behavior dataset provided by the Taobao shopping website, which contains four behavior types. The preprocessing method of the dataset is as follows: (1) In the three datasets, this paper regards purchase behavior as the target behavior. JD-Computer and JD-Application retain the add-to-cart behavior as an auxiliary behavior and delete other behavior types. Taobao retains the click behavior as an auxiliary behavior and deletes other behavior types. (2) Delete items with too few interactions in the target behavior and sessions containing too few target behavior items. Delete all auxiliary behavior items after the second to last target behavior item in each session in ascending order of timestamp. (3) Use the sequence splitting method to expand the dataset. For session s=[v1,v2,…,v m ], split it into m subsessions [v1,v2,…,v m ],[v1,v2,…,v m-1 ],…[v1,v2],[v1]. Divide the longest subsession of each session into the test set, i.e., [v1,v2,…,v m ], the last target behavior item in the longest sub-session is taken as the test set label, that is, v m, the remaining sub-sessions are divided into the training set, and the label is the first target behavior item after the last auxiliary behavior item in the sub-session in the original session. (4) Delete the sessions in the training set that have no target behavior items except the label. The information of the preprocessed dataset is shown in Table 1, which gives the number of sessions, the number of items, the number of interactions between target behavior and auxiliary behavior, the average length of all sessions, and the average length of target behavior sequences in all sessions in the training set and test set.
[0177] Table 1 Dataset information statistics
[0178]
[0179]
[0180] 4.2.2 Evaluation Metrics
[0181] To evaluate the performance of MHCL and all baselines, we adopt three widely used evaluation metrics: Hit@k (abbreviated as H@k), MRR@k (abbreviated as M@k), and NDCG@k (abbreviated as N@k), where k (k = 10 or 20) means recommending k candidate items.
[0182] H@k: It is used to measure whether the true target item is in the recommendation list, but does not consider its position in the recommendation list. It is calculated as follows:
[0183]
[0184] Among them, n hit Represents the number of test samples of the real target item in the recommendation list, N test is the number of sessions in the test set. A larger H@k value indicates better performance.
[0185] M@k: The average reciprocal ranking measures the position of the true target item in the recommendation list. The larger the value, the higher the position. If the true target item is not in the recommendation list, it is 0. The calculation is as follows:
[0186]
[0187] Among them, v represents the real target item, and the function rank(v) outputs the position of the real target item v in the recommendation list. k Represents the recommendation list. The larger the M@k value, the better the performance.
[0188] N@k: Relative to M@k, the normalized discounted cumulative gain assigns higher scores to the front positions. The larger the N@k value, the better the performance. It is calculated as follows:
[0189]
[0190] 4.2.3 Baseline
[0191] This article will compare the conversation recommendation model based on graph neural network, the conversation recommendation model based on hypergraph, the conversation recommendation model based on contrastive learning, and the conversation recommendation model based on multiple behaviors.
[0192] SR-GNN treats each session as a graph modeling one-to-one correlation between items and obtains item embedding through graph neural network.
[0193] And use the last item as the query vector to learn the soft attention mechanism to get the session embedding.
[0194] MGIR sets sequence relations, co-occurrence relations, and incompatible relations, and uses the three relations to model the conversation graph and extract more fine-grained project-related relations.
[0195] MSGAT designs multiple sparse graph attention networks and utilizes two parallel channels to model intra-session and inter-session item correlations.
[0196] GIUA-GNN is a model applied to streaming conversation recommendation, and its offline version is used in this paper. It uses a global conversation graph and a bidirectional attention graph convolutional network to learn user embedding and item embedding.
[0197] ·S 2 -DHCN introduces hypergraphs into conversational recommendations, uses hypergraph convolutional networks, and constructs line graphs as contrast signals to further improve performance through contrastive learning.
[0198] CLHHN introduces item attribute information and designs a category-aware lossless heterogeneous hypergraph neural network to recommend possible items to target users based on the item categories.
[0199] COTREC explores intra- and extra-item correlations of a session via contrastive learning and co-training.
[0200] GPG4HSR introduces auxiliary behavior sequences, leveraging global and personalized graphs to learn behavior transitions and user intentions.
[0201] BA-GNN considers the impact of different interactive behaviors and constructs directed graphs for the sequences of two interactive behaviors. Finally, a gating mechanism is used to aggregate the representations of different behavior sequences.
[0202] 4.2.4 Parameter settings
[0203] The parameter settings of MHCL are shown in Table 2, which gives the value settings of the main parameters such as vector embedding dimension, training batch size, maximum length of session sequence, learning rate, number of convolution layers, etc. Among them, the learning rate (lr), number of convolution layers (L g and L s ), μ and δ are determined by grid search to find their optimal values.
[0204] For the baseline, in order to make a fair comparison, the session sequence length is generally set to 50. Since BA-GNN performs poorly when the session length is 50, this paper adjusts its length through grid search and selects the optimal length. Specifically, it is set to 3 on JD-Computer and JD-Application, and to 5 on Taobao. For other hyperparameters of the baseline, this chapter refers to the best parameter settings reported in the original paper of the baseline, and then fine-tunes the hyperparameters of all baselines to ensure that the baseline achieves the best performance.
[0205] Table 2MHCL parameter settings
[0206]
[0207] 4.3 Experimental Results Analysis
[0208] 4.3.1 Overall Performance (Question 1)
[0209] Table 3 shows the overall performance of all models on the three datasets, where the best results are in bold and the second best results are underlined. It can be seen that the baselines show certain differences on different datasets. Compared with other baselines, BA-GNN and GPG4HSR both introduced auxiliary behaviors and achieved better results. Therefore, it can be concluded that auxiliary behaviors contain richer user preference information, and the reasonable introduction of auxiliary behaviors can improve recommendation performance. The results of GPG4HSR are generally better than BA-GNN. This is because GPG4HSR models the correlation between items in different behaviors, retains the position information between items in different behaviors, and is more in line with the realistic perspective modeling, so that user preferences can be extracted more accurately.
[0210] Table 3 Overall performance comparison of MHCL and all baseline models
[0211]
[0212] S 2-DHCN achieved better results among the baselines on the Taobao dataset, and also showed good results on JD-Computer and JD-Application. This is due to the advantages brought by hypergraph modeling. The high-order correlation of items extracted by hypergraph can better express the relationship between items and is more realistic. In addition, the performance of BA-GNN and GPG4HSR on Taobao is worse than that of the other two datasets. The reason is that the auxiliary behaviors in the Taobao dataset contain more noise information, which leads to a certain deviation in the extracted user preferences (we define the noise ratio of each session as the ratio of the number of items in the auxiliary behavior sequence whose categories are different from the categories of the true target items to the total number of auxiliary behavior items in this session. When there is no auxiliary behavior item in the session, the noise ratio of this session is defined as 0, and then the noise ratio of all sessions is averaged to obtain the noise ratio of this dataset. The larger the noise ratio, the more noise information in the auxiliary behavior. Therefore, the noise ratios in the auxiliary behaviors on the JD-Computer, JD-Application and Taobao datasets are calculated to be 0.4826, 0.4016 and 0.9056 respectively. It can be seen that the auxiliary behaviors in the Taobao dataset contain more noise information). CLHHN also shows good results in the baseline, which is because CLHHN introduces item attribute information and models user preferences in a more fine-grained manner. This shows that in session recommendation, due to the sparse nature of session data, introducing additional auxiliary information is helpful to alleviate the sparsity problem of data and can effectively improve recommendation performance. In addition, S 2 -DHCN and COTREC show performance advantages over the baselines, demonstrating that contrastive learning is effective in enhancing user preference representation.
[0213] MHCL showed the best performance in all indicators on all datasets, and the reasons are as follows. (1) Auxiliary behavior information was introduced to extract the user preference information hidden in the auxiliary behavior, and a novel local session representation module was designed to capture the high-order correlation between items in different behaviors in the current session. (2) Hypergraph modeling of high-order correlation of items is adopted, which is more in line with reality. (3) Contrastive learning is used in both global item embedding learning and global session representation learning, and effective contrast signals are designed to enhance user preference representation. (4) In the local session representation module, personalized weights are set for items of different behaviors during hypergraph convolution, and a soft attention mechanism with two different query functions is used during local item embedding aggregation, which filters out the noise information in the auxiliary behavior to a certain extent. Therefore, MHCL can provide more accurate personalized recommendations.
[0214] 4.3.2 Ablation experiment (Question 2)
[0215] In order to explore the necessity of key components in MHCL and their impact on the overall performance, four variants of MHCL are designed in this section, and ablation experiments are performed on three datasets. The four variants are as follows:
[0216] (1)MHCL NML : Multi-behavior line graphs and their corresponding contrastive learning modules are not used.
[0217] (2) MHCL NICL : The contrastive learning module is not used in item embedding learning. The input of subsequent components adopts the global item embedding based on the target behavior obtained by learning the global hypergraph of the target behavior.
[0218] (3)MHCL NLH : The local session heterogeneous hypergraph in the local session representation module is replaced by modeling each behavior sequence separately, that is, the target behavior sequence and the auxiliary behavior sequence are mean pooled respectively, and then gated aggregation is used to generate the local session representation.
[0219] (4)MHCL NG : Only local session representation is used, that is, the global session representation is removed when calculating the candidate item score.
[0220] Table 4 shows the results of the ablation experiment. It can be seen that removing any key component will reduce the performance of the model. Only MHCL NICL There is a slight improvement in N@10 and N@20 on JD-Application. It is speculated that this is because the global item embedding based on auxiliary behavior has learned a certain amount of noise, which has a slight negative impact on the final result through comparative learning. NML Comparison shows that this paper uses novel multi-behavior line graphs as contrast signals for contrastive learning, providing another angle for generating global session representations. The global session representation enhanced by contrastive learning can significantly improve the recommendation performance. NICL Compared with MHCL, it can be seen that using contrastive learning can extract richer item information from auxiliary behaviors, which verifies that the introduction of auxiliary behaviors can enhance the recommendation performance. NLH Compared with MHCL, the effectiveness of using heterogeneous hypergraphs to model the current session is verified, which retains the relative positions of items of different behaviors in the original session and models the high-order correlation between items of different behaviors, which is better than modeling the target behavior sequence and auxiliary behavior sequence separately. NG In comparison, it is verified that modeling with all sessions can obtain richer information from other sessions and extract more accurate user intentions.
[0221] Table 4 Comparison of MHCL ablation experimental results
[0222]
[0223]
[0224] Obviously, the specific implementation of the present invention is not limited to the above-mentioned methods. As long as various non-substantial improvements are made using the method concept and technical solution of the present invention, they are all within the protection scope of the present invention.
Claims
1. Hypergraph comparative learning conversation recommendation method based on multiple behaviors, Features: The steps include: S1. Construct a global hypergraph based on the target behavior sequence and auxiliary behavior sequence of all sessions respectively, combine the two global hypergraphs to perform multi-layer hypergraph convolution operations on the initial item embedding, learn the global item embedding based on the target behavior and the global item embedding based on the auxiliary behavior respectively, and use contrastive learning to enhance the global item embedding based on the target behavior from the global item embedding based on the auxiliary behavior, and use the enhanced global item embedding based on the target behavior as the input of the subsequent components; S2, learn local session representation and global session representation respectively: construct a local session heterogeneous hypergraph based on the current session, perform multi-layer heterogeneous hypergraph convolution operations on the local session heterogeneous hypergraph, and then generate local session representation through the readout function. Combined with the enhanced global item embedding, and based on the target behavior sequence and auxiliary behavior sequence in all sessions, use the soft attention mechanism with time information to obtain the global session representation based on the target behavior and the global session representation based on the auxiliary behavior, and then generate the global session representation through the mean pooling operation; S3. Construct a multi-behavior line graph for all sessions in the current batch, use the local session representation as the initial embedding of the nodes in the multi-behavior line graph, obtain the line graph session representation through multi-layer line graph convolution operations, and compare it with the global session representation as a self-supervisory signal to enhance the global session representation; S4, respectively, the enhanced global session representation, the local session representation and the enhanced global project embedding are subjected to similarity calculation and aggregation operations through the prediction layer, so as to obtain the predicted interaction probability of each project; S5. Recommend items based on the predicted interaction probability of each item.
2. The method for recommending conversations based on hypergraph comparative learning based on multiple behaviors as claimed in claim 1, Features: The initial item embedding consists of encoding the input item ID to obtain each item v i The one-hot encoding representation of i , using the learnable embedding matrix W 1 Convert it into a low-dimensional dense vector representation, i.e., item v i The initial embedding h i ,h i It is expressed as: in, W 1 The transpose of .
3. The method for recommending conversations based on hypergraph comparative learning based on multiple behaviors as claimed in claim 1, Features: The method also includes constructing a global hypergraph based on target behaviors using target behavior sequences of all sessions when learning the global item embedding; Then perform L on this global hypergraph g Layer hypergraph convolution operation to obtain global item embedding based on target behavior; Construct a global hypergraph based on auxiliary behaviors using auxiliary behavior sequences of all sessions; Then L is performed on the global hypergraph based on the auxiliary behavior g Layer hypergraph convolution operation to obtain global item embedding based on auxiliary behaviors; The two learned global item embeddings are used as contrast signals to each other, and contrastive learning is used so that they can acquire information from each other, thereby obtaining an enhanced global item embedding.
4. The method for recommending conversations based on hypergraph comparative learning based on multiple behaviors as claimed in claim 3, Features: Constructing a global hypergraph based on target behavior includes: The global hypergraph based on target behavior is defined as All items in V are nodes of the global hypergraph, is a hyperedge set, a session set The number of sessions in is the number of hyperedges. One session corresponds to one hyperedge. Any hyperedge in Usually contains multiple nodes, each of which is a session s j For each target behavior item contained in (j=1, 2, ..., J), for the hypergraph The matrix P is used to represent its adjacency relationship, where the element in the i-th row and j-th column is p ij , if the super edge Contains Project v i , then p ij is 1 if the value is true, otherwise it is 0.
5. The method for recommending conversations based on hypergraph comparative learning based on multiple behaviors as claimed in claim 3, Features: The global hypergraph convolution operation based on the target behavior includes: The global hypergraph convolution is the process of information propagation along the node-hyperedge-node. The specific convolution operation is defined as: in, is the lth (l=1, 2, ..., L g ) Layer Project v i Embedded, is the l-1th layer project v k Embedding That is, project v i The initial embedding of Each project v i The embedding needs to be done by L g The global hypergraph convolution is performed on each layer, and the embedding obtained in each layer and the initial embedding of the item are averaged and pooled to obtain the global item embedding based on the target behavior. Here’s how: Construct a global hypergraph for auxiliary action sequences of all sessions Then, after the global hypergraph convolution operation, the project v is learned from the global hypergraph based on the auxiliary behavior i Global item embedding based on auxiliary behavior 6. The method for recommending conversations based on hypergraph comparative learning based on multiple behaviors as claimed in claim 3, Features: In the recommendation method, contrastive learning for global item embeddings involves learning embeddings of the same item v from different hypergraphs. i Different global embeddings of and Use contrastive learning to complement each other and embed the global and Each is regarded as the true value label of the other, and the contrastive loss function is used for contrastive learning tasks. The standard binary cross entropy loss between positive and negative samples is used as the target of contrastive learning, as shown in formula (4): in, is a matrix The negative samples obtained by row and column transformation are Is an enhanced global project embedding for all projects The matrix is composed of i as the row index, f(·) is a discriminator function used to score the consistency of the two embeddings of the input, and σ(·) is the Sigmoid function.
7. The method for recommending conversations based on hypergraph comparative learning based on multiple behaviors as described in any one of claims 1 to 6, Features: The local conversation representation in the recommendation method includes: (1) Constructing a local session heterogeneous hypergraph includes constructing a local session heterogeneous hypergraph based on the current session and defining the local session heterogeneous hypergraph as G loc =(B, V loc , E loc ), V loc is a node set, that is, a set of items in the current session s. Each node corresponds to an item. loc is a set of hyperedges, and the number of hyperedges is J loc It is determined by the number of target behavior items in s, that is, Each target behavior item is constructed as a hyperedge with an auxiliary behavior item whose timestamp is less than that of this item and greater than that of the previous target behavior item, and is constructed as a hyperedge with an auxiliary behavior item whose timestamp is greater than that of this item and less than that of the next target behavior item. In this way, the target behavior item and its preceding and succeeding auxiliary behavior items are included in two different hyperedges, and the auxiliary behavior item and its preceding and succeeding target behavior items are also included in two different hyperedges, so as to retain the relative position between the target behavior item and the auxiliary behavior item; all target behavior items are constructed as a hyperedge, and the local session heterogeneous hypergraph uses the matrix Indicates that if the project Belongs to the hyperedge e loc:j , that is, the τth timestamp interaction behavior in s is Project In the super edge loc:j Then the element q in the matrix Q kjτ =1, otherwise q kjτ =0; hyperedge e loc:j It is Figure G loc The first target behavior item in s and the previous auxiliary behavior item constitute the first hyperedge, and the group of auxiliary behavior items that follow it constitute the second hyperedge, and so on. All target behavior items constitute the Jth hyperedge. loc Super edge; (2) Local item embedding aggregation: Local item embedding aggregation is to extract user preferences from the local item embeddings of session s. The local item embeddings are aggregated using a soft attention mechanism. The soft attention mechanism uses two different query vectors, one for each hyperedge containing all target behavior items in the current session. and a hyperedge containing the last target behavior item in the current session and its subsequent auxiliary behavior items Based on these two different query vectors, different session representations are generated after the soft attention mechanism.
8. The method for conversation recommendation based on multi-behavior hypergraph comparative learning according to any one of claims 1 to 6, Features: In the recommendation method, the enhanced global item embedding is combined with the position information of the items in the two types of behavior sequences in the current session, and the soft attention mechanism is used to obtain the global session representation based on the target behavior sequence and the global session representation based on the auxiliary behavior sequence. Then, the mean pooling operation is used to fuse the preferences of the two angles for the global session representation under the two behaviors to generate the global session representation. A multi-behavior line graph is constructed based on the global hypergraph, and a line graph session representation is generated through convolution operations on the multi-behavior line graph. Finally, the global session representation and the line graph session representation are used as each other's true value labels for comparative learning to maximize the similarity between the global session representation and the line graph session representation representing the same session, so as to enhance the global session representation. The generation of the global session representation includes: for the target behavior sequence in the current session s The enhanced global item embedding for each item in ascending order of timestamp is for Middle The enhanced global item embedding of items is then used with a learnable position embedding matrix for Add a reverse position embedding to each item in the , as follows: Among them, W m ∈R 2d×d is a trainable parameter matrix, for The item embedding after adding the position information of the i-th item in ; Subsequently, the item embeddings with added position information are aggregated using a soft attention mechanism to obtain a global session representation based on the target behavior of session s. Among them, η i is the attention coefficient of the item at the corresponding position, Among them, σ(·) is the Sigmoid function, and (both∈R d×d ) is a trainable parameter matrix, is the mean of all item embeddings after adding position information, that is, Query vector as soft attention mechanism; For auxiliary behavior sequences, the corresponding auxiliary behavior-based global session representation can be generated by aggregating the enhanced global item embeddings of the items in the auxiliary behavior sequence Finally, the global session representations based on the target behavior and auxiliary behaviors are aggregated using a mean pooling operation to generate a comprehensive global session representation z for session s. g(s) :
9. The method for recommending conversations based on hypergraph comparative learning based on multiple behaviors as claimed in claim 8, Features: Conversational contrastive learning includes: (1) Construction of multi-behavior line graph: Multi-behavior line graph G lim It is a heterogeneous ordinary graph constructed by extracting key information from the global hypergraph. Each node in the graph represents a session, and the edge represents the connectivity information between sessions. Undirected edges are used here. lin In the example, two types of edges are set according to different types of interactive behaviors. Each type of edge is extracted from the global hypergraph of the corresponding behavior. Each hyperedge in the graph G lin If a node in If two hyperedges have common nodes, then G lin There is an edge connection between the corresponding two nodes in the graph; the multi-behavior line graph can be formally defined as G lin =(V lin , E lin ), where the node set Edge Set Among them, k∈{1,2}, represents the kth type of edge in a multi-line graph, and Based on b k The set of hyperedges of the global hypergraph of the behavior and the i-th hyperedge in it, is the number of common nodes contained in the two hyperedges; for each edge Set a weight Used to indicate the degree of association between two nodes in a multi-behavior line graph, where is the total number of nodes contained in the two hyperedges; (2) Multi-behavior graph convolution: The multi-behavior graph contains the connectivity information between sessions in the global hypergraph. If there is an edge connection between nodes, the nodes are defined as neighbors of each other. The multi-behavior graph convolution is the process of aggregating the neighbor information of the node to the node. According to the constructed multi-behavior graph G lin , define the matrix in is the number of nodes, that is, the number of sessions in the current batch during training, and its elements are the weights a of each edge in the line graph kij ; Add self-connection edges to each node in the line graph, and define the association matrix ( and k = 1, 2) = A k +I (I is the identity matrix, the purpose is to add the self-connection edge of the node; A k is a matrix with index k in the first dimension of A); define the degree matrix ( and k = 1, 2) is a diagonal matrix, Elements in yes The elements in ; the multi-line graph convolution is defined as: Among them, θ (l) ,θ (l-1) They are the lth (l=1, 2, ..., L g ) layer and the multi-line graph node embedding of the l-1th layer, and Respectively represent the node's neighbor information from the target behavior type edge and the auxiliary behavior type edge aggregation to the node, and then use Normalize, (·) -1 represents the inverse operation of the matrix, μ is a hyperparameter used to control the degree to which the neighbors of the node in the line graph aggregate information from the edges of the auxiliary behavior type to the node. In the multi-behavior line graph, each node represents a session. The session corresponding to each node is taken as the current session, and the corresponding local session representation is obtained as the initial embedding of the corresponding node (i.e., θ (0) =z s ), through multi-layer convolution operations on multi-behavior line graphs, the information in the local session representation is enhanced to the line graph session representation, and further enhanced to the global session representation through contrastive learning operations; After the above convolution process, the nodes of each layer are embedded into θ (l) Take the average to get the line graph session representation corresponding to the current session s (3) Create self-supervisory signals: The global session representation and the line graph session representation obtained by learning multi-behavior line graphs are used as each other's true value labels, and the information of each other is supplemented through comparative learning; in the sessions trained in the same batch, the global session representation and the line graph session representation representing the same session are regarded as positively correlated samples, otherwise they are marked as negatively correlated samples; specifically, if the session s j In the current batch training session, use and Represent their global session representation and line graph session representation respectively, then and For positively correlated samples, perform row and column transformations on the matrix composed of the global session representation (or line graph session representation) of all sessions in the current training batch, and take session s j The vector at the original index position is obtained (or ) (or ), for which the standard binary cross entropy loss between positive and negative samples is used as the learning objective and is defined as: Where σ(·) is the Sigmoid function and f(·): is a discriminator function.
10. The method for conversation recommendation based on multi-behavior hypergraph comparative learning according to any one of claims 1 to 7, Features: In the recommendation method, the prediction layer outputs the predicted probability of each interactive item. Given the current session s, use the corresponding local session representation z s and the enhanced global session representation z g(s) Respectively with project v i Enhanced global project embedding Do the inner product, calculate the similarity between the session representation and the enhanced global item embedding, and then aggregate to get each candidate item v i Score The specific method is as follows: Among them, the LeakyRelu(·) function with a hyperparameter γ is used to introduce nonlinear transformation. are the corresponding transposes respectively; After that, the softmax function is used to calculate the probability of all items becoming the next target behavior item in the current session: Among them, the vector
Citation Information
Cited By
Personalized learning content recommendation method and system based on neighborhood and hypergraph collaboration
CN120386941A
Personalized learning content recommendation method and system based on neighborhood and hypergraph collaboration
CN120386941B
Recommendation method and device, equipment, storage medium and program product
CN121579796A
