A conversation recommendation method, system and device based on self-supervised graph learning
By employing a self-supervised graph learning method, and utilizing graph attention networks and global graph information to optimize item representations, the problems of transition smoothing and overfitting in conversation recommendation are solved, thereby improving recommendation accuracy.
Patent Information
- Application Number
- CN202210510720.7
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2022-05-11
- Publication Date
- 2024-11-15
- Estimated Expiration
- 2042-05-11
AI Technical Summary
Existing conversation recommendation methods suffer from transition smoothing and overfitting issues, leading to a decrease in recommendation accuracy.
We employ a self-supervised graph learning-based approach, using a graph attention network to propagate information and generate item representations. We then utilize the connectivity and disconnection information in the global graph to calculate the self-supervised loss and combine it with target adaptive masking to optimize item recommendation.
It effectively solves the problems of model transition smoothing and overfitting, improves the accuracy of recommendations, and experimental results show that it outperforms existing benchmark models in terms of Recall and MRR evaluation metrics.
Smart Images

Figure CN114969561B_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the technical field of recommendation systems, and in particular to a conversation recommendation method, system and device based on self-supervised graph learning. Background Art
[0002] With the rapid growth of Internet information, recommendation systems, as an effective way to solve the problem of information overload, can help people make accurate choices when faced with a wealth of products or services. Traditional recommendation methods, such as collaborative filtering recommendations, focus on users' long-term static preferences and ignore their short-term interactive behaviors, which may lead to inaccurate recommendations. In addition, in some real-life scenarios, it is difficult to obtain users' long-term historical interactive behaviors. Therefore, a conversational recommendation system was proposed to generate recommendations by modeling users' limited interactive behaviors in the short term.
[0003] Traditional conversational recommendation methods can use Markov Chain (MC) to capture the sequential dependencies between adjacent interactions. Considering the advantages of Recurrent Neural Network (RNN) in sequential data modeling, some people propose to use Gated Recurrent Unit (GRU) to model conversational sequences, and introduce batch parallel training and other measures to optimize GRU to make it more suitable for conversational recommendation tasks. In addition, some people use attention mechanism to obtain the importance of items, which can be used to generate the main intention of users. In recent years, thanks to the powerful performance of Graph Neural Network (GNN) in modeling the transfer relationship between items, conversational recommendation methods using GNN have been widely used and achieved the most advanced performance in the field of conversational recommendation. For example, some people proposed the GCE-GNN model, which specifically generates item representations by aggregating information from local and global graphs through soft attention mechanism.
[0004] However, it has been proven that conversational recommendation methods using Markov chains and recurrent neural networks do not fully consider the complex transmission relationship between items, and the current conversational recommendation methods based on graph neural networks also have the following limitations: On the one hand, deep graph neural networks may have the problem of over-smoothing, that is, the representations of adjacent items learned by graph neural networks in a session are highly similar and cannot be accurately distinguished, resulting in inaccurate recommendations; On the other hand, the training goal of the model is to minimize the cross entropy loss. During the training process, the score of the target item continues to increase, and the scores of other items including its neighboring nodes as negative samples continue to decrease. However, in theory, closely connected items in the graph are similar. Therefore, it is unreasonable that the scores of the target item's neighboring items and other items unrelated to the target item are almost zero, which will cause the model to overfit and thus reduce model performance. Summary of the invention
[0005] Based on this, the present invention provides a conversation recommendation method, system and device based on self-supervised graph learning to solve the problem of decreased recommendation accuracy caused by transition smoothing and overfitting problems in existing conversation recommendation methods.
[0006] Based on the above purpose, the present invention provides a conversation recommendation method based on self-supervised graph learning, comprising:
[0007] Obtaining a current session from the session set, learning the representation of each item in the current session through information propagation using a graph attention network, and obtaining a session representation by generating a user's long-term interest and short-term interest;
[0008] Establishing a global graph showing connections between items in different sessions according to the session set, obtaining connection information and non-connection information between items from the global graph to generate a self-supervisory signal, and calculating a self-supervisory loss according to the self-supervisory signal;
[0009] Obtain a candidate set including m candidate items, and obtain a predicted recommendation probability of each of the candidate items using the session representation;
[0010] Performing target adaptive masking processing on the candidate set to obtain a new candidate set, and calculating the main supervision loss based on the new candidate set;
[0011] Obtaining a comprehensive loss according to the self-supervised loss and the main supervised loss, and updating the network parameters of the item representation and the graph attention network by the comprehensive loss to optimize the predicted recommendation probability of each of the candidate items;
[0012] When the comprehensive loss is less than a preset value, the recommendation optimization is terminated, and the optimal recommendation probability of each candidate item is obtained.
[0013] Preferably, the acquiring the current session from the session set, learning the representation of each item in the current session through information propagation through a graph attention network, and obtaining the session representation by generating the user's long-term interests and short-term interests, comprises:
[0014] A local graph is constructed based on the current session. The local graph is represented as: G τ = {V τ ,E τ}, where V τ is a node set of the local graph, including all items in the current session; τ is the edge set of the local graph, including the pairwise transitive relations between items in the current session;
[0015] Inputting the local graph into a graph attention network, and obtaining an embedding vector of each node in the local graph after information propagation on the graph attention network;
[0016] After restoring the items from the local graph according to the embedding vector of each node, a sequence of item representations in the current session arranged in chronological order is obtained;
[0017] Determine the embedding representation of the last item in the item representation sequence as the user's short-term interest, and aggregate the embedding representations of the items through an attention mechanism to generate the user's long-term interest;
[0018] A session representation is generated according to the short-term interests and long-term interests of the user.
[0019] Preferably, the user's long-term interest is expressed as:
[0020]
[0021] γ i =softmax(β i ),
[0022] β i =W4σ(W5u i +W6C s ),
[0023] Among them, C l For the long-term interests of users; C s is the user's short-term interest; W4, W5 and W6 are the weight parameters of the attention mechanism; softmax() is the softmax function; σ is the sigmoid function.
[0024] Preferably, inputting the local graph into a graph attention network, and obtaining an embedding vector of each node in the local graph after information propagation on the graph attention network, comprises:
[0025] Obtain each node x in the local graph through the embedding layer of the graph attention network i The initial embedding vector of
[0026] For the node x at layer l i , obtain the node x through the self-attention mechanism in the graph attention network i Its adjacent node x j The attention coefficient between
[0027] Normalizing the attention coefficient through the softmax layer of the graph attention network to obtain a normalized coefficient;
[0028] The node x i Combine the neighbor node information of the node x and use the normalization coefficient to update the node x i The embedding vector of is obtained.
[0029] Preferably, establishing a global graph showing connections between items in different sessions according to the session set, acquiring connection information and non-connection information between items from the global graph to generate a self-supervisory signal, and calculating a self-supervisory loss according to the self-supervisory signal comprises:
[0030] Building a global graph based on the n sessions in the session set;
[0031] For each item v in the current session i , get item v on the global graph i Neighbor node v j and non-neighbor nodes Among them, the number of neighbor nodes is the same as the number of non-neighbor nodes;
[0032] The self-supervised loss is calculated by JS divergence, which is used to maximize the item v i With neighbor node v j distance, minimizing item v i With non-neighboring nodes The distance between them is specifically expressed as:
[0033]
[0034] Among them, L self is the self-supervision loss; u i For item v i The embedding representation of u j is the neighbor node v j The corresponding embedding representation; Non-neighbor node The corresponding embedding representation; T is the number of neighbor nodes or non-neighbor nodes; f() is the similarity evaluation function, and the similarity evaluation function f() is expressed as:
[0035] f(u i ,u j )=u i T u j ,
[0036] Among them, u i T For item v i The transpose of the embedding representation of .
[0037] Preferably, obtaining a candidate set including m candidate items, and obtaining a predicted recommendation probability of each candidate item using the session representation, includes:
[0038] Get each candidate item v in the candidate set i Embedding representation of ;
[0039] For each candidate item v i The embedding representation and session representation of are input into the preset recommendation model to obtain each candidate item v output by the recommendation model i The predicted recommendation probability of ; the recommendation model is:
[0040]
[0041] in, is the candidate item v i The probability of being recommended to the user for interaction at the next timestamp; C T is the transpose of the session representation; u i is the candidate item v i Embedding representation of ; m is the number of candidate items in the candidate set.
[0042] Preferably, performing target adaptive masking processing on the candidate set to obtain a new candidate set, and calculating the primary supervision loss according to the new candidate set, comprises:
[0043] Get the target item v a , and sample from the global graph to obtain the target item v a The most similar t neighbors generate a neighbor set;
[0044] After shielding t neighbors from the candidate set, a new candidate set is obtained; the new candidate set is expressed as:
[0045] M*=M\N(v a ),
[0046] Among them, M* is the new candidate set; M is the original candidate set; N(v a ) is the target item v a The neighbor set of ; “\” is the subtraction operation between sets;
[0047] According to the new candidate set, the main supervision loss is calculated by the cross entropy loss function, which is specifically expressed as:
[0048]
[0049] Among them, L main Mainly supervise the loss; i , For each candidate item v in the new candidate set M* i The true predicted recommendation probability and the predicted predicted recommendation probability.
[0050] Based on the same inventive concept, the present invention also provides a conversation recommendation system based on self-supervised graph learning, comprising:
[0051] A session set acquisition module is used to acquire a session set, which includes n sessions;
[0052] A session representation module, used to obtain a current session from the session set, learn the representation of each item in the current session through information propagation through a graph attention network, and obtain a session representation by generating a user's long-term interest and short-term interest;
[0053] A self-supervised learning module, configured to establish a global graph showing connections between items in different sessions based on the session set, obtain connection information and non-connection information between items from the global graph to generate a self-supervised signal, and calculate a self-supervised loss based on the self-supervised signal;
[0054] A recommendation prediction module, used to obtain a candidate set including m candidate items, and obtain a predicted recommendation probability of each candidate item using the session representation;
[0055] A shielding module is used to perform target adaptive shielding processing on the candidate set to obtain a new candidate set, and calculate the main supervision loss according to the new candidate set;
[0056] A recommendation optimization module is used to obtain a comprehensive loss based on the self-supervised loss and the main supervised loss, and to update the network parameters of the item representation and the graph attention network through the comprehensive loss to optimize the predicted recommendation probability of each of the candidate items; when the comprehensive loss is less than a preset value, the recommendation optimization is terminated, and the optimal recommendation probability of each of the candidate items is obtained.
[0057] Preferably, the session representation module comprises:
[0058] The local graph construction submodule is used to construct a local graph according to the current session. The local graph is represented as: G τ = {V τ ,E τ}, where V τ is a node set of the local graph, including all items in the current session; τ is the edge set of the local graph, including the pairwise transitive relations between items in the current session;
[0059] A representation learning submodule, used for inputting the local graph into a graph attention network, and obtaining an embedding vector of each node in the local graph after information propagation on the graph attention network;
[0060] An item representation submodule, configured to recover items from the local graph according to the embedding vector of each node, and obtain a chronologically ordered sequence of item representations in the current session;
[0061] A preference generation submodule, used to determine the embedding representation of the last item in the item representation sequence as the user's short-term interest, and to generate the user's long-term interest by aggregating the embedding representations of the items through an attention mechanism;
[0062] The session representation submodule is used to generate a session representation according to the short-term interests and long-term interests of the user.
[0063] Based on the same inventive concept, the present invention also provides an electronic device, comprising a memory, a processor, and a computer program stored on the memory and executable on the processor, wherein when the processor executes the computer program, the conversation recommendation method based on self-supervised graph learning as described in any one of the above items is implemented.
[0064] As can be seen from the above, the conversation recommendation method, system and device based on self-supervised graph learning of the present invention first obtains the representation of each item in the current conversation through information propagation through the graph attention network for the current conversation given in the conversation set, and obtains the conversation representation through the generated short-term interest and long-term interest, then builds a global graph based on all the conversations in the conversation set, obtains the connection information and non-connection information of the items from the global graph, so as to enhance the learning of the item representation in the conversation and calculate the self-supervised loss, then uses the conversation representation and the embedded representation of each candidate item in the candidate set to predict the recommendation probability of the candidate item, and calculates the main supervision loss after adaptively shielding the candidate set, and finally optimizes the item recommendation process through back propagation based on the comprehensive loss obtained by the self-supervised loss and the main supervision loss. Compared with the existing conversation recommendation method, the conversation recommendation method based on self-supervised graph learning of the present invention enhances the learning of the item representation in the conversation by mining the item connection information between all conversations from the global graph through self-supervised learning, effectively solves the problem of model transition smoothness, and at the same time, effectively overcomes the overfitting problem caused by the cross entropy loss through target adaptive shielding, and achieves the purpose of optimizing item recommendation and effectively improving the accuracy of recommendation. In addition, experimental results on two public datasets show that the session recommendation method based on self-supervised graph learning outperforms the current best baseline model in both Recall and MRR evaluation indicators. BRIEF DESCRIPTION OF THE DRAWINGS
[0065] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.
[0066] Figure 1 A flowchart of a conversation recommendation method based on self-supervised graph learning in one embodiment of the present invention;
[0067] Figure 2 Schematic diagram of the process of step S20 of the conversation recommendation method based on self-supervised graph learning in one embodiment of the present invention;
[0068] Figure 3 Schematic diagram of the structure of a conversation recommendation system based on self-supervised graph learning in one embodiment of the present invention;
[0069] Figure 4 A schematic diagram of the structure of a conversation representation module of a conversation recommendation system based on self-supervised graph learning in one embodiment of the present invention;
[0070] Figure 5 FIG. 4 is a schematic diagram of an electronic device according to an embodiment of the present invention. DETAILED DESCRIPTION
[0071] In order to make the technical problems, technical solutions and beneficial effects to be solved by the present invention more clearly understood, the present invention is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not used to limit the present invention.
[0072] like Figure 1 As shown, a conversation recommendation method based on self-supervised graph learning provided by an embodiment of the present invention specifically includes the following steps:
[0073] Step S10, obtaining a session set, the session set including n sessions.
[0074] In this embodiment, all the user's conversations within a preset time period can be collected and combined to construct a conversation set U = {S1, S2, ..., S n}, and for any session S in the session set U i ={v1,v2,…v t ,…,v p}, can contain p items in chronological order, v t Indicates session S i The interaction at timestamp t.
[0075] Step S20, obtain the current session from the session set, learn the representation of each item in the current session through information propagation through the Graph Attention Network (GAT), and obtain the session representation by generating the user's long-term interests and short-term interests.
[0076] In this embodiment, any session can be randomly selected from the session set U as the current session S τ , and then construct a directed local graph G τ = {V τ ,E τ} to indicate the current session S τ Next, the local graph G τ Input to GAT and obtain the local graph G τ The output embedding representation of each item in the current session (i.e., item representation) is finally used to generate the user’s short-term interest C through the preset attention mechanism. s and long-term interests C l As user preference information, a conversation representation C is generated based on the user preference information.
[0077] As a preference, Figure 2 As shown, step S20 includes the following steps:
[0078] Step S201, based on the current session S τ Construct a local graph, which is represented by: G τ = {V τ ,E τ}, where V τ is the local graph G τ A collection of nodes, including the current session S τ All items in E τ is the local graph G τ The edge set of the current session S τ The pairwise transfer relationship between items in .
[0079] In step S201, the node set V τ ={x1,x2,…,x q}Contains the current session S τ All unique items in , and q≤p, the edge set E τ Any edge e in i,j Indicates that the user is in the current session S τ In node x i Then for node x j to interact.
[0080] Step S202: local graph G τ Input the graph attention network, and obtain the local graph G after information propagation on the graph attention network τ For each node x i The embedding vector z i .
[0081] Specifically, first obtain the local graph G τ For each node x i The initial embedding vector z i , and then use the self-attention mechanism of GAT to aggregate information from neighbor nodes and dynamically determine the local graph G τ The importance of different neighbor nodes in the final node x i Update node x with the neighbor node information and corresponding importance i The embedding vector z i , and get the new embedding vector.
[0082] Step S203: According to each node x i The embedding vector z i , from the local graph G τ After restoring the items in the current session, get the current session S a Items arranged in chronological order represent a sequence.
[0083] Specifically, after obtaining the local graph G τFor each node x i The embedding vector z i Afterwards, according to the correspondence between nodes and items, from the local graph G τ Restore the items in chronological order and get the current session S τ The item representation sequence {u1,u2,…,u q}, which represents the sequence containing each node x i The corresponding item embedding representation u i .
[0084] Step S204: embed the last item in the item representation sequence u q Determined as the user's short-term interest C s , and the embedding representation u of the aggregated items is obtained through the attention mechanism i Generate user's long-term interest C l .
[0085] Understandable, current session S τ The last item in the interaction can reflect the user’s current interest, and the embedding representation u of the last item can be directly used. q As the user's short-term interest C s ∈R d (d is the embedding dimension), that is, C s =u q In addition, the embedding representation u of the aggregated items is obtained through the attention mechanism i (i={1,2,…,q}) to generate the user’s long-term interest C l Preferably, the user's long-term interest C l It can be expressed as:
[0086]
[0087] γ i =softmax(β i ),
[0088] β i =W4σ(W5u i +W6C s ) (1)
[0089] Among them, q is the current session S τ The number of items or nodes; W4, W5 and W6 are the weight parameters of the attention mechanism, and W4∈R d ,W5∈R d×d , W6∈R d×d ; softmax() is the softmax function; σ is the sigmoid function.
[0090] Step S205: according to the user's short-term interest C s and long-term interests C l Generate a session representation C.
[0091] Specifically, for short-term interests C s and long-term interests C l After concatenation, we get the session representation C, which is specifically expressed as:
[0092] C=W7*[Cs;C l ] (2)
[0093] Among them, [;] is the concatenation operation; W7 is the weight parameter, and W7∈R d×d .
[0094] In an optional embodiment, step S202 includes the following steps:
[0095] Step S2021, obtain the local graph G through the embedding layer of the graph attention network τ For each node x i The initial embedding vector The initial embedding vector It can be expressed as:
[0096]
[0097] Among them, Embedding is the embedding operation. For node x i The initial embedding vector of d is the dimension of the embedding.
[0098] Step S2022: for each node x in the lth layer i , obtain node x through the self-attention mechanism in the graph attention network i Its adjacent node x j The attention coefficient e ij , which can be expressed as:
[0099]
[0100] Among them, W0, W1, W2 are the network parameters of the graph attention network, and W0∈R 2d ,W1∈R d×d , W2∈R d×d ; [;] is the concatenation operation; δ is the LeakyRelu function.
[0101] Step S2023, the attention coefficient e is paid attention to by the softmax layer of the graph attention network ij Perform normalization processing to obtain a normalization coefficient; the normalization coefficient can be expressed as:
[0102]
[0103] Where N(i) is the node x i In the local graph G τ The set of neighbors in .
[0104] Step S2024: node x i Combine the neighbor node information of x and use the normalization coefficient to update the node x i The embedding vector of The new embedding vector It can be expressed as:
[0105]
[0106] Among them, W3 is the network parameter of the graph attention network, and W3∈R d×d ;σ is the sigmoid function; is the node x at the l-1th layer j The embedding vector of .
[0107] Step S30, establishing a global graph showing the connection between items in different sessions based on the session set, obtaining connection information and non-connection information between items from the global graph to generate a self-supervisory signal, and calculating a self-supervisory loss based on the self-supervisory signal.
[0108] Understandably, existing conversational recommendation methods usually ignore the transition smoothness problem of graph neural networks, which makes the learned item representations difficult to distinguish, thereby reducing the accuracy of recommendations. In this embodiment, self-supervised learning is used to mine the connection and disconnection information of items from the global graph to enhance item representation learning, which can solve the transition smoothness problem of graph neural networks and provide basic support for the recommendation optimization process by calculating the self-supervised loss.
[0109] Preferably, step S30 includes the following steps:
[0110] Step S301, construct a global graph based on n conversations in the conversation set, and the global graph is expressed as: G = {V, E}, where V is all items that appear in the n conversations, and E is all item transfer relationships in the conversation set.
[0111] In step S301, in order to capture the pairwise transfer relationship between items in different sessions, a global graph G = {V, E} is constructed using all the sessions in the session set U. For the item set V = {v1, v2, …, v |V|} contains all unique items that appear in n sessions, for any relation e in the relation set E i,j Indicates item vi and v j Is clicked adjacently in a session.
[0112] Step S302: for each item v in the current session i , get item v on the global graph G i Neighbor node v j and non-neighbor nodes Among them, the number of neighbor nodes is the same as the number of non-neighbor nodes.
[0113] In step S302, for the current session S τ Given an item v i , you can set the items v i Neighbor v on the global graph G j And the embedding representation u learned by GAT j As the corresponding connection information, non-neighbors on the global graph G And the embedding representation learned by GAT As the corresponding connection information.
[0114] Step S303, calculate the self-supervised loss L through JS divergence self , the self-supervised loss is used to maximize the item v i With neighbor node v j distance, minimizing item v i With non-neighboring nodes The distance between.
[0115] Understandably, for the current session S τ Given an item v i , the embedding representation learned by GAT and its neighbor node v j More similar to its non-neighboring nodes There is a more obvious difference. Based on this, JS divergence can be used to calculate the self-supervised loss. The self-supervised loss L self It can be expressed as:
[0116]
[0117] Among them, u i is the item v learned by GAT i The embedding representation of u j For item v i Neighbor node v j The corresponding embedding representation; Items i Non-neighbor nodes The corresponding embedding representation; T is the number of neighbor nodes or non-neighbor nodes; f() is the similarity evaluation function, whose input parameter is the embedding representation of the two items, and the output parameter is the similarity score between the two items. The similarity evaluation function f() can be expressed as:
[0118] f(u i ,u j )=u i T u j (8)
[0119] Among them, u i T For item v i The transpose of the embedding representation of .
[0120] In an optional embodiment, step S302 includes the following steps:
[0121] Step S3021, for each item v in the current session i , get item v on the global graph G i The connection set N(v i ) and the unconnected set O(v i ), and O(v i )=V\N(v i );
[0122] Step S3022: select the connection set N(v i ) to obtain the most similar T nodes and determine them as item v i The final neighbor node v j ;
[0123] Step S3023, from the unconnected set O(v i ) randomly obtains a node with the same number of neighbor nodes and determines it as item v i Final non-neighbor nodes
[0124] In this embodiment, for popular items, there may be a large number of neighboring items. In order to avoid the deviation caused by accidental clicks, the maximum sampling method can be used to select T most relevant items as each item v in the global graph G. i The final neighbors of the global graph depend on the edge weights, which can represent the item v i The corresponding target node and its neighbor node v j The degree of similarity between them. At this point, we can use the maximum sampling method to select items from v i The connection set N(v i ) to obtain the most similar T nodes as the final neighbor nodes v of the target node j, and through random sampling from the unconnected set O(v i ) to obtain T nodes as the final non-neighbor nodes of the target node Understandably, the loose connections between nodes can be filtered out by the maximum sampling method to determine the high correlation between the final neighbors and the target node.
[0125] Step S40, obtaining a candidate set including m candidate items, and obtaining a predicted recommendation probability of each candidate item using the session representation.
[0126] In this embodiment, the candidate set M = {v1, v2, ..., v m}, where the number of candidate items m in the candidate set is less than or equal to the total number of items |V| in the n sessions in the session set U, that is, m≤|V|.
[0127] After obtaining the session representation C, the session representation C is combined with each candidate item v in the candidate set. i The embedding representation u i Multiply them together to get each candidate item v i The predicted recommendation probability.
[0128] Preferably, step S40 includes the following steps:
[0129] Step S401, obtain each candidate item v in the candidate set M i The embedding representation u i ;
[0130] Step S402: each candidate item v i The embedding representation u i The session representation C is input into the preset recommendation model to obtain the output of each candidate item v by the recommendation model. i The predicted recommendation probability of ; the recommendation model is:
[0131]
[0132] in, is the candidate item v i The probability of being recommended to the user for interaction at the next timestamp; softmax() is the softmax function; C T is the transpose of the session representation; u i is the candidate item v i Embedding representation of ; m is the number of candidate items in the candidate set.
[0133] Step S50, after performing target adaptive masking processing on the candidate set, a new candidate set is obtained, and the main supervision loss is calculated based on the updated candidate set.
[0134] Understandably, for the candidate set C, directly using the cross entropy loss will lead to the problem of overfitting, that is, the target item v a The scores of neighbor items and non-neighbor items as negative samples (i.e., predicted recommendation probability) are almost zero. Therefore, in order to solve the overfitting problem caused by the indistinguishable scores, the candidate set C can be subjected to target adaptive shielding.
[0135] Preferably, step S50 includes the following steps:
[0136] Step S501, obtain the target object v a , and sample from the global graph G = {V, E} to obtain the target item v a The most similar t neighbors generate the neighbor set N(v a );
[0137] Step S502, after shielding t neighbors from the candidate set M, a new candidate set M* is obtained; the new candidate set can be expressed as:
[0138] M*=M\N(v a ) (10)
[0139] Among them, “\” is the subtraction operation between sets.
[0140] Step S503: Calculate the main supervision loss L using the cross entropy loss function based on the new candidate set M* main ; The master monitors the loss L main It can be expressed as:
[0141]
[0142] Among them, y i , For each candidate item v in the new candidate set M* i The true predicted recommendation probability and the predicted predicted recommendation probability.
[0143] Step S60, obtaining a comprehensive loss according to the self-supervision loss and the main supervision loss, and updating the network parameters of the item representation and the graph attention network through the comprehensive loss to optimize the predicted recommendation probability of each candidate item.
[0144] As a preference, the main supervision loss L is obtained by formula (11) main , and the auxiliary self-supervisory loss L is obtained by formula (7) self After that, the self-supervision loss L main and the main supervision loss L self Combined, we get the comprehensive loss L COM , which can be expressed as:
[0145] LCOM =L main +λL self (12)
[0146] Among them, λ is a trade-off parameter used to control the scale of self-supervised learning.
[0147] Furthermore, based on the comprehensive loss L COM , the back propagation algorithm is used to update the network parameters of GAT, the weight parameters of the attention mechanism, etc., so as to update the current session S τ The embedding representation of each item in the candidate set M is used to obtain a new session representation, so as to use the new session representation to optimize each candidate item v in the candidate set M i The predicted recommendation probability.
[0148] Step S70, when the comprehensive loss is less than a preset value, the recommendation optimization is terminated, and the optimal recommendation probability of each candidate item is obtained.
[0149] That is, when the comprehensive loss L is detected COM When it is less than the preset value, the recommendation optimization ends, and the updated GAT network parameters, attention mechanism weight parameters, etc. are saved, and each candidate item v in the candidate set M is obtained. i The optimal recommendation probability is obtained, and further, the top K candidate items are obtained according to the optimal recommendation probability to generate a user recommendation list to recommend products to the user.
[0150] As can be seen from the above, the conversation recommendation method based on self-supervised graph learning in this embodiment first obtains the representation of each item in the current session through information propagation through GAT for a given current session in the conversation set, and obtains the conversation representation through the generated short-term interest and long-term interest, then builds a global graph based on all the sessions in the conversation set, obtains the connection information and non-connection information of the items from the global graph, so as to enhance the learning of the item representation in the conversation and calculate the self-supervised loss, then uses the conversation representation and the embedded representation of each candidate item in the candidate set to predict the recommendation probability of the candidate item, and calculates the main supervision loss after adaptively shielding the candidate set, and finally optimizes the item recommendation process through back propagation based on the comprehensive loss obtained by the self-supervised loss and the main supervision loss. Compared with the existing conversation recommendation method, such as the conversation recommendation method based on graph neural network, the conversation recommendation method based on self-supervised graph learning in this embodiment can effectively solve the problem of model transition smoothness by using the item connection information between all sessions through self-supervised learning, and at the same time, effectively overcomes the overfitting problem caused by the cross entropy loss through target adaptive shielding, thereby achieving the purpose of optimizing item recommendation and effectively improving the recommendation accuracy.
[0151] In order to verify the effectiveness and recommendation accuracy of the conversation recommendation method based on self-supervised graph learning provided by this embodiment, this embodiment conducts experiments on two real datasets that have been made public, namely, Gowalla and Diginetica. Among them, the Gowalla dataset is a dataset that collects check-in information from February 2009 to October 2010; the Diginetica dataset is used as a challenge dataset for the 2016 CIKM Cup, which contains transaction data; both datasets are suitable for conversation recommendation. In addition, the two datasets can be preprocessed to filter out items that appear less than 5 times, and to delete conversations with a length of 1. Some statistical results of the two datasets after preprocessing are shown in Table 1.
[0152] Table 1 Statistical results of the data set
[0153]
[0154] This example compares the performance of the conversation recommendation method based on self-supervised graph learning with the following representative baseline models: 1) FPMC is a hybrid method based on personalized Markov chain to capture sequential signals; 2) NARM uses RNN to model continuous conversations and adopts attention mechanism to capture the main intention of users; 3) NextItNet uses CNN to model conversation sequences and learns high-level representations from item dependencies; 4) FGNN builds a global graph to obtain cross-session information and adopts item embedding through weighted attention layer; 5) SR-GNN converts each session into a graph and uses GGCN to obtain item representation; 6) GCE-GNN uses soft attention mechanism to aggregate information in local graph and global graph to model user preferences.
[0155] This example uses Recall@K and MRR@K to evaluate the recommendation performance of the conversational recommendation method based on self-supervised graph learning. Recall@K is an indicator for calculating the proportion of correct items in the first K positions of the recommendation list for a test case, and the calculation formula is:
[0156]
[0157] MRR@K considers the position of the target item in the list of recommended items. If the target item is not in the top K positions, it is set to 0. Otherwise, the calculation formula is:
[0158]
[0159] Among them, S test is a test case, and Rank() is the Rank function.
[0160] In this embodiment, the effectiveness and recommendation accuracy of the verification method are verified. Each dataset is divided into a training set and a test set for training and evaluation, respectively. For the Gowalla dataset, the sessions of the previous week are used as the test set; for the Diginetica dataset, the last 20% of the session data are separated as the test set. During the training process, Adam is used as the optimizer, the batch size is set to 512, the item embedding size is set to 128, the learning rate is initialized to 0.001, decayed by 0.5 every 3 epochs, and the number of recommendations K is set to 20, and the number of neighbors and non-neighbors in self-supervised learning is set to 3. In addition, this embodiment attempts to find the optimal performance parameters in dropout = {0, 0.1, 0.25} and number of layers = {1, 2, 3}. Table 2 shows the performance results of this method and 6 benchmark models.
[0161] Table 2 Performance results of this method and six benchmark models
[0162]
[0163] As shown in Table 2, this method achieves the best performance on all evaluation indicators of the two datasets, which shows that this method is effective in the conversational recommendation task. In addition, the improvement of this method in Recall@20 and MRR@20 over the best baseline is 1.40% and 4.29% on the Gowalla dataset, and 2.63% and 4.85% on the Diginetica dataset. Compared with SR-GNN and GCE-GNN, this method uses self-supervised learning to utilize the item information in other sessions to generate more accurate item representations, effectively solving the over-smoothing problem caused by GNN; secondly, through the target adaptive shielding method, it effectively avoids the overfitting problem caused by the similar scores of adjacent items and other irrelevant items during the optimization process of the target item, thereby improving the accuracy of recommendation prediction.
[0164] like Figure 3 As shown, based on the same inventive concept, corresponding to any of the above-mentioned embodiments and methods, an embodiment of the present invention further provides a conversation recommendation system based on self-supervised graph learning, including a conversation set acquisition module 110, a conversation representation module 120, a self-supervised learning module 130, a recommendation prediction module 140, a shielding module 150 and a recommendation optimization module 160, and the detailed description of each functional module is as follows:
[0165] A session set acquisition module 110 is used to acquire a session set, where the session set includes n sessions;
[0166] The session representation module 120 is used to obtain the current session from the session set, learn the representation of each item in the current session through information propagation through the graph attention network, and obtain the session representation by generating the user's long-term interests and short-term interests;
[0167] A self-supervised learning module 130, configured to establish a global graph showing connections between items in different sessions based on the session set, obtain connection information and non-connection information between items from the global graph to generate a self-supervised signal, and calculate a self-supervised loss based on the self-supervised signal;
[0168] A recommendation prediction module 140 is used to obtain a candidate set including m candidate items, and obtain a predicted recommendation probability of each candidate item using the session representation;
[0169] A masking module 150 is used to perform target adaptive masking processing on the candidate set to obtain a new candidate set, and calculate the main supervision loss based on the new candidate set;
[0170] The recommendation optimization module 160 is used to obtain a comprehensive loss based on the self-supervision loss and the main supervision loss, and to update the network parameters of the item representation and the graph attention network through the comprehensive loss to optimize the predicted recommendation probability of each candidate item; when the comprehensive loss is less than a preset value, the recommendation optimization is ended and the optimal recommendation probability of each candidate item is obtained.
[0171] In an alternative embodiment, if Figure 4 As shown, the session representation module 120 includes the following submodules, and the detailed description of each functional submodule is as follows:
[0172] The local graph construction submodule 121 is used to construct a local graph according to the current session. The local graph is represented as: τ = {V τ ,E τ}, where V τ is a node of the local graph, containing all items in the current session; E τ is the edge of the local graph, containing the pairwise transitive relations between items in the current session;
[0173] A representation learning submodule 122 is used to input the local graph into the graph attention network, and obtain the embedding vector of each node in the local graph after information propagation on the graph attention network;
[0174] The item representation submodule 123 is used to obtain a chronologically ordered sequence of item representations in the current session after recovering the items from the local graph according to the embedding vector of each node;
[0175] The preference generation submodule 124 is used to determine the embedding representation of the last item in the item representation sequence as the user's short-term interest, and to generate the user's long-term interest by aggregating the embedding representations of the items through an attention mechanism;
[0176] The session representation submodule 125 is used to generate a session representation according to the short-term interests and long-term interests of the user.
[0177] In an optional implementation, the representation learning submodule 122 includes the following units, and the detailed description of each functional unit is as follows:
[0178] Vector acquisition unit, used to obtain each node x in the local graph through the embedding layer of the graph attention network i The initial embedding vector of
[0179] Information exchange unit, used for each node x in the lth layer i , obtain node x through the self-attention mechanism in the graph attention network i Its adjacent node x j The attention coefficient between
[0180] A normalization unit is used to normalize the attention coefficient by passing it through the softmax layer of the graph attention network to obtain a normalized coefficient;
[0181] Vector update unit, used to update node x i Combine the neighbor node information of x and use the normalization coefficient to update the node x i The embedding vector of is obtained.
[0182] In an optional implementation, the self-supervised learning module 130 includes the following submodules, and the detailed description of each functional submodule is as follows:
[0183] A global graph construction submodule is used to construct a global graph based on n sessions in the session set;
[0184] The node information acquisition submodule is used to obtain the node information of each item v in the current session. i , get item v on the global graph i Neighbor node v j and non-neighbor nodes Among them, the number of neighbor nodes is the same as the number of non-neighbor nodes;
[0185] The self-supervised loss calculation submodule is used to calculate the self-supervised loss through JS divergence. The self-supervised loss is used to maximize the item v i With neighbor node v j distance, minimizing item v i With non-neighboring nodes The distance between them is specifically expressed as:
[0186]
[0187] Among them, L self is the self-supervision loss; u i For item v i The embedding representation of u j For item v i Neighbor node v j The corresponding embedding representation; Items i Non-neighbor nodes The corresponding embedding representation; T is the number of neighbor nodes or non-neighbor nodes; f() is the similarity evaluation function, and the similarity evaluation function f() is expressed as:
[0188] f(u i ,u j )=u i T u j ,
[0189] Among them, u i T For item v i The transpose of the embedding representation of .
[0190] In an optional implementation, the recommendation prediction module 140 includes the following submodules, and the detailed description of each functional submodule is as follows:
[0191] The representation acquisition submodule is used to obtain each candidate item v in the candidate set i Embedding representation of ;
[0192] The recommendation calculation submodule is used to convert each candidate item v i The embedding representation and session representation of are input into the preset recommendation model to obtain each candidate item v output by the recommendation model i The predicted recommendation probability of ; the recommendation model is:
[0193]
[0194] in, is the candidate item v i The probability of being recommended to the user for interaction at the next timestamp; softmax() is the softmax function; C T is the transpose of the session representation; u i is the candidate item v i Embedding representation of ; m is the number of candidate items in the candidate set.
[0195] In an optional implementation, the shielding module 150 includes the following submodules, and the detailed description of each functional submodule is as follows:
[0196] Neighbor acquisition submodule, used to obtain the target item v a , and sample from the global graph to obtain the target item v a The most similar t neighbors generate a neighbor set;
[0197] The shielding submodule is used to shield t neighbors from the candidate set to obtain a new candidate set; the new candidate set is expressed as:
[0198] M*=M\N(v a ),
[0199] Among them, M* is the new candidate set; M is the original candidate set; N(v a ) is the target item v a The neighbor set of ; “\” is the subtraction operation between sets;
[0200] The main supervision loss calculation submodule is used to calculate the main supervision loss based on the new candidate set through the cross entropy loss function, which is specifically expressed as:
[0201]
[0202] Among them, L main Mainly supervise the loss; i , For each candidate item v in the new candidate set M* i The true predicted recommendation probability and the predicted predicted recommendation probability.
[0203] The system of the above embodiment is used to implement the corresponding method in the above embodiment, and has the beneficial effects of the corresponding method embodiment, which will not be described in detail here.
[0204] Based on the same inventive concept, corresponding to any of the above-mentioned embodiments and methods, an embodiment of the present invention also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, the conversation recommendation method based on self-supervised graph learning described in any of the above-mentioned embodiments is implemented.
[0205] Figure 5 A more specific hardware schematic diagram of an electronic device provided by this embodiment is shown, and the device may include: a processor 100, a memory 200, an input / output interface 300, a communication interface 400, and a bus 500. The processor 100, the memory 200, the input / output interface 300, the communication interface 400, and the bus 500 realize communication connection with each other inside the device.
[0206] The processor 100 can be implemented by a general-purpose CPU (Central Processing Unit), a microprocessor, an application-specific integrated circuit (ASIC), or one or more integrated circuits, and is used to execute relevant programs to implement the technical solutions provided in the embodiments of the present invention.
[0207] The memory 200 may be implemented in the form of ROM (Read Only Memory), RAM (Random Access Memory), static storage device, dynamic storage device, etc. The memory 200 may store an operating system and other application programs. When the technical solution provided by the embodiment of the present invention is implemented by software or firmware, the relevant program code is stored in the memory 200 and is called and executed by the processor 100.
[0208] The input / output interface 300 is used to connect the input / output module to realize information input and output. The input / output module can be configured as a component in the device (not shown in the figure), or it can be externally connected to the device to provide corresponding functions. The input device may include a keyboard, a mouse, a touch screen, a microphone, various sensors, etc., and the output device may include a display, a speaker, a vibrator, an indicator light, etc.
[0209] The communication interface 400 is used to connect a communication module (not shown) to realize communication interaction between the device and other devices. The communication module can realize communication through a wired mode (such as USB, network cable, etc.) or a wireless mode (such as mobile network, WIFI, Bluetooth, etc.).
[0210] The bus 500 includes a path that transmits information between various components of the device (eg, the processor 100, the memory 200, the input / output interface 300, and the communication interface 400).
[0211] It should be noted that although the above device only shows the processor 100, the memory 200, the input / output interface 300, the communication interface 400 and the bus 500, in the specific implementation process, the device may also include other components necessary for normal operation. In addition, it can be understood by those skilled in the art that the above device may also only include the components necessary for implementing the embodiments of the present specification, without necessarily including all the components shown in the figure. It should be understood by those skilled in the art that the discussion of any of the above embodiments is only exemplary and is not intended to imply that the scope of the present invention is limited to these examples; under the idea of the present invention, the technical features in the above embodiments or different embodiments may also be combined, the steps may be implemented in any order, and there are many other changes in different aspects of the embodiments of the present invention as described above, which are not provided in detail for the sake of simplicity.
[0212] The embodiments of the present invention are intended to cover all such substitutions, modifications and variations that fall within the broad scope of the present invention. Therefore, any omissions, modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the embodiments of the present invention should be included in the protection scope of the present invention.
Claims
1. A conversation recommendation method based on self-supervised graph learning, characterized in that: include: Get a session set, which contains n sessions; Obtaining a current session from the session set, learning the representation of each item in the current session through information propagation using a graph attention network, and obtaining a session representation by generating a user's long-term interest and short-term interest; Establishing a global graph showing connections between items in different sessions according to the session set, obtaining connection information and non-connection information between items from the global graph to generate a self-supervisory signal, and calculating a self-supervisory loss according to the self-supervisory signal; Obtain a candidate set including m candidate items, and obtain a predicted recommendation probability of each of the candidate items using the session representation; Performing target adaptive masking processing on the candidate set to obtain a new candidate set, and calculating the main supervision loss based on the new candidate set; Obtaining a comprehensive loss according to the self-supervised loss and the main supervised loss, and updating the network parameters of the item representation and the graph attention network by the comprehensive loss to optimize the predicted recommendation probability of each of the candidate items; When the comprehensive loss is less than a preset value, the recommendation optimization is terminated, and the optimal recommendation probability of each candidate item is obtained.
2. The conversation recommendation method based on self-supervised graph learning according to claim 1, characterized in that: The obtaining of the current session from the session set, learning the representation of each item in the current session through information propagation in a graph attention network, and obtaining the session representation by generating the user's long-term interests and short-term interests, includes: A local graph is constructed based on the current session. The local graph is represented as: G τ = {V τ ,E τ }, where V τ is a node set of the local graph, including all items in the current session; τ is the edge set of the local graph, including the pairwise transitive relations between items in the current session; Inputting the local graph into a graph attention network, and obtaining an embedding vector of each node in the local graph after information propagation on the graph attention network; After restoring the items from the local graph according to the embedding vector of each node, a sequence of item representations in the current session arranged in chronological order is obtained; Determine the embedding representation of the last item in the item representation sequence as the user's short-term interest, and aggregate the embedding representations of the items through an attention mechanism to generate the user's long-term interest; A session representation is generated according to the short-term interests and long-term interests of the user.
3. The conversation recommendation method based on self-supervised graph learning according to claim 2, characterized in that: The user's long-term interest is expressed as: c i =softmax(β i ), b i =W4σ(W5u i +W6C s ), Among them, C l For the long-term interests of users; C s is the user's short-term interest; W4, W5 and W6 are the weight parameters of the attention mechanism; softmax() is the softmax function; σ is the sigmoid function.
4. The conversation recommendation method based on self-supervised graph learning according to claim 2, characterized in that: The step of inputting the local graph into a graph attention network and obtaining an embedding vector of each node in the local graph after information propagation on the graph attention network includes: Obtain each node x in the local graph through the embedding layer of the graph attention network i The initial embedding vector of For the node x at layer l i , obtain the node x through the self-attention mechanism in the graph attention network i Its adjacent node x j The attention coefficient between Normalizing the attention coefficient through the softmax layer of the graph attention network to obtain a normalized coefficient; The node x i Combine the neighbor node information of the node x and use the normalization coefficient to update the node x i The embedding vector of is obtained.
5. The conversation recommendation method based on self-supervised graph learning according to claim 1, characterized in that: The step of establishing a global graph showing connections between items in different sessions according to the session set, acquiring connection information and non-connection information between items from the global graph to generate a self-supervisory signal, and calculating a self-supervisory loss according to the self-supervisory signal includes: Building a global graph based on the n sessions in the session set; For each item v in the current session i , get item v on the global graph i Neighbor node v j and non-neighbor nodes Among them, the number of neighbor nodes is the same as the number of non-neighbor nodes; The self-supervised loss is calculated by JS divergence, which is used to maximize the item v i With neighbor node v j distance, minimizing item v i With non-neighboring nodes The distance between them is specifically expressed as: Among them, L self is the self-supervision loss; u i For item v i The embedding representation of u j is the neighbor node v j The corresponding embedding representation; Non-neighbor node The corresponding embedding representation; T is the number of neighbor nodes or non-neighbor nodes; f() is the similarity evaluation function, and the similarity evaluation function f() is expressed as: f(in i ,in j )=in i T in j , Among them, u i T For item v i The transpose of the embedding representation of .
6. The conversation recommendation method based on self-supervised graph learning according to claim 1, characterized in that: The step of obtaining a candidate set including m candidate items and obtaining a predicted recommendation probability of each candidate item by using the session representation includes: Get each candidate item v in the candidate set i Embedded representation of ; For each candidate item v i The embedding representation and session representation of are input into the preset recommendation model to obtain each candidate item v output by the recommendation model i The predicted recommendation probability of ; the recommendation model is: in, is the candidate item v i The probability of being recommended to the user for interaction at the next timestamp; C T is the transpose of the session representation; u i is the candidate item v i Embedding representation of ; m is the number of candidate items in the candidate set.
7. The conversation recommendation method based on self-supervised graph learning according to claim 1, characterized in that: The step of performing target adaptive shielding processing on the candidate set to obtain a new candidate set, and calculating the main supervision loss according to the new candidate set, includes: Get the target item v a , and sample from the global graph to obtain the target item v a The most similar t neighbors generate a neighbor set; After shielding t neighbors from the candidate set, a new candidate set is obtained; the new candidate set is expressed as: M*=M\N(v a ), Among them, M* is the new candidate set; M is the original candidate set; N(v a ) is the target item v a The neighbor set of ; "\" is the subtraction operation between sets; According to the new candidate set, the main supervision loss is calculated by the cross entropy loss function, which is specifically expressed as: Among them, L main Mainly supervise the loss; i , For each candidate item v in the new candidate set M* i The true predicted recommendation probability and the predicted predicted recommendation probability.
8. A conversation recommendation system based on self-supervised graph learning, characterized in that: include: A session set acquisition module is used to acquire a session set, which includes n sessions; A session representation module, used to obtain a current session from the session set, learn the representation of each item in the current session through information propagation through a graph attention network, and obtain a session representation by generating a user's long-term interest and short-term interest; A self-supervised learning module, configured to establish a global graph showing connections between items in different sessions based on the session set, obtain connection information and non-connection information between items from the global graph to generate a self-supervised signal, and calculate a self-supervised loss based on the self-supervised signal; A recommendation prediction module, used to obtain a candidate set including m candidate items, and obtain a predicted recommendation probability of each candidate item using the session representation; A shielding module is used to perform target adaptive shielding processing on the candidate set to obtain a new candidate set, and calculate the main supervision loss according to the new candidate set; A recommendation optimization module is used to obtain a comprehensive loss based on the self-supervised loss and the main supervised loss, and to update the network parameters of the item representation and the graph attention network through the comprehensive loss to optimize the predicted recommendation probability of each of the candidate items; when the comprehensive loss is less than a preset value, the recommendation optimization is terminated, and the optimal recommendation probability of each of the candidate items is obtained.
9. The conversation recommendation system based on self-supervised graph learning according to claim 8, characterized in that: The session representation module comprises: The local graph construction submodule is used to construct a local graph according to the current session. The local graph is represented as: G τ = {V τ ,E τ }, where V τ is a node set of the local graph, including all items in the current session; τ is the edge set of the local graph, including the pairwise transitive relations between items in the current session; A representation learning submodule, used for inputting the local graph into a graph attention network, and obtaining an embedding vector of each node in the local graph after information propagation on the graph attention network; An item representation submodule, configured to recover items from the local graph according to the embedding vector of each node, and obtain a chronologically ordered sequence of item representations in the current session; A preference generation submodule, used to determine the embedding representation of the last item in the item representation sequence as the user's short-term interest, and to generate the user's long-term interest by aggregating the embedding representations of the items through an attention mechanism; The session representation submodule is used to generate a session representation according to the short-term interests and long-term interests of the user.
10. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the computer program, the conversation recommendation method based on self-supervised graph learning as described in any one of claims 1 to 7 is implemented.
Citation Information
Patent Citations
Session recommendation method and system based on graph neural network and comment similarity
CN113610610A
Session-based recommendation method
CN114155063A
Item recommendation method based on user intention in session and system thereof
US20210366025A1
Learning word representations via commonsense reasoning
US20220051083A1
