A dual-channel graph fusion conversation recommendation method with information loss
By building edge directed graphs and global shortcut graphs, combined with dual-channel project-global dependency neural network, the problems of information loss and long dependency capture in the session recommendation system are solved, and more accurate project relationship modeling and recommendation system are achieved. Performance improvement.
Patent Information
- Application Number
- CN202410283076.3
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2024-03-13
- Publication Date
- 2024-12-31
- Estimated Expiration
- 2044-03-13
AI Technical Summary
Existing session recommendation systems have problems of information loss and invalid long-term dependency capture in graph structure transformation and model propagation learning, making it difficult to accurately model complex high-order relationships between projects.
Build edge directed graphs and global shortcut graphs, combine dual-channel project-global dependency neural network, and process graph structures through edge directed layer embedding learning modules and global shortcut layer embedding learning modules, realize project embedding learning, and pass multiple times Interleaving layer processing and information gating fusion improve the accuracy of recommendation systems.
It effectively solves the problems of information loss and long-term dependence, improves the recommendation system's understanding and prediction ability of user behavior, and significantly improves the accuracy and effectiveness of recommendations.
Smart Images
Figure CN118014004B_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to a conversation recommendation technology, and in particular to a conversation recommendation method for processing information loss by dual-channel graph fusion. Background Art
[0002] With the rapid advancement of science and technology, people have to deal with a lot of information every day. However, this information is often mixed with a lot of irrelevant content. Therefore, it is crucial to identify and filter out this redundant information. By analyzing the interests of users, the recommendation system can effectively filter out the information that matches them and provide users with personalized recommendation services. Therefore, the recommendation system has broad application prospects in various fields.
[0003] Traditional recommendation systems usually rely on known user information to make recommendation decisions. However, due to privacy considerations, user information is not completely known in many cases. Therefore, session-based recommendation systems are gradually emerging. This system uses the behavior sequence of anonymous users to predict the next possible behavior, and analyzes the user's interactive behavior in the session to gain insight into the user's preferences, thereby providing users with more accurate personalized recommendations.
[0004] In early research, the methods of conversational recommendation systems were proposed based on the idea of machine learning. The initial methods mainly used the ideas of Markov chain and matrix factorization, but these methods can only capture first-order dependencies and are difficult to capture long-term high-order dependencies. With the popularity of recurrent neural network methods, the use of recurrent neural networks to model conversation data into ordered sequences and model the order of items in conversations has achieved good results. However, the conversion of conversation data is not a strict temporal conversion, so recurrent neural networks cannot model more complex associations between items well. In contrast, graph neural networks can model conversation data into graph data, so graph neural networks are widely used in conversational recommendation systems. However, in actual scenarios, the emergence of an item is often the result of the joint action of a series of previous items, and the relationship between items is not a simple binary relationship, but a more complex many-to-many relationship. Therefore, how to more accurately model complex high-order relationships between items has always been a challenge in the field of conversational recommendation systems.
[0005] At present, there are related studies. For example, patent CN114330299A discloses a conversation recommendation method, system, device and medium based on multi-faceted global relationship project representation learning, and patent CN117194765A discloses an interest-aware dual-channel graph contrast learning conversation recommendation method. However, these existing technical solutions often only consider adding global preferences and interest perception as auxiliary information to improve the accuracy of recommendations, and often ignore the information loss problem in graph conversion and model propagation learning, namely, lossy session encoding problem and invalid long-range dependency capture problem. For the lossy session encoding problem, due to the lossy encoding from session to graph and the permutation-invariant aggregation during message passing, some sequential information about project conversion is ignored. For the invalid long-term dependency capture problem, due to the limited number of layers, some long-range dependencies in the session cannot be captured. These problems can be partially solved by annotating some auxiliary information, but how to reintegrate the information and improve the accuracy of prediction is still a problem that needs to be studied.
[0006] In other words, existing technologies focus on solving the fusion of global preferences, long-term or short-term preferences as auxiliary information, and do not consider the problem of graph structure transcoding itself and the problem of invalid long dependencies in the propagation learning process. Summary of the invention
[0007] Purpose of the invention: The purpose of the present invention is to solve the deficiencies in the prior art and to provide a conversation recommendation method for dual-channel graph fusion that handles information loss.
[0008] Technical solution: A dual-channel graph fusion conversation recommendation method for processing information loss of the present invention comprises the following steps: Step S1, constructing an item set V = {v1, v2, v3, ..., v |v|}、session set S={s1,s2,s3,...,s |s|} and similar conversation sets; where |v| is the number of items in the item set, |s| is the number of conversations in the conversation set, and each conversation is represented by a set s, s = {v1,v2,v3,...,v |n|}, n is the length of the session; N s is the set of similar sessions to session s, where s |n| is the number of similar sessions in the similar session set;
[0009] Step S2: construct corresponding edge-ordered directed graphs and global shortcut graphs based on the current session and similar session set data, and use time information and global preference to enhance the session representation respectively;
[0010] The edge-ordered directed graph is constructed as follows: First, define G h =(V h ,Eh ), represents all items in the item set V, Represents the set of edges. Since the edge-ordered directed graph is a multi-element structure, multiple edges from the target point to v will be generated. Therefore, we define E in (v) indicates the number of times the target node has appeared, through E in (v) to record the edge order information of each edge;
[0011] The method for constructing the global shortcut graph is as follows: First, define G g =(V g ,E g ), represents all items in the item set V, Represents a set of edges, and then for each ordered pair of nodes, a shortcut edge e is created i =(v i →v j )∈E g ;
[0012] Here, the global shortcut edges and global shortcut graphs can solve the problem of ineffective long-range dependency capture in the existing technology; and shortcut dependencies are added to each session ordered pair, so that information between non-adjacent items can be better transmitted; it helps to capture broader and deeper project associations, and improves the recommendation system's ability to understand and predict user behavior.
[0013] Step S3, inputting the edge-ordered directed graph and the global shortcut graph obtained in step S2 into a dual-channel project-global dependency neural network, the dual-channel project-global dependency neural network comprising an edge-ordered directed layer embedding learning module and a global shortcut layer embedding learning module, the edge-ordered directed layer embedding learning module and the global shortcut layer embedding learning module process the edge-ordered directed graph and the global shortcut graph respectively, realize project node embedding learning, and finally output project embedding and global dependency embedding under two graph structures;
[0014] The edge-ordered directed layer embedding learning module processes the edge-ordered directed graph as follows:
[0015]
[0016]
[0017]
[0018] Where E in (i) indicates that the conversation starts with v i The order of the edges as endpoints in edge order, represents the updated item embedding obtained by aggregating the feature information of all adjacent nodes of the target item embedding, represents the i-th item embedding in the l-th layer network, Represents the summary state after aggregating the information of all adjacent nodes, GRU (l) represents the information aggregation calculation of the lth layer network, represents the learnable parameters used to aggregate neighboring nodes in the l-th layer network, Indicates that the l-th layer network follows v i The k-th item embedding corresponding to the edge order of Represents the hidden state information of the first k-1 GRUs in series in the l-th layer network, 0≤k≤d i ,|| indicates concatenation; is the learnable parameter matrix;
[0019] The global shortcut layer embedding learning module processes the global shortcut graph as follows:
[0020]
[0021]
[0022]
[0023] in express The updated item embedding is obtained by aggregating the feature information of all adjacent nodes. Indicates that in the lth layer network The attention score between , softmax represents the activation function, which is used for normalization. express The weight vector, σ represents the sigmoid function, express The weight between represents the item embedding of a certain adjacent node, p (l) ,b (l) ∈R d and are all learnable parameters, Indicates about The attention score of , T represents the transposition operation;
[0024] Step S4: Perform multiple interleaved layer processing and embedding on the project embedding and global dependency embedding obtained in step S3 respectively. The specific method is as follows:
[0025] First, interleave the items of the current session into the global shortcut layer and the edge-ordered directed layer, and cascade the outputs of all previous layers. Then, multiple convolutional network layers are stacked to obtain high-level information, and the embedding representation of each item is output in the last layer to finally obtain the item embedding of the current session.
[0026] First, items from similar conversation sets are interleaved and embedded into edge-ordered directed layers and global shortcut layers, and the outputs of all previous layers are cascaded. Then, multiple convolutional network layers are stacked to obtain high-level information, and the embedding representation of each item is output in the last layer, and finally the item embedding of similar session sets is obtained.
[0027] Step S5, performing information gating fusion based on the current session embedding and similar session set embedding obtained; Step S6, using the attention mechanism to aggregate nodes to achieve graph representation;
[0028] Step S7: Calculate the recommendation probability of the candidate items and give a loss function.
[0029] Furthermore, the step S5 is to fuse the final current session embedding and the similar session set embedding information obtained; the method is as follows:
[0030]
[0031]
[0032] in represents the weight matrix in the fusion gate, b f represents the offset vector, which are all learnable parameters, σ represents the activation function, and f represents the importance of the input. is the final project embed.
[0033] Furthermore, the detailed method of step S6 is:
[0034] Embed x using the last item in session s n As the local level session embedding, the attention mechanism is then used to calculate the weights to obtain the global level session embedding. The specific calculation formula is as follows:
[0035]
[0036] β=softmax(ε)
[0037]
[0038] where q,r∈R d and M1,M2∈R d×d are all learnable parameters, q is the projection vector, r is the offset vector, represents the item embedding of the i-th item in the session, x n is the local level session embedding, β represents the attention score of item i for the entire session, and h gRepresents a global level session embedding.
[0039] Furthermore, the specific calculation formula of step S6 is as follows:
[0040] s h =M h (s g ||s l )
[0041]
[0042]
[0043]
[0044] Among them, M h is a learnable parameter, s g =h g ,s l =x n is the session embedding representation, z i Indicates project v i The prediction score, y i represents the true value label of the i-th candidate item, represents the predicted probability of the i-th candidate item, represents the cross entropy loss function.
[0045] Beneficial effect: The present invention pays attention to lossy session coding and invalid long dependencies through edge-ordered directed graphs and global shortcut graphs, solves the information loss problem of multivariate graph structures by setting edge order information, solves the problem of invalid long dependencies by using shortcut edges, and solves the respective bias problems by using fusion functions, thereby improving recommendation efficiency. Compared with the prior art, it has made significant progress. BRIEF DESCRIPTION OF THE DRAWINGS
[0046] Figure 1 It is a schematic diagram of the overall structure of the model of the present invention.
[0047] Figure 2 It is a schematic diagram of constructing the edge-ordered directed graph and the global shortcut graph in the present invention. DETAILED DESCRIPTION
[0048] The technical solution of the present invention is described in detail below, but the protection scope of the present invention is not limited to the embodiments.
[0049] like Figure 1 As shown, a dual-channel graph fusion conversation recommendation method for processing information loss of the present invention comprises the following steps:
[0050] Step S1: Construct a set of items V = {v1, v2, v3, ..., v |v|}、session set S={s1,s2,s3,...,s |s|} and similar conversation sets and conversations; where |v| is the number of items, |s| is the number of conversations, and each conversation is represented by a set s, s = {v1,v2,v3,...,v |n|}, n is the length of the session; N s is the set of similar sessions to session s, where s |n| is the number of similar sessions;
[0051] Step S2: Figure 2 As shown, according to the current session and similar session set data, the corresponding edge-ordered directed graph and global shortcut graph are constructed respectively, and the session representation is enhanced using time information and global preference respectively; the construction method of the edge-ordered directed graph is as follows: first define represents all items in the item set V, Represents a set of edges, and then labels the time sequence according to the number of times the node currently appears;
[0052] The method for constructing the global shortcut map is as follows: first define represents all items in the item set V, Represents a set of edges, and then for each ordered pair of nodes, a shortcut edge is created;
[0053] Step S3, inputting the edge-ordered directed graph and the global shortcut graph obtained in step S2 into a dual-channel project-global dependency neural network, the dual-channel project-global dependency neural network comprising an edge-ordered directed layer embedding learning module and a global shortcut layer embedding learning module, the edge-ordered directed layer embedding learning module and the global shortcut layer embedding learning module process the edge-ordered directed graph and the global shortcut graph respectively, realize project node embedding learning, and finally output project embedding and global dependency embedding under two graph structures;
[0054] The edge-ordered directed layer embedding learning module processes the edge-ordered directed graph as follows:
[0055]
[0056]
[0057]
[0058] Where E in (i) indicates that the conversation starts with v i The order of the edges as endpoints in edge order, represents the updated item embedding obtained by aggregating the feature information of all adjacent nodes of the target item embedding, represents the i-th item embedding in the l-th layer network, Represents the summary state after aggregating the information of all adjacent nodes, GRU (l) represents the information aggregation calculation of the lth layer network, represents the learnable parameters used to aggregate neighboring nodes in the l-th layer network, Indicates that the l-th layer network follows v i The k-th item embedding corresponding to the edge order of Represents the hidden state information of the first k-1 GRUs in series in the l-th layer network, 0≤k≤d i ,|| indicates concatenation; is the learnable parameter matrix;
[0059] The global shortcut layer embedding learning module processes the global shortcut graph as follows:
[0060]
[0061]
[0062]
[0063] in express The updated item embedding is obtained by aggregating the feature information of all adjacent nodes. Indicates that in the lth layer network The attention score between , softmax represents the activation function, which is used for normalization. express The weight vector, σ represents the sigmoid function, express The weight between represents the item embedding of a certain adjacent node, p (l) ,b (l) ∈R d and are all learnable parameters, Indicates about The attention score of , T represents the transposition operation;
[0064] Step S4: Perform multiple interleaved layer processing and embedding on the project embedding and global dependency embedding obtained in step S3 respectively. The specific method is as follows:
[0065] First, interleave the items of the current session into the global shortcut layer and the edge-ordered directed layer, and cascade the outputs of all previous layers. Then, multiple convolutional network layers are stacked to obtain high-level information, and the embedding representation of each item is output in the last layer to finally obtain the item embedding of the current session.
[0066] First, items from similar conversation sets are interleaved and embedded into edge-ordered directed layers and global shortcut layers, and the outputs of all previous layers are cascaded. Then, multiple convolutional network layers are stacked to obtain high-level information, and the embedding representation of each item is output in the last layer, and finally the item embedding of similar session sets is obtained.
[0067] Step S5: perform information gating fusion based on the obtained current session embedding and similar session set embedding, the method is as follows:
[0068]
[0069]
[0070] in represents the weight matrix in the fusion gate, b f represents the offset vector, which are all learnable parameters, and f expresses the importance of the input. It is the final embedded project;
[0071] Step S6: Use the attention mechanism to aggregate nodes to achieve graph representation;
[0072] Embed x using the last item in session s n As the local level session embedding, the attention mechanism is then used to calculate the weights to obtain the global level session embedding. The specific calculation formula is as follows:
[0073]
[0074] β=softmax(ε)
[0075]
[0076] where q,r∈R d and M1,M2∈R d×d are all learnable parameters, q is the projection vector, r is the offset vector, represents the item embedding of the i-th item in the session, x n is the local level session embedding, β represents the attention score of item i for the entire session, and h g Represents a global level session embedding;
[0077] Step S7, calculating the recommendation probability of the candidate items and giving a loss function;
[0078] sh =M h (s g ||s l );
[0079]
[0080]
[0081]
[0082] Among them, M h is a learnable parameter, s g =h g ,s l =x n is the session embedding representation, z i Indicates project v i The prediction score, y i represents the true value label of the i-th candidate item, represents the predicted probability of the i-th candidate item, represents the cross entropy loss function.
[0083] Example
[0084] In order to evaluate the effect of the technical solution of the present invention, two public datasets are selected. The datasets are preprocessed first and then experimentally verified. They are Diginetica and Yoochoose 1 / 64. The Diginetica dataset is derived from CIKMCup 2016 (http: / / cikm2016.cs.iupui.edu / cikm-cup), while the Yoochoose1 / 64 dataset is derived from RecSys Challenge 2015.
[0085] (http: / / 2015.recsyschallenge.com / challenge.html).
[0086] The Diginetica dataset includes 186,705 training sessions and 16,121 test sessions, involving a total of 43,097 different items. The Yoochoose 1 / 64 dataset includes 124,632 training sessions and 15,436 test sessions, containing a total of 17,598 different items.
[0087] During the training process, this embodiment sets the dimension of the item representation vector to 100 and uses the Adam optimizer. The initial learning rate is 0.001, which decays to 0.1 of the original value after every three cycles. The specific training results are shown in Table 1.
[0088] Table 1 Data comparison between the present invention and the prior art
[0089]
[0090] Table 1 Comparison of experimental results of the method of the present invention with other prior art experimental results According to the results in the above table, the present invention has significantly improved both the hit rate and the average reciprocal ranking. Recall@20 indicates the frequency of correct items in the recommendation list, that is, the degree of recommendation effect. The higher the value, the better the effect. MRR@20 indicates the sum of the reciprocals of the positions of correct items in the recommendation list. Similarly, the higher the value, the better the recommendation effect.
[0091] It can be seen from the above experiments that the present invention constructs two different graph structures, the edge-ordered directed graph and the global shortcut graph, which focus on collecting different aspects of information between projects. The edge-ordered directed graph focuses on mining the temporal relationship between projects. By using the edge-ordered directed graph structure, the loss of original information that may be caused when converting the multivariate relationship between projects into a binary relationship is effectively reduced, which helps to retain the temporal information between projects and improve the accuracy and richness of the relationship between projects. The global shortcut graph combines global session information to model the dependencies of projects in the session. At the same time, the present invention uses a fusion function to enhance the information of the project embeddings in the two graph structures, so that the information in the two views complements each other; after fusing the project embeddings of the two views, the present invention uses a self-attention layer to assign weights to different projects, and then obtains session embeddings, thereby improving the accuracy of recommendations.
Claims
1. A dual-channel graph fusion conversation recommendation method for processing information loss, characterized in that: The following steps are involved: Step S1: Construct a set of items V = {v1, v2, v3, ..., v |v| }、session set S={s1,s2,s3,...,s |s| } and similar conversation sets; where |v| is the number of items in the item set, |s| is the number of conversations in the conversation set, and each conversation is represented by a set s, s = {v1,v2,v3,...,v |n| }, n is the length of the session; N s is the set of similar sessions to session s, where s |n| is the number of similar sessions in the similar session set; Step S2: construct corresponding edge-ordered directed graphs and global shortcut graphs based on the current session and similar session set data, and use time information and global preference to enhance the session representation respectively; The method for constructing the edge-ordered directed graph is as follows: First, define G h =(V h ,E h ), represents all items in the item set V, Represents the set of edges, and then according to the number of times the node currently appears E in (v) marking the timing; The method for constructing the global shortcut map is as follows: first define represents all items in the item set V, Represents a set of edges, and then for each ordered pair of nodes, a shortcut edge e is created i =(v i →v j )∈E g ; Step S3, inputting the edge-ordered directed graph and the global shortcut graph obtained in step S2 into a dual-channel project-global dependency neural network, the dual-channel project-global dependency neural network comprising an edge-ordered directed layer embedding learning module and a global shortcut layer embedding learning module, the edge-ordered directed layer embedding learning module and the global shortcut layer embedding learning module process the edge-ordered directed graph and the global shortcut graph respectively, realize project node embedding learning, and finally output project embedding and global dependency embedding under two graph structures; The edge-ordered directed layer embedding learning module processes the edge-ordered directed graph as follows: Where E in (i) indicates that the conversation starts with v i The order of the edges as endpoints in edge order, represents the updated item embedding obtained by aggregating the feature information of all adjacent nodes of the target item embedding, represents the i-th item embedding in the l-th layer network, Represents the summary state after aggregating the information of all adjacent nodes, GRU (l) represents the information aggregation calculation of the lth layer network, represents the learnable parameters used to aggregate neighboring nodes in the l-th layer network, Indicates that the l-th layer network follows v i The k-th item embedding corresponding to the edge order of Represents the hidden state information of the first k-1 GRUs in series in the l-th layer network, 0≤k≤d i ,|| indicates concatenation; is the learnable parameter matrix; The global shortcut layer embedding learning module processes the global shortcut graph as follows: in express The updated item embedding is obtained by aggregating the feature information of all adjacent nodes. Indicates that in the lth layer network The attention score between , softmax represents the activation function, which is used for normalization. express The weight vector, σ represents the sigmoid function, express The weight between represents the item embedding of a certain adjacent node, p (l) ,b (l) ∈R d and are all learnable parameters, Indicates about The attention score of , T represents the transposition operation; Step S4: Perform multiple interleaved layer processing and embedding on the project embedding and global dependency embedding obtained in step S3 respectively. The specific method is as follows: First, interleave the items of the current session into the global shortcut layer and the edge-ordered directed layer, and cascade the outputs of all previous layers. Then, multiple convolutional network layers are stacked to obtain high-level information, and the embedding representation of each item is output in the last layer to finally obtain the item embedding of the current session. First, items from similar conversation sets are interleaved and embedded into edge-ordered directed layers and global shortcut layers, and the outputs of all previous layers are cascaded. Then, multiple convolutional network layers are stacked to obtain high-level information, and the embedding representation of each item is output in the last layer, and finally the item embedding of similar session sets is obtained. Step S5: performing information gating fusion based on the obtained current session embedding and similar session set embedding; Step S6: Use the attention mechanism to aggregate nodes to achieve graph representation; Step S7: Calculate the recommendation probability of the candidate items and give a loss function; the specific formula is as follows: s h =M h (s g ||s l ) Among them, M h is a learnable parameter, s g =h g ,s l =x n is the session embedding representation, z i Indicates project v i The prediction score, y i represents the true value label of the i-th candidate item, represents the predicted probability of the i-th candidate item, represents the cross entropy loss function.
2. The conversation recommendation method of dual-channel graph fusion with information loss processing according to claim 1, characterized in that: The detailed fusion method of step S5 is: in represents the weight matrix in the fusion gate, b f represents the offset vector, which are all learnable parameters, σ represents the activation function, and f represents the importance of the input. is the final project embed.
3. The conversation recommendation method of dual-channel graph fusion with information loss processing according to claim 1, characterized in that: The detailed process of step S6 is as follows: Embed x using the last item in session s n As the local level session embedding, the attention mechanism is then used to calculate the weights to obtain the global level session embedding. The specific calculation formula is as follows: β=softmax(ε) where q,r∈R d and M1,M2∈R d×d are all learnable parameters, q is the projection vector, r is the offset vector, represents the item embedding of the i-th item in the session, x n is the local level session embedding, β represents the attention score of item i for the entire session, and h g Represents a global level session embedding.