Session recommendation method based on multi-intention perception and dual gating
By adopting multi-intention perception and dual gating mechanisms in conversation recommendation technology, combining location, frequency and user behavior embedding, the problems of short-term user interest changes and noise interference are solved, and more accurate and personalized recommendation results are achieved.
Patent Information
- Application Number
- CN202411608814.3
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-11-12
- Publication Date
- 2025-03-07
AI Technical Summary
Existing session recommendation technologies are difficult to effectively deal with short-term user interest changes and noise interference, resulting in limited accuracy of recommendation results.
A session recommendation method based on multi-intention perception and dual-gated gating is adopted. By combining absolute position embedding, frequency embedding and user behavior embedding, a two-layer gating mechanism is designed to dynamically model user intentions, and a two-way self-attention layer self-attention captures the complex relationship between session items.
It significantly reduces the noise impact generated by users' random behavior, improves the model's sensitivity to users' real intentions, and enhances the accuracy and personalization of recommended results.
Smart Images
Figure CN119577083A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to a conversation recommendation method based on multi-intention perception and dual gating. Background Art
[0002] In today's digital age, the popularity of social platforms and online services has led to increasingly frequent interactions between users. As an important tool to improve user experience, the conversational recommendation system can provide users with personalized recommendations by intelligently analyzing the user's conversation content, historical preferences, and real-time behavior. This recommendation method can not only help users quickly find topics or content of interest, but also enhance user interactivity and participation.
[0003] At the same time, facing the ever-changing interests of users and the complexity of conversation contexts, existing conversational recommendation technologies still face many challenges. First, users' conversation sequences are often affected by noise, such as misclicks, redundant information, and outdated interests, which seriously interfere with the model's judgment of the user's true intentions. Second, traditional methods focus on capturing users' long-term preferences, but fail to effectively handle short-term sudden changes in interests, which limits the accuracy of recommendation results.
[0004] Most existing studies use self-attention networks to mine long-term dependencies in user sequences to predict the content that users are interested in. However, this approach does not take into account the large amount of noise in user sequences and focuses more on the long-term dependencies of users, ignoring the sudden interactions of users, resulting in overly smooth predictions.
[0005] In recent years, gating mechanisms have shown significant effects in improving the performance of recommendation systems. Gating mechanisms enhance the ability to capture users' true intentions by intelligently filtering and adjusting information in user behavior sequences. For example, some methods use dynamic weight allocation strategies to flexibly adjust the influence of different features in recommendation decisions based on user context information, thereby constructing more representative user feature vectors. The core idea of this mechanism can be summarized as two interrelated steps: first, filtering and selecting key information in user sequences through gating strategies; second, using optimization algorithms to adjust gating parameters to minimize prediction errors.
[0006] However, the existing gating mechanism is still insufficient in dealing with short-term changes in users' interests and noise interference. Many methods often fail to effectively deal with noise in the sequence when considering long-term dependencies, such as misclicks and redundant information, which may cause the model to deviate from the user's preference judgment and ultimately affect the accuracy of the recommendation. Therefore, further research and optimization of the gating mechanism is needed to improve the model's sensitivity to the user's real needs and improve the recommendation effect.
[0007] Meanwhile, most existing conversational recommendation models focus on designing complex neural encoders to learn conversational representations that capture the relationships between conversation items. However, they tend to focus on the last item and ignore the various user intentions that may exist in a conversation.
[0008] Therefore, there is an urgent need for a method to solve the problem of noise caused by users' random behaviors in the recommendation model data in the conversation recommendation method and how to extract users' multiple intentions from the conversation. Summary of the invention
[0009] The purpose of the present invention is to solve the problem that the noise data generated by the random behavior of the user misleads the final prediction of the model and how to extract the multiple intentions of the user from the conversation. A conversation recommendation method based on multi-intention perception and dual gating is provided.
[0010] To achieve the above purpose, the technical solution adopted by the present invention is as follows:
[0011] S1. First, we transform each session item Mapped to an embedding vector through the embedding layer And generate the average term embedding vector Will With v m Perform splicing to obtain global information v i .
[0012]
[0013] To better capture the importance of each item in the session, we combine the absolute position embedding vector To distinguish the order, frequency embedding vector to represent the importance of repeated items in the session. Finally, the item is embedded in the vector m i , position embedding vector a i and frequency embedding vector are combined into the input variables of the model.
[0014]
[0015] S2. We have learned three different embeddings from different perspectives, namely user embedding v u =m i , position embedding Frequency Embedding We design a double-gating mechanism to hierarchically integrate these three embeddings for dynamic intent modeling. Specifically, we first embed the session and user embed v u The merger is as follows:
[0016]
[0017]
[0018]
[0019]
[0020] in is the element product, σ is the sigmoid function, is a learnable parameter. We combine session embedding and user embedding in a complementary way, balancing the contribution of session and user information. We call it user-aware item embedding For different sessions, we can learn different user-aware item embeddings for an item, which is consistent with the fact that an item should be represented differently for different users. We then embed the user behavior and To perform integration:
[0021]
[0022]
[0023]
[0024]
[0025] in is a learnable parameter, v i is the final item embedding. We unrestrictedly combine the user embedding with the user behavior embedding through two independent gates (f, i), v i Highlights the importance of user behavior. i The user's dynamic intention is encoded from the perspectives of item perception, user perception and time perception.
[0026] S3. Through the above processing, we have removed the noise in the data to a certain extent, and then we use the bidirectional self-attention layer to capture the complex relationship between session items. The attention formula is as follows, where Q is the query vector and K is the key vector:
[0027]
[0028] Use a bidirectional self-attention layer to self-attentionally generate multiple contextual representations of the session information associated with each item, as shown below:
[0029] c1,…,c |s| , c m =Self-attention([v1,…,v|s|， u m ])
[0030] To ensure that multiple context representations do not become similar and better reflect different user intentions, we emphasize item embedding by leveraging a high-speed network.
[0031] O i =g⊙v i +(1-g)⊙c i for i∈{1,…,|s|,m},
[0032] where g=σ(W g [v i ;c i ] T )
[0033] in is a learnable matrix, is a gating vector and σ(·) is the sigmoid function.
[0034] S4. We use multiple session representations to fully exploit the potential of each session item. However, not all session items are necessary and some may be noisy. To extract important user intents in a session, we calculate the importance of multiple representations and remove unimportant ones.
[0035]
[0036]
[0037]
[0038] Depending on the delay, 1-entmax and 2-entmax are equivalent to softmax and sparasemax, respectively. A larger α value will produce a sparser probability distribution. We set α to 1.5 based on experience. We extract session representations by masking unnecessary user intents.
[0039] γ=α-entmax(w·[o1;…;o |s| ;o m ] T ),
[0040] {h1,…,h k}={γ i o i |γ i >0, i∈{1,…,|s|,m}}
[0041] in is a learnable parameter, is the importance weight vector of each item. k is the number of non-zero elements in γ.
[0042] S5. To decode each session representation into an item distribution, we use the cosine similarity of the item embedding matrix, i.e., L2- no rmalized normalized dot product.
[0043]
[0044] in and are the normalized session vector and the normalized item embedding matrix, respectively. Represents the session vector h i The item distribution of .
[0045] To aggregate multiple item distributions, we employ max pooling and average pooling functions. Max pooling maintains the main features of multiple intents, and average pooling captures consistent intents in a session.
[0046]
[0047] in, For the final aggregate term distribution, β is a combined hyperparameter, MaxPool(.) and AvgPool(.) represent the maximum pooling and average pooling functions, which obtain the maximum or average value of each dimension from multiple vectors.
[0048] Finally, we use the cross entropy loss function to train the model. The cross entropy loss function is defined as follows:
[0049]
[0050] in is the one-hot vector of the target item. In order to better converge, the hyperparameter τ that controls the temperature is used.
[0051] In summary, due to the adoption of the above technical solutions, the advantages and positive effects of the present invention are as follows: the present invention aims at the problem that the noise data generated by the random behavior of users affects the final prediction of the model and that most deep learning models often ignore the potential intentions of user behavior, and provides a conversation recommendation method based on multi-intention perception and dual gating. This method captures the location information and frequency information of the conversation to distinguish the importance of the project, combines user embedding with user behavior embedding through a double-layer gating mechanism, and highlights the importance of user behavior. At the same time, the dynamic intention of the user is encoded from the perspectives of project perception, user perception, and time perception, significantly removing the influence of the user's random behavior. A bidirectional self-attention layer is used to self-attentionally generate multiple contextualized representations of the conversation information related to each project, and the user intention embedding is emphasized through a high-speed network to prevent excessive similarity in context, and to a certain extent, multiple intentions of users in the conversation are extracted. BRIEF DESCRIPTION OF THE DRAWINGS
[0052] Figure 1 It is a schematic diagram of the method flow of the present invention.
[0053] Figure 2 It is a schematic diagram of the model framework of the present invention. DETAILED DESCRIPTION
[0054] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.
[0055] The present invention proposes a conversation recommendation method based on multi-intention perception and dual gating. The main process of the method is as follows: Figure 1 The specific implementation process is as follows:
[0056] S1. First, we transform each session item Mapped to an embedding vector through the embedding layer And generate the average term embedding vector Will With v m Perform splicing to obtain global information v i .
[0057]
[0058] To better capture the importance of each item in the session, we combine the absolute position embedding vector To distinguish the order, frequency embedding vector to represent the importance of repeated items in the session. Finally, the item is embedded in the vector m i , position embedding vector a i and frequency embedding vector are combined into the input variables of the model.
[0059]
[0060] S2. We have learned three different embeddings from different perspectives, namely user embedding v u =m i , position embedding Frequency Embedding We design a double-gating mechanism to hierarchically integrate these three embeddings for dynamic intent modeling. Specifically, we first embed the session and user embed v u The merger is as follows:
[0061]
[0062]
[0063]
[0064]
[0065] in is the element product, σ is the sigmoid function, is a learnable parameter. We merge session embedding and user embedding in a complementary way, balancing the contribution of session and user information. We call it user-aware item embedding For different sessions, we can learn different user-aware item embeddings for an item, which is consistent with the fact that an item should be represented differently for different users. We then embed the user behavior and To perform integration:
[0066]
[0067]
[0068]
[0069]
[0070] in is a learnable parameter, v i is the final item embedding. We unrestrictedly combine the user embedding with the user behavior embedding through two independent gates (f, i), vi Highlights the importance of user behavior. i The user's dynamic intention is encoded from the perspectives of item perception, user perception and time perception.
[0071] S3. Through the above processing, we have removed the noise in the data to a certain extent, and then we use the bidirectional self-attention layer to capture the complex relationship between session items. The attention formula is as follows, where Q is the query vector and K is the key vector:
[0072]
[0073] Use a bidirectional self-attention layer to self-attentionally generate multiple contextual representations of the session information associated with each item, as shown below:
[0074] c1,…,c |s| , c m =Self-attention([v1,…,u |s| ,u m ])
[0075] To ensure that multiple context representations do not become similar and better reflect different user intentions, we emphasize item embedding by leveraging a high-speed network.
[0076] O i =g⊙v i +(1-g)⊙c i for i∈{1,…,|s|,m},
[0077] where g=σ(W g [v i ;c i ] T )
[0078] in is a learnable matrix, is a gating vector and σ(·) is the sigmoid function.
[0079] S4. We use multiple session representations to fully exploit the potential of each session item. However, not all session items are necessary and some may be noisy. To extract important user intents in a session, we calculate the importance of multiple representations and remove unimportant ones.
[0080]
[0081]
[0082]
[0083] Depending on the delay, 1-entmax and 2-entmax are equivalent to softmax and sparasemax, respectively. A larger α value will produce a sparser probability distribution. We set α to 1.5 based on experience. We extract session representations by masking unnecessary user intents.
[0084] γ=α-entmax(w·[o1;…;o |s| ;o m ] T ),
[0085] {h1,…,h k}={γ i p i |γ i >0, i∈{1,…,|s|,m}}
[0086] in is a learnable parameter, is the importance weight vector of each item. k is the number of non-zero elements in γ.
[0087] S5. To decode each session representation into an item distribution, we use the cosine similarity of the item embedding matrices, i.e., the L2-normalized dot product.
[0088]
[0089] in and are the normalized session vector and the normalized item embedding matrix, respectively. Represents the session vector h i The item distribution of .
[0090] To aggregate multiple item distributions, we employ max pooling and average pooling functions. Max pooling maintains the main features of multiple intents, and average pooling captures consistent intents in a session.
[0091]
[0092] in, For the final aggregate term distribution, β is a combined hyperparameter, MaxPool(.) and AvgPool(.) represent the maximum pooling and average pooling functions, which obtain the maximum or average value of each dimension from multiple vectors.
[0093] Finally, we use the cross entropy loss function to train the model. The cross entropy loss function is defined as follows:
[0094]
[0095] in is the one-hot vector of the target item. In order to better converge, the hyperparameter τ that controls the temperature is used.
[0096] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit the present invention. Although the embodiments of the present invention have been shown and described, it is understood by those skilled in the art that various changes, modifications, substitutions and variations may be made to these embodiments without departing from the principles and spirit of the present invention, and the scope of the present invention is defined by the appended claims and their equivalents.
Claims
1. A conversation recommendation method based on multi-intention perception and dual gating, characterized in that: The following steps are involved: S1. Embed the session information through the embedding layer to generate the session embedding vector v b i , and we capture the session location information a i To distinguish order and frequency information To represent the importance of the session, the session embedding vector, position embedding vector and frequency embedding vector are combined as the input variable x of the model i S2. We input the three different embeddings learned from different perspectives into the dual gating mechanism to remove the noise of the dataset to a certain extent and generate the final output v i S3. The final input v i The input is fed into a bidirectional self-attention layer to obtain the contextual representation of the session information, and then the high-speed network is used to emphasize the item embedding to avoid excessive similarity between different session information. S4. In order to extract the important user intent of the session, we calculated the importance of multiple representations and deleted the unimportant representations. S6. Each session is decoded into an item distribution using cosine similarity, and multiple item distributions are aggregated using the maximum pooling and average pooling functions. Finally, the cross entropy loss function is used to train the model.
2. According to claim 1, a conversation recommendation method based on multi-intention perception and dual gating is characterized in that: Step S2 includes: First, we extract the user's session embedding vector, position embedding vector, and frequency embedding vector through S1, and input these three embedding vectors from different angles into the double-gating mechanism. Through the double-layer gating mechanism, the user embedding is combined with the user behavior embedding, highlighting the importance of user behavior. At the same time, the user's dynamic intention is encoded from the perspectives of item perception, user perception, and time perception to obtain the final embedding v i .
3. According to claim 1, a conversation recommendation method based on multi-intention perception and dual gating is characterized in that: Step S3 includes: first, embedding the final output of the double-layer gating mechanism into v i The input is fed into the self-attention layer, and the lower triangular mask is used to ensure that each position only pays attention to the current and previous positions. Bidirectional self-attention is used to capture the complex relationship between session items, and the output t is obtained. i , the output t i With session input x i Residual connections are made through high-speed network layers to emphasize item embedding, ensuring that multiple context representations do not become similar and extracting different user intentions.