Recommendation method for user interest decoupling based on hierarchical intention

Abstract

The invention discloses a recommendation method for user interest decoupling based on hierarchical intention. Most of current fine-grained interest recommendation systems only decouple intention factors of various surfaces from interaction, and neglects the hierarchical relationship between intentions. Secondly, in the intention learning process, most researches seriously depend on division of interaction data. The learning normal form cannot explore potential intentions from an overall perspective, possibly aggravates data sparseness and damages recommendation performance. In order to explore a hierarchical relationship between intentions, an intention-aware interest modeling layer is designed to learn diversified user / article representations in each intention hierarchy. Then, multiple interest modeling layers are stacked to extract more abstract and deeper intention semantics from low levels to enrich collaborative filtering signals. In addition, in order to avoid low training efficiency caused by division interaction and relieve the problem of sparse interaction, a self-adaptive intention learning strategy is designed in each interest modeling layer.

Description

Technical Field

[0001] The present invention relates to collaborative filtering technology, specifically a model-based collaborative filtering algorithm; the present invention relates to a representation learning method based on graph convolution, specifically a GNN algorithm; the present invention relates to a learning method based on an attention mechanism; the present invention also relates to related technologies such as deep learning. Background Art

[0002] With the popularization of the Internet, a large amount of information is flooding the network. Although this information meets the needs of users, the amount of information is growing exponentially, making it impossible for users to select the information they need efficiently and quickly, which reduces the utilization rate of user information. The above phenomenon is information overload. In order to alleviate information overload and improve information utilization, recommendation systems came into being.

[0003] A recommendation system is a system that uses historical behavior data, personal preferences and other information of users to recommend items or services that users may be interested in. The goal of a recommendation system is to provide personalized recommendations based on user needs and interests, help users find items or services that meet their preferences more quickly, and improve user experience and satisfaction. Content-based recommendations (CB) is a classic recommendation algorithm. Specifically, content-based recommendations have the following advantages: First, users are not interdependent. The construction of each user's features depends only on his or her own preferences for items, and has nothing to do with the behavior of others. Second, it is easy to explain. In certain specific scenarios, the recommendation system needs to explain to users why an item is recommended. It only needs to tell users that the recommended item has certain attributes, and these attributes often appear in items that users like, to complete the explanation of the recommendation results. Finally, it is not constrained by new users or new items. When a new user enters the recommendation system, content-based recommendations can be made based on the user's personal attribute information.

[0004] After content-based recommendation algorithms were proposed, researchers found that these algorithms have great limitations. For example, the content and quality of movies of the same type or even movies directed by the same director may vary greatly, and users' interests in them may also vary greatly. Simply recommending based on content will lead to low accuracy. In order to solve the above problems, the idea of ​​collaborative filtering was innovatively proposed. That is, analyzing the historical interaction records between users and items to predict the next item that users may interact with, and making full use of the existing user interaction data to find the content that best meets the user's interests.

[0005] The current deep learning methods are closely related to the above methods. Recommendation algorithms based on deep learning attempt to model data types that traditional algorithms cannot model, hoping to improve the effectiveness of recommendation algorithms by introducing more information. The vigorous development of deep learning-related technologies has also spawned a large number of new research directions for recommendation problems, greatly promoting the development of the recommendation algorithm field. For example, some research works have begun to explore the fine-grained interest composition of users, and to mine the various intentions of user interactions through decoupling or multi-perspective representation. However, the current fine-grained interest recommendation system has limitations. Most methods based on fine-grained interests only decouple multiple surface intention factors from the interaction, ignoring the hierarchical relationship between intentions. To some extent, existing methods, especially when data is sparse, find it difficult to extract rich collaborative signals from surface intentions, resulting in poor quality of interest modeling.

[0006] The present invention starts from this point and proposes a recommendation method based on user interest decoupling of hierarchical intent. Summary of the invention

[0007] There are two key limitations of current fine-grained interest recommendation systems. First, most fine-grained interest-based methods only decouple multiple surface intent factors from interactions, ignoring the hierarchical relationship between intents. To some extent, existing methods, especially in the case of sparse data, have difficulty extracting rich collaborative signals from surface intents, resulting in poor interest modeling quality, such as Figure 1 As shown. Secondly, user decisions show intentions at different levels. Low-level intentions focus on general and superficial user preferences, while high-level intentions focus on deep and abstract preferences. Therefore, an intention hypothesis is proposed that user interests are highly correlated with multi-level hierarchical intentions. Considering the hierarchical relationship between intentions, fine-grained interests can be modeled more accurately, and user decisions in real scenarios can be better understood and explained. In addition, in the process of intent learning, most studies rely heavily on the partitioning of interaction data. This learning paradigm cannot explore potential intentions from a holistic perspective, which may aggravate data sparsity and impair recommendation performance. In addition, whether it is hard partitioning or soft partitioning, the recommendation model needs more time to find the optimal partitioning ratio according to the task, which greatly reduces the training efficiency of the model.

[0008] In order to solve the above problems, the present invention discloses a recommendation method based on user interest decoupling based on hierarchical intent. Figure 2As shown in the figure. First, an intent-aware interest modeling layer is designed to learn diverse user-item representations in each intent level to explore the hierarchical relationship between intents. Specifically, in the interest modeling layer, a cross-attention mechanism is used to measure the response strength of node representations driven by specific intent factors in each dimension and obtain the corresponding intent-aware embedding. Then, multiple interest modeling layers are stacked to extract more abstract and in-depth intent semantics from the lower level to enrich the collaborative filtering signal. Subsequently, an adaptive intent learning strategy is designed in each interest modeling layer to solve the low training efficiency caused by partitioning interactions and alleviate the problem of sparse interactions. Specifically, the traditional attention mechanism needs to obtain the Query matrix from the input features through linear transformation, while the recommendation method based on user interest decoupling of hierarchical intent sets multiple learnable and independent intent vectors as Queries to calculate the attention score of a specific intent. At the same time, the conversion matrix of Key and Value is removed to maintain a lightweight model.

[0009] The invention contents of each main module of this method are as follows:

[0010] 1. Multi-channel interest encoding module

[0011] With the ability of GNN to capture different semantics in different convolutional layers, we extract unified user interest features as the initialization of subsequent interest modules. Although different convolutional layers focus on different aspects of user interests, the practice of convolution on the entire interaction graph essentially encodes the consistency of interests.

[0012] 2. Interest decoupling module based on hierarchical intent

[0013] Based on the lightweight cross-attention mechanism, a novel interest decoupling module is proposed, called the intent-aware interest modeling layer. The method can not only realize intent-guided embedding learning to improve the training efficiency of the model, but also improve the recommendation performance by stacking multiple interest layers to model hierarchical intents. It is assumed that L hierarchical intents drive user interactions and all low-level intents form higher-level intents. Therefore, user intents with high-level concepts are extracted through the bottom-level intents and high-level user / item representations are obtained. In each level, K(l) intents of the same level together constitute the user's interest expression at that level (l=1,2,...,L).

[0014] 3. Recommendation and model training module

[0015] For different intent levels (l=1,2,...,L), the proposed model learns multi-level decoupled embeddings of user u and item i. The inner product is used as the interaction function to match the hierarchical embeddings between user u and item i. Since different levels have different importance in user decision-making, different weights are further used to combine the scores of each level to form the final prediction score, and the items with the highest scores are recommended to the user. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] Figure 1 Example diagram of multiple intents and their hierarchical relationships between specific user-item interactions

[0017] Figure 2 This is the overall framework of the recommendation method for decoupling user interests based on hierarchical intent in the present invention. DETAILED DESCRIPTION

[0018] The present invention discloses a recommendation method based on user interest decoupling based on hierarchical intent. The specific implementation steps of the invention are as follows:

[0019] Step 1: Data preprocessing and division of training and test sets:

[0020] First, select appropriate datasets: Amazon Movies, Last-fm, Movielens-1M, and number the users and items in the datasets in sequence starting from 0. Record the historical user-item interactions in the form of implicit feedback, that is, if the user has interacted with the item (browsing, purchasing, collecting, etc.), the label is 1 (positive sample), and if there has been no interaction, the label is 0 (negative sample). If the interaction of the dataset is in the form of rating, the rated item label can be recorded as 1, otherwise it is 0. After obtaining the processed dataset, for each user, randomly divide the interaction records in a ratio of 8:2 to generate training sets and test sets. Observed user-item interactions are regarded as positive samples, while unobserved interactions are regarded as negative samples.

[0021] Step 2: Generate multi-channel interest encoding embedding:

[0022] Embedding user initialization Item Initialization Embed As the input of the multi-channel interest encoding module, it generates multi-channel input features for the subsequent interest decoupling module for user u and item i respectively. In addition, lightweight graph convolution (LGC) is adopted as the backbone network for multi-channel interest encoding.

[0023] Step 3: Message passing and aggregation mechanism:

[0024] In the aggregation stage, the target node features from the previous layer are further removed so that the nodes in different convolutional layers can focus on their respective semantic information. The detailed process of propagation and aggregation is as follows:

[0025]

[0026] in, Represents the embedding of user u and item i in the cth interest channel. They encode the c-order collaborative filtering signal into the representation (c = 1, 2, ..., C), where C is the maximum order. The value of C with the best performance will be determined through subsequent experiments. N u and N i Represent the first-order neighborhood sets of user u and item i respectively. Symmetric normalization term is the decay factor between user u and item i.

[0027] After obtaining the multi-channel features of nodes containing different collaborative semantics, they are sent as input to the subsequent interest decoupling module. The specific formula is:

[0028]

[0029] in is the multi-channel input feature of user u and item i in the interest decoupling module, which is composed of node embeddings from different GNN layers. It should be noted that and They represent the user and item node representations at the 0th level, which are directly obtained by the interest decoupling module.

[0030] Step 4: Intent setting:

[0031] Set multiple intent vectors as queries and impose constraints on them to guide intent-aware embedding learning. Define and initialize the intent vector at each level as:

[0032]

[0033] Among them, q (l) is a set of intent vectors in the lth level. is the kth intention vector in the lth level (k=1,2,...,K(l)); K(l-1) represents the number of intention vectors in level l-1, l is the current level, and K is the Kth part.

[0034] Step 5: Intent-aware interest modeling:

[0035] We will introduce specific intent vectors and user node representation matrix The process of adaptively generating specific intent-aware embeddings in the first level. First, use As the Key matrix, and measure its correlation with the specific intent vector The inner product is used as the similarity measurement function.

[0036]

[0037] in, Represents the similarity score between the kth intent vector and the representation of all channel nodes of user u. Here, the conversion matrix in the attention mechanism is removed to keep the operation lightweight, and the value of the similarity score is normalized to the range of 0-1 by the sigmoid function to smooth the gradient.

[0038] Characterize the nodes Set as the Value matrix, use the similarity value to weight the multi-channel vectors in the Value matrix to obtain a specific intent-aware embedding. This process can be expressed as: in, is the intent-aware embedding of user u at the first intent-aware interest modeling layer, which extracts the kth intent signal from the multi-channel representation. represents the initialization intent signal of other channels, and k' represents other intent channels. ⊙ is the element-wise product operator. All intent-aware embeddings at this level are obtained by guiding other query vectors. They are fused to obtain the user-item representation in the first intent level. Concatenation is selected as the fusion method, and the specific formula is as follows:

[0039]

[0040] in, represents the node representation of user u at the first level, and K(l) is the lth layer under the Kth part.

[0041] Step 6: Low-level to high-level embedding learning

[0042] By stacking multiple intent-aware interest modeling layers, low-level to high-level embedding learning is performed, interest decoupling is iteratively performed in each layer, and the user intent of the (l-1)th layer is combined to obtain the user representation of the lth layer.

[0043]

[0044] in, represents the user embedding of the lth intent level, d is the embedding dimension, We embed the user intentions at different levels and feed them into the interaction function to fully capture the user preferences. k(l)*dis the user-item interaction matrix at the lth intent level of the kth part. For different intent levels (l=1,2,...,L), we get the multi-level decoupled embedding of user u and multi-level decoupled embedding of item i The inner product is used as the interaction function to match the hierarchical embedding between user u and item i. Since different levels have different importance in user decision making, different weights are used to combine the scores of each level to form the final prediction score.

[0045]

[0046] where α l It represents the importance of the l-th layer score in forming the final prediction score and is an adaptive hyperparameter. The result is sent to the softmax function, which converts the input into a probability distribution, in which the value of each element is between 0 and 1, and the sum of all elements is 1. This value between 0 and 1 is α l For user u, the item set is sorted according to the score Sort from high to low and generate a recommended list

[0047] Step 7: Personalized global enhanced ranking loss:

[0048] The Bayesian personalized ranking (BPR) loss is used to encourage user u to give a higher score to positive sample item i than to unobserved negative sample item j. In addition, inspired by the focus loss for dealing with class imbalance, a personalized global enhancement coefficient is introduced to focus on the mining of weakly discriminative users. The personalized global enhancement ranking loss is expressed as:

[0049]

[0050] σ is the sigmoid function. ω u It is a user-centric global enhancement coefficient, which is determined by the user's activity on the platform, that is, the ratio of the number of interactions of each user to the number of interactions of all users. is the positive result predicted by the model, is the negative result predicted by the model.

[0051] Step 8: Verify the effectiveness of the method:

[0052] After the model training is completed, in order to verify the effectiveness of the method of the present invention, the method of the present invention is tested on the public data sets Amazon Movies, Last-fm and Movielens 1M. After obtaining the target user's interactive prediction score for the recommended items, for each target user, the top 20, 40, 60, 80, 100 items with the highest scores are selected to form a Top-N personalized recommendation list. The evaluation indicators used in the experiment are Recall (the accuracy relative to all positive samples), Precision (the accuracy relative to the Top-N list), Hit Rate (testing the probability of hitting any positive sample in the Top-N list) and Normalized Discounted Cumulative Gain (NDCG focuses on the position of the positive sample in the Top-N recommendation list. The higher the position, the greater the NDCG).

Claims

1. A recommendation method based on user interest decoupling based on hierarchical intent; characterized by: The following steps are involved: S1. Data preprocessing: Obtain a data set, including user ID, item ID, and labels of user-item interactions; for each user, randomly divide the interaction records to generate training and test sets; observed user-item interactions are considered positive examples, while unobserved interactions are considered negative examples; S2, into multi-channel interest coding embedding: initialize the embedding As the input of the multi-channel interest encoding module, it generates multi-channel input features for the subsequent interest decoupling module for user u and item i respectively. Lightweight graph convolution LGC is used as the backbone network for multi-channel interest coding; S3, message passing and aggregation: In the aggregation stage, the target node features from the previous layer are further removed so that the nodes in different convolutional layers can focus on their own semantic information; after obtaining the multi-channel features of the nodes containing different collaborative semantics, they are fed into the subsequent interest decoupling module as input; S4, Intent-aware modeling: multiple intent vectors are set as queries and constraints are imposed on them to guide intent-aware embedding learning; Measure the similarity between the Key matrix and the specific intent vector, using the inner product as the similarity measurement function; Remove the conversion matrix in the attention mechanism to keep the operation lightweight, and normalize the similarity score value to the range of 0-1 through the sigmoid function to smooth the gradient; S5. Low-level to high-level embedding learning: We perform low-level to high-level embedding learning by stacking multiple intent-aware interest modeling layers, iteratively decouple interests in each layer, and combine the user intent of the (l-1)th layer to obtain the user representation of the lth layer. S6. Personalized global enhancement ranking loss: Bayesian personalized ranking (BPR) loss is used to encourage user u to score positive sample item i higher than unobserved negative sample item j. Inspired by the focus loss for dealing with class imbalance problems, a personalized global enhancement coefficient is introduced to focus on mining users with weak discrimination. S7. Generate a recommendation list: After obtaining the predicted user-item interaction score, for each user, all items are sorted in descending order according to the score, and a list of the top N items is generated and recommended to the user.

2. According to claim 1, a recommendation method based on user interest decoupling based on hierarchical intent is characterized in that: The user-item interaction records are recorded in the form of implicit feedback: that is, if the user and the item have interacted, the label is recorded as 1, which is a positive sample, otherwise it is recorded as 0, which is a negative sample. If the interaction in the dataset is in the form of rating, the rated items with a rating greater than or equal to 4 are labeled as 1, which is a positive sample, and the rest are 0, which is a negative sample; the rating is based on a 5-point system.

3. The recommendation method based on user interest decoupling based on hierarchical intent according to claim 1, characterized in that: In the aggregation stage, the target node features from the previous layer are further removed so that the nodes in different convolutional layers can focus on their respective semantic information; the detailed process of propagation and aggregation is as follows: in, represents the embedding of user u and item i in the cth interest channel, encoding the c-order collaborative filtering signal into the representation, c = 1, 2, ..., C; C is the maximum order; N u and N i Represent the first-order neighborhood sets of user u and item i respectively; symmetric normalization term is the decay factor between user u and item i.

4. The recommendation method based on user interest decoupling based on hierarchical intent according to claim 1, characterized in that: After obtaining the multi-channel features of nodes containing different collaborative semantics, they are sent as input to the subsequent interest decoupling module; the specific formula is: in is the multi-channel input feature of user u and item i in the interest decoupling module, which is composed of node embeddings from different GNN layers; it should be noted that and They represent the user and item node representations at the 0th level, which are directly obtained by the interest decoupling module.

5. The recommendation method based on user interest decoupling based on hierarchical intent according to claim 1, characterized in that: Set multiple intent vectors as queries and impose constraints on them to guide intent-aware embedding learning; define and initialize the intent vector at each level as: Among them, q (l) is a set of intention vectors in the lth level; is the kth intention vector in the lth level (k=1, 2, ..., K(l)); K(l-1) represents the number of intention vectors in level l-1, l is the current level, and K is the Kth part.

6. The recommendation method based on user interest decoupling based on hierarchical intent according to claim 1, characterized in that: We will introduce specific intent vectors and user node representation matrix The process of adaptively generating specific intent-aware embeddings in the first level; first using As the Key matrix, and measure its correlation with the specific intent vector The similarity between them is measured using the inner product as the similarity measurement function; in, represents the similarity score between the kth intent vector and the representation of all channel nodes of user u, express Here, the conversion matrix in the attention mechanism is removed to keep the operation lightweight, and the similarity score value is normalized to the range of 0-1 by the sigmoid function to smooth the gradient.

7. The recommendation method based on user interest decoupling based on hierarchical intent according to claim 1, characterized in that: Characterize the nodes Set as the Value matrix, use the similarity value to weight the multi-channel vectors in the Value matrix to obtain a specific intent-aware embedding. This process can be expressed as: in, is the intent-aware embedding of user u at the first intent-aware interest modeling layer, which extracts the kth intent signal from the multi-channel representation; represents the initialization intent signal of other channels, k' represents other intent channels; ⊙ is the element-wise product operator; all intent-aware embeddings at this level are obtained by guiding other query vectors; fusion is used to obtain the user-item representation in the first intent level; splicing is selected as the fusion method, and the specific formula is as follows: in, represents the node representation of user u at the first level, and K(l) is the lth layer under the Kth part.

8. The recommendation method based on user interest decoupling based on hierarchical intent according to claim 1, characterized in that: We stack multiple intent-aware interest modeling layers to learn low-level to high-level embedding, iteratively decouple interests in each layer, and combine the user intent of the (l-1)th layer to obtain the user representation of the lth layer. in, represents the user embedding of the lth intent level, d is the embedding dimension, Embed the user intentions at different levels and feed them into the interaction function to fully capture the user preferences; k(l)*d is the user-item interaction matrix at the lth intention level of the kth part; For different intent levels, l = 1, 2, ..., L, we get the multi-level decoupled embedding of user u and multi-level decoupled embedding of item i Use the inner product as the interaction function to match the hierarchical embedding between user u and item i; since different levels have different importance in user decision-making, different weights are used to combine the scores of each level to form the final prediction score. where α l Indicates the importance of the l-th layer score in forming the final prediction score, which is an adaptive hyperparameter; The result is sent to the softmax function, which converts the input into a probability distribution, in which the value of each element is between 0 and 1, and the sum of all elements is 1. This value between 0 and 1 is α l ; For user u, the item set is sorted according to the score Sort from high to low and generate a recommendation list.

9. The recommendation method based on user interest decoupling based on hierarchical intent according to claim 1, characterized in that: The loss function Bayesian personalized ranking is inspired by the focus loss for dealing with class imbalance problems and introduces a personalized global enhancement coefficient to focus on the mining of weakly discriminative users. The personalized global enhanced ranking loss is expressed as: σ is the sigmoid function; ω u It is a user-centric global enhancement coefficient, which is determined by the user's activity on the platform, that is, the ratio of each user's interaction times to the total number of user interactions; is the positive result predicted by the model, is the negative result predicted by the model.

10. The recommendation method based on user interest decoupling based on hierarchical intent according to claim 1, characterized in that: After the interactive prediction scores of all recommended items are calculated, for each user, all items are sorted in descending order according to the scores, and a list of the top N items is generated and recommended to the user.

Images