A recommendation method based on hierarchical intention decoupling of user interest

Abstract

The application discloses a recommendation method based on user interest decoupling of hierarchical intention. Most of the current fine-grained interest recommendation systems only decouple multiple surface intention factors from the interaction, ignoring the hierarchical relationship between intentions. Secondly, in the process of intention learning, most researches seriously depend on the division of interaction data. This learning paradigm cannot explore potential intentions from a holistic perspective, which may exacerbate data sparsity and damage recommendation performance. In order to explore the hierarchical relationship between intentions, an intention-aware interest modeling layer is designed to learn diversified user / item representations in each intention hierarchy. Then, multiple interest modeling layers are stacked to extract more abstract and deep intention semantics from low-level to enrich the collaborative filtering signal. In addition, in order to avoid low training efficiency caused by dividing interactions and alleviate the problem of interaction sparsity, an adaptive intention learning strategy is also designed in each interest modeling layer.

Description

Technical Field

[0001] This invention relates to collaborative filtering technology, specifically model-based collaborative filtering algorithms; it relates to graph convolution-based representation learning methods, specifically GNN algorithms; it relates to attention-based learning methods; and it also relates to deep learning and other related technologies. Background Technology

[0002] With the widespread adoption of the internet, a vast amount of information floods the network. While this information satisfies users' needs, the exponential growth in the amount of information makes it impossible for users to efficiently and quickly select the information they need, thus reducing information utilization. This phenomenon is known as information overload. To alleviate information overload and improve information utilization, recommendation systems have emerged.

[0003] A recommender system is a system that uses users' historical behavioral data and personal preferences to recommend items or services that users may be interested in. The goal of a recommender system is to provide personalized recommendations based on users' needs and interests, helping them find items or services that match their preferences more quickly, thereby improving user experience and satisfaction. Content-based recommendation algorithms (CB) are a classic type of recommender algorithm. Specifically, CB algorithms have the following advantages: First, users are independent of each other. The construction of each user's features depends only on their own preferences for items, and is unrelated to the behavior of others. Second, it is easy to interpret. In certain specific scenarios, the recommender system needs to explain to the user why a certain item is recommended. This can be achieved by simply telling the user that the recommended item has certain attributes that frequently appear among the items the user likes. Finally, it is not constrained by new users or new items. When a new user enters the recommender system, content-based recommendations can be made based on the user's personal attribute information.

[0004] After content-based recommendation algorithms were proposed, researchers found that these algorithms had significant limitations. For example, movies of the same genre, or even those directed by the same person, can vary greatly in content and quality, and users' interests in them can also differ significantly. Simply relying on content for recommendations leads to low accuracy. To address these issues, collaborative filtering was innovatively proposed. It analyzes historical user-item interactions to predict the next item a user might interact with, fully utilizing existing user interaction data to identify content that best satisfies the user's interests.

[0005] Current deep learning methods are closely related to the methods mentioned above. Deep learning-based recommendation algorithms attempt to model data types that traditional algorithms cannot model, hoping to improve the performance of recommendation algorithms by introducing more information. The rapid development of deep learning-related technologies has also spurred a large number of new research directions in recommendation problems, greatly promoting the development of the recommendation algorithm field. For example, some research has begun to explore the fine-grained composition of users' interests, mining multiple intentions of user interactions through decoupling or multi-view representation. However, current fine-grained interest recommendation systems have limitations. Most fine-grained interest-based methods only decouple multiple surface intention factors from interactions, ignoring the hierarchical relationships between intentions. To some extent, existing methods, especially in the case of sparse data, struggle to extract rich collaborative signals from surface intentions, resulting in poor quality interest modeling.

[0006] This invention addresses this issue by proposing a recommendation method based on the decoupling of user interests from hierarchical intent. Summary of the Invention

[0007] Current fine-grained interest recommendation systems suffer from two key limitations. First, most fine-grained interest-based methods merely decouple various surface intent factors from interactions, neglecting the hierarchical relationships between intents. To some extent, existing methods, especially in data-sparse scenarios, struggle to extract rich collaborative signals from surface intents, leading to poor interest modeling quality, such as... Figure 1 As shown. Secondly, user decisions reveal intents at different levels. Lower-level intents focus on general and superficial user preferences, while higher-level intents focus on deep and abstract preferences. Therefore, an intent hypothesis is proposed: user interests are highly correlated with multi-level hierarchical intents. Considering the hierarchical relationship between intents, fine-grained interests can be modeled more accurately, and user decisions in real-world scenarios can be better understood and interpreted. Furthermore, in the process of intent learning, most studies heavily rely on the partitioning of interaction data. This learning paradigm cannot explore potential intents from a holistic perspective, potentially exacerbating data sparsity and impairing recommendation performance. Moreover, whether it's hard or soft partitioning, recommendation models require more time to find the optimal partitioning ratio based on the task, significantly reducing the model's training efficiency.

[0008] To address the aforementioned problems, this invention discloses a recommendation method based on hierarchical intent-based user interest decoupling. The overall framework is as follows: Figure 2As shown, firstly, an intent-aware interest modeling layer is designed to learn diverse user item representations at each intent level to explore hierarchical relationships between intents. Specifically, in the interest modeling layer, a cross-attention mechanism is used to measure the response strength of node representations driven by specific intent factors in each dimension and obtain the corresponding intent-aware embeddings. Then, multiple interest modeling layers are stacked to extract more abstract and in-depth intent semantics from lower levels to enrich the collaborative filtering signal. Subsequently, an adaptive intent learning strategy is designed in each interest modeling layer to address the low training efficiency caused by interaction segmentation and alleviate the interaction sparsity problem. Specifically, traditional attention mechanisms require linear transformation to obtain the Query matrix from input features, while the user interest decoupling recommendation method based on hierarchical intents sets multiple learnable and independent intent vectors as the Query to calculate the attention score for a specific intent. Simultaneously, the key and value transformation matrices are removed to maintain a lightweight model.

[0009] The invention details of each main module of this method are as follows:

[0010] 1. Multi-channel interest coding module

[0011] By leveraging the ability of GNNs to capture different semantics across different convolutional layers, unified user interest features are extracted as initialization for subsequent interest modules. Although different convolutional layers focus on different aspects of user interests, the practice of performing convolutions across the entire interaction graph essentially encodes the consistency of interests.

[0012] 2. Interest decoupling module based on hierarchical intent

[0013] A novel interest decoupling module, called the Intent-Aware Interest Modeling Layer, is proposed based on a lightweight cross-attention mechanism. This method not only improves model training efficiency through intent-guided embedding learning but also enhances recommendation performance by stacking multiple interest layers to model hierarchical intents. It is assumed that L hierarchical intents drive user interactions, and all lower-level intents form higher-level intents. Therefore, high-level user / item representations are obtained by extracting user intents with high-level concepts from lower-level intents. In each level, K(l) intents of the same level collectively constitute the user's interest expression at that level (l=1,2,...,L).

[0014] 3. Recommendation and Model Training Module

[0015] For different intent levels (l = 1, 2, ..., L), the proposed model learns multi-level decoupled embeddings of user u and item i. The inner product is used as the interaction function to match the hierarchical embeddings between user u and item i. Since different levels have different importance in user decision-making, different weights are further used to combine the scores of each level to form the final predicted score, recommending the item with the highest score to the user. Attached Figure Description

[0016] Figure 1 Example diagram of multiple intents and their hierarchical relationships between specific user-item interactions.

[0017] Figure 2 This is the overall framework of the recommendation method based on hierarchical intent decoupling of user interests in this invention. Detailed Implementation

[0018] This invention discloses a recommendation method based on hierarchical intent-based user interest decoupling. The specific implementation steps of this invention are as follows:

[0019] Step 1: Data preprocessing and partitioning of training and test sets:

[0020] First, select suitable datasets: Amazon Movies, Last-fm, and Movielens-1M. Number the users and items in the datasets sequentially, starting from 0. Record the historical user-item interactions using implicit feedback: if a user has interacted with an item (browsing, purchasing, adding to favorites, etc.), the label is 1 (positive sample); otherwise, the label is 0 (negative sample). If the dataset's interactions are in the form of ratings, items that have been rated are labeled 1, and those that haven't are labeled 0. After obtaining the processed dataset, for each user, randomly split the interaction records in an 8:2 ratio to generate the training and test sets. Observed user-item interactions are considered positive samples, while unobserved interactions are considered negative samples.

[0021] Step 2: Generate multi-channel interest coding embeddings:

[0022] Embed user initialization Item initialization embedding This serves as the input to the multi-channel interest coding module, and generates multi-channel input features for the subsequent interest decoupling module for user u and item i, respectively. In addition, lightweight graph convolution (LGC) is used as the backbone network for multi-channel interest coding.

[0023] Step 3: Message passing and aggregation mechanisms:

[0024] In the aggregation stage, features from the target nodes in the previous layer are further removed so that nodes in different convolutional layers can focus on their respective semantic information. The detailed process of propagation and aggregation is as follows:

[0025]

[0026] in, The embeddings of user u and item i in the c-th interest channel represent the encoding of c-th order collaborative filtering signals into the representation (c = 1, 2, ..., C), where C is the maximum order. The best-performing value of C will be determined through subsequent experiments. N u and N i Let represent the first-order neighborhood sets of user u and item i, respectively. Symmetric normalization term. It is the decay factor between user u and item i.

[0027] After obtaining multi-channel features of nodes containing different collaborative semantics, these features are fed as input into the subsequent interest decoupling module. The specific formula is as follows:

[0028]

[0029] in This represents the multi-channel input features of user u and item i in the interest decoupling module, composed of node embeddings from different GNN layers. It is important to note that... and These represent the user and item node representations at the 0th level, which are obtained directly from the interest decoupling module.

[0030] Step 4: Intent Setting:

[0031] Multiple intent vectors are set as queries, and constraints are imposed on them to guide intent-aware embedding learning. The intent vectors at each level are defined and initialized as follows:

[0032]

[0033] Where, q (l) It is a set of intent vectors in the l-th level. It is the k-th intention vector in the l-th level (k = 1, 2, ..., K(l)); K(l-1) represents the number of intention vectors in the l-1 level, l is the current level, and K is the K-th part.

[0034] Step 5: Intent-Aware Interest Modeling

[0035] This will introduce specific intent vectors. and user node representation matrix This stems from the process of adaptively generating a specific intent-aware embedding in the first level. First, using... As a key matrix, it is measured against a specific intent vector. The similarity between them is calculated, and the inner product is used as the similarity metric function.

[0036]

[0037] in, Let represent the similarity score between the k-th intent vector and the user u across all channel node representations. Here, the transformation matrix in the attention mechanism is removed to maintain lightweight computation, and the similarity score is normalized to the 0-1 range using the sigmoid function to smooth the gradient.

[0038] Node representation Set as the Value matrix, and use the similarity values ​​to perform a weighted sum of the multi-channel vectors in the Value matrix to obtain a specific intent-aware embedding. This process can be expressed as: in, It is the intent-aware embedding of user u in the first intent-aware interest modeling layer, which extracts the k-th intent signal from the multi-channel representation. This represents the initial intent signal for other channels, where k' represents other intent channels. ⊙ is the element-wise multiplication operator. All intent-aware embeddings at this level are obtained by guiding other query vectors. These are then fused to obtain the user-item representation in the first intent level. Concatenation is chosen as the fusion method, and the specific formula is as follows:

[0039]

[0040] in, Let K(l) represent the node representation of user u in the first level, and K(l) represent the l level in the Kth part.

[0041] Step Six: Low-Level to High-Level Embedding Learning

[0042] By stacking multiple intent-aware interest modeling layers, low-level to high-level embedding learning is performed. Interest decoupling is performed iteratively in each layer, and the user intent of the (l-1)th layer is combined to obtain the user representation of the lth layer.

[0043]

[0044] in, This represents the user embedding at the l-th intent level, where d is the embedding dimension. By embedding user intent perception at different levels, these are fed into interaction functions to comprehensively capture user preferences. k(l)*dLet be the user-item interaction matrix for the k-th part and the l-th intent level. For different intent levels (l = 1, 2, ..., L), we obtain the multi-level decoupled embedding of user u. Multi-level decoupling embedding of item i The inner product is used as the interaction function to match the hierarchical embedding between user u and item i. Since different levels have different importance in user decision-making, different weights are used to combine the scores of each level to form the final predicted score.

[0045]

[0046] Where α l The value representing the importance of the l-th layer score in constructing the final predicted score is an adaptive hyperparameter. The result is fed into the softmax function, which transforms the input into a probability distribution where each element's value is between 0 and 1, and the sum of all elements is 1. This value between 0 and 1 is α. l For user u, the item set is sorted by score. Sort by value from highest to lowest and generate a recommendation list.

[0047] Step 7: Personalize the global augmentation ranking loss:

[0048] The Bayesian Personalized Ranking (BPR) loss is used to encourage user u to score positive sample item i higher than unobserved negative sample item j. Furthermore, inspired by the focusing loss used to address class imbalance, a personalized global augmentation coefficient is introduced to focus on mining users with weak discrimination. The personalized global augmentation ranking loss is expressed as:

[0049]

[0050] σ is the sigmoid function. ω u It is a user-centric global enhancement coefficient, determined based on user activity on the platform, that is, the ratio of the number of interactions per user to the total number of interactions per user. These are positive examples predicted by the model. These are negative examples predicted by the model.

[0051] Step 8: Verify the effectiveness of the method:

[0052] After model training, to verify the effectiveness of the proposed method, it was tested on public datasets Amazon Movies, Last-fm, and Movielens 1M. After obtaining the interaction prediction scores of target users for recommended items, a Top-N personalized recommendation list was formed for each target user by selecting the top 20, 40, 60, 80, and 100 items with the highest scores. The evaluation metrics used in the experiment were Recall (accuracy relative to all positive samples), Precision (accuracy relative to the Top-N list), Hit Rate (the probability of hitting any positive sample in the Top-N list), and Normalized Discounted Cumulative Gain (NDCG), which focuses on the position of a positive sample in the Top-N recommendation list; the earlier the position, the larger the NDCG.

Claims

1. A recommendation method based on hierarchical intention decoupling of user interest; characterized in that: Comprising the following steps: S1, data preprocessing: obtaining a data set including user ID, item ID, and user-item interaction label; for each user, randomly dividing the interaction records to generate a training set and a test set; the observed user-item interaction is regarded as a positive example, while the unobserved interaction is regarded as a negative example; S2, multi-channel interest encoding embedding: generating an initialization embedding as the input of the multi-channel interest encoding module and generating the multi-channel input features of the subsequent interest decoupling module for user u and item i respectively adopting light graph convolution LGC as the backbone network of the multi-channel interest encoding S3, message passing and aggregation: further removing the target node features from the previous layer in the aggregation stage, so that the nodes in different convolution layers can focus on the respective semantic information; after obtaining the multi-channel features of the nodes containing different collaborative semantics, they are sent as input to the subsequent interest decoupling module; S4, intent-aware modeling: setting multiple intent vectors as queries and applying constraints to them to guide intent-aware embedding learning; Measuring the similarity between the Key matrix and the specific intent vector, using the inner product as the similarity measure function; Removing the conversion matrix in the attention mechanism to maintain lightweight operation, and normalizing the value of the similarity score to the interval of 0-1 through the sigmoid function to smooth the gradient; S5, low-level to high-level embedding learning: low-level to high-level embedding learning is performed by stacking multiple intent-aware interest modeling layers, iteratively decoupling interest in each layer, and combining the user intent of the (l-1) layer to obtain the user representation of the l layer; S6, personalized global enhanced ranking loss: using Bayesian personalized ranking (BPR) loss to encourage the score of user u for the positive example item i to be higher than the unobserved negative example item j; inspired by the focus loss for handling class imbalance problems, a personalized global enhancement coefficient is introduced to focus on weakly differentiated user mining; S7, generating a recommendation list: after obtaining the predicted user-item interaction score, for each user, all items are sorted in descending order according to the score, and the top N items are recommended to the user.

2. The recommendation method based on hierarchical intent user interest decoupling according to claim 1, characterized in that: The user-item interaction record is recorded in the form of implicit feedback: that is, the user and the item have interacted, and the label is marked as 1, which is a positive sample, otherwise as 0, which is a negative sample; if the interaction of the data set is in the form of rating, the rated items and the rating greater than or equal to 4 are marked as 1, which is a positive sample, and the rest are marked as 0, which is a negative sample; the rating is a 5-point system.

3. The recommendation method based on hierarchical intent user interest decoupling according to claim 1, characterized in that: Further removing the target node features from the previous layer in the aggregation stage, so that the nodes in different convolution layers can focus on the respective semantic information; the detailed process of propagation and aggregation is as follows: where, represents the embedding of user u and item i in the c-th interest channel, c = 1, 2, …, C; C is the maximum order; N u and N i respectively represent the first-order neighborhood set of user u and item i; the symmetric normalization term is the decay factor between user u and item i.

4. The recommendation method based on hierarchical intent user interest decoupling according to claim 1, characterized in that: After obtaining the multi-channel features of the nodes containing different collaborative semantics, they are sent as input to the subsequent interest decoupling module; the specific formula is: wherein is the multi-channel input feature of user u and item i in the interest decoupling module, composed of node embeddings from different GNN layers; it is worth noting that and respectively represent the user and item node representations of the 0th level, that is, directly obtained from the interest decoupling module.

5. The recommendation method based on hierarchical intent user interest decoupling according to claim 1, characterized in that: The plurality of intention vectors are set as queries, and constraints are imposed thereon to guide intention-aware embedding learning; the intention vectors under each level are defined and initialized as: where q (l) is a set of intent vectors in the lth hierarchy level; is the kth intent vector in the lth hierarchy level, k = 1, 2,..., K(l); K(l-1) represents the number of intent vectors in the hierarchy level l-1, l is the current hierarchy level, and K is the Kth part.

6. The recommendation method based on user interest decoupling according to hierarchical intention according to claim 1, characterized in that, Introduce the specific intent vector and the user node representation matrix Adaptive process to generate the specific intent-aware embedding in the first level; where k = 1, 2, …, K(l); first use as the Key matrix, and measure the similarity between it and the specific intent vector , and use the inner product as the similarity measure function; where, represents the similarity score of the kth intent vector with all the channel node representations of user u, represents the transpose; here, the transition matrix in the attention mechanism is removed to keep the lightweight operation, and the value of the similarity score is normalized to the interval of 0-1 by the sigmoid function to smooth the gradient.

7. The recommendation method based on user interest decoupling according to hierarchical intention according to claim 6, characterized in that, Characterizing nodes is set to the Value matrix, and the multi-channel vectors in the Value matrix are weighted and summed using the values of the similarity to obtain a specific intent-aware embedding, which can be expressed as: wherein, is the intent-aware embedding of the user u in the first intent-aware interest modeling layer, which extracts the kth intent signal from the multi-channel representation; denotes the initialized intent signal of other channels, and k' denotes other intent channels; is an element-wise multiplication operator; all intent-aware embeddings at this level are obtained by guiding through other query vectors; the user-item representation in the first intent level is obtained by fusion; the splicing is selected as the fusion mode, and the specific formula is as follows: wherein, represents the node representation of user u at the first level, K(l) is the l-th layer at the K-th part.

8. The recommendation method based on user interest decoupling according to hierarchical intention according to claim 6, characterized in that, The embedding learning from low level to high level is performed by stacking a plurality of intention-aware interest modeling layers, the interest decoupling is iteratively performed in each layer, and the user representation of the lth layer is obtained by combining the user intention of the (l-1)th layer; wherein, user embedding representing the lth intent hierarchy, d is the embedding dimension, are the perceived embeddings of user intent at different hierarchies, which are fed into the interaction function to capture user preferences comprehensively; R k(l)*d is the user-item interaction matrix at the kth part and the lth intent hierarchy. For different intent levels, l = 1, 2,..., L, get the multi-level decoupled embedding of user u and the multi-level decoupled embedding of item i Use inner product as interaction function to match the hierarchical embedding between user u and item i; since different levels have different importance in user decision, use different weights to combine the score of each level to form the final prediction score, where α l represents the importance of the l-th layer score in the final prediction score and is an adaptive hyper-parameter; and the result of the softmax function is a probability distribution, where each element's value is between 0 and 1, and the sum of all elements is 1, and the value between 0 and 1 is α l ; for user u, the item set is sorted according to the score from high to low to generate a recommendation list.

9. The recommendation method based on user interest decoupling according to hierarchical intention according to claim 1, characterized in that, The loss function Bayesian personalized ranking is inspired by the focused loss for dealing with the class imbalance problem, and a personalized global enhancement coefficient is introduced to pay attention to the weak discriminant user mining; The personalized global enhancement ranking loss is represented as: σ is a sigmoid function; ω u is a user-centered global enhancement coefficient determined according to the activity of the user on the platform, i.e. the ratio of the number of interactions of each user to the number of interactions of all users; is a positive example result predicted by the model, is a negative example result predicted by the model.

10. The recommendation method based on user interest decoupling according to hierarchical intention according to claim 1, characterized in that, After the interaction prediction scores of all the recommended items are obtained, for each user, all the items are sorted in descending order according to the scores, and the first N items are generated to form a list and recommended to the user.

Images