Recommendation method based on implicit hierarchical interest of user

Abstract

The invention discloses a recommendation method based on user implicit hierarchical interests. Most of existing graph-based recommendations are to perform graph learning in a traditional Euclidean space, but when a hierarchical structure is learned from a graph, the grid characteristics of the Euclidean space can introduce serious structural distortion. The hyperbolic geometry is applicable to modeling data of potential non-Euclidean topologies as compared to Euclidean geometry. The invention provides a recommendation method based on user implicit and hierarchical interests, which comprises the following steps: firstly, mining the implicit and hierarchical interests of a user in an interaction graph by using a hyperbolic graph neural network, and designing self-distillation loss to embed the hierarchical interests into Euclidean representation learning to improve the model reasoning efficiency; the effectiveness and reasoning efficiency of the method are verified through experimental results on a plurality of data sets. According to the method, the public data sets are used for training and testing to optimize the performance of the model, and the effectiveness of the method is verified.

Description

Technical Field

[0001] The present invention relates to a knowledge distillation model, specifically a geometrically enhanced self-distillation model; the present invention relates to information transmission technology based on graph convolutional networks; the present invention relates to hyperbolic geometry technology, specifically a Poincaré sphere model for representation learning; the present invention also relates to data mining and deep learning and other related technologies. Background Art

[0002] With the development of the Internet and the explosive growth of information, users are faced with a vast amount of information and choices. Traditional information retrieval methods can no longer meet user needs. Recommendation systems can filter information for users through personalized recommendations, reduce the trouble of information overload, and improve the efficiency of users' acquisition of information, thereby better meeting user needs. Recommendation systems are widely used in e-commerce, social networks, music, video, news and other fields, and have become an important tool to help users discover more valuable information.

[0003] The recommendation paradigm based on collaborative filtering (CF) usually maps users and items to a latent space through embedding learning, and uses an interaction function based on the learned embedding to predict the user-item interaction score. Early models such as matrix factorization only modeled a single user-item pair and ignored the CF signal between interactions. Therefore, recent studies have conducted in-depth research on structured signals in interaction graphs. They use graph neural networks to effectively capture high-order CF signals in user-item interaction graphs, and this technology has been shown to significantly improve recommendation performance. In addition to the traditional flat graph structure, the interaction graph presents a tree structure. This tree structure is manifested in that all items in the interaction data present a power-law degree distribution, so the graph structure shows an obvious hierarchical structure. Although graph learning in traditional Euclidean space is computationally good at capturing local structural signals, when learning hierarchical structures from graphs, the grid characteristics of Euclidean space introduce serious structural distortions, such as Figure 1 shown.

[0004] Compared with Euclidean geometry, hyperbolic geometry is suitable for modeling data with potential non-Euclidean topology. The advantages of hyperbolic geometry are as follows: 1) Hyperbolic geometry can embed a complex structure graph in a smaller space, which means that hyperbolic embedding can capture structured information with a lower-dimensional representation; 2) Hyperbolic space can extract the tree structure in the graph and retain the global hierarchical structure. In order to take advantage of the above advantages, some researchers apply graph neural networks to hyperbolic space. Hierarchical signals are mined from user interactions to complete interest modeling, but these studies are still insufficient to capture complex geometric structures from interaction graphs. Hierarchical signals can only describe part of user interests, resulting in suboptimal performance. In addition, the inevitable nonlinear transformations in hyperbolic space will bring huge computational costs and impose a great burden on model reasoning, which is unbearable in real-world scenarios with millions of users.

[0005] In order to take advantage of the complementarity of the two, an intuitive way is to extend the network structure from Euclidean space and hyperbolic space to learn user interests. In order to make the two work together, following the knowledge transfer paradigm, a recommendation method based on user implicit hierarchical interests is proposed. Summary of the invention

[0006] The interactive graph data in the recommendation scenario presents a tree structure. This tree structure is manifested in that all items in the interactive data present a power-law degree distribution, so the graph structure shows an obvious hierarchical structure. The hierarchical structure in the user-item interaction graph also implies the user hierarchical interests hidden in the structured data. Most existing methods use graph convolutional networks in Euclidean space to capture user interests, but when they learn hierarchical graph structures, they will be severely distorted, making it difficult to accurately mine the user hierarchical interests hidden in the structure, which in turn affects the recommendation performance. Other studies combine the ability of hyperbolic space representation to learn hierarchical structures with graph convolutional networks and apply them to recommendation systems to mine hierarchical relationships. However, such models often have a long reasoning time during interactive learning, which is unbearable in real scenarios with millions of users. The optimization strategy module can affect the quality of representation learning to achieve accurate interest modeling; it can also affect the form of interactive learning, which plays an important role in model reasoning efficiency. Therefore, how to effectively incorporate users' hierarchical interests into the optimization strategy is of great significance for accurately modeling user interests and reducing model reasoning efficiency.

[0007] In order to explore how to integrate the relationship between interests into the optimization strategy, the present invention discloses a recommendation method based on the user's implicit hierarchical interests. Figure 2As shown. The datasets used in this method are all public datasets: 1) social network: Gowalla, 2) e-commerce platform: Amazon-book, 3) review website: Yelp2018). This method uses a parameter-sharing self-teacher network to transfer the implicit hierarchical interests mined from hyperbolic geometry as knowledge to the backbone student network in Euclidean space to enrich collaborative filtering signals and strengthen interest modeling. At the same time, it abandons the use of feature fusion methods in representation learning, but chooses to design a response-based adaptive restriction in the optimization strategy to transfer hierarchical knowledge in hyperbolic space to Euclidean space to improve the efficiency of model reasoning. The invention consists of two key components: 1) Interest propagation layer, which uses graph convolutional network (GCNGCN) to extract high-order neighborhood information in Euclidean space and hyperbolic space respectively. 2) Prediction and optimization, which uses metrics in different spaces to calculate the interaction scores of user-item pairs, and feeds them into recommendation loss and self-distillation loss, implements geometric enhanced embedding learning in Euclidean space, and mines complex geometric structure signals from interaction graphs to comprehensively learn user interests.

[0008] The invention contents of each module of this method are as follows:

[0009] 1. Interest propagation layer

[0010] First, the representations of users and items are mapped to the hyperbolic space through exponential mapping, and the corresponding interest propagation layer is established as a self-teacher network to naturally capture the hierarchical structure. Second, the hyperbolic embeddings of users and items are mapped to the tangent space, which is approximately Euclidean space. Then, traditional GCN aggregation is performed in the tangent space and projected back to the hyperbolic space using exponential mapping to obtain the hyperbolic representation during interest propagation.

[0011] 2. Prediction and Optimization

[0012] After L layers of interest propagation, the embeddings of each layer can be obtained in different spaces, and these embeddings contain different structural information. In order to obtain better quality representation, average pooling is used to fuse collaborative filtering signals from multiple layers. For hyperbolic embedding, hyperbolic distance is used to evaluate the similarity between user-item pairs. In order to combine the recommendation task with the structured knowledge transfer task, a multi-task training strategy is used to optimize the entire model. BRIEF DESCRIPTION OF THE DRAWINGS

[0013] Figure 1 This is an example diagram of the hierarchical relationship implied in user interaction and the interaction graph structure in Euclidean space and hyperbolic space.

[0014] Figure 2 This is the overall framework of the recommendation method based on user implicit hierarchical interests of the present invention. DETAILED DESCRIPTION

[0015] The present invention discloses a recommendation method based on user's implicit hierarchical interests. The specific implementation steps of the invention are as follows:

[0016] Step 1: Data preprocessing and division of training and test sets:

[0017] First, select appropriate datasets: 1) social network: Gowalla, 2) e-commerce platform: Amazon-book, 3) review website: Yelp2018), number the users and items in the dataset in order starting from 0, and record the historical user-item interactions in the form of implicit feedback, that is, if the user has interacted with the item (browsing, purchasing, collecting, etc.), the label is 1 (positive sample), if there has been no interaction, the label is 0 (negative sample). In these three datasets, for each user, the interaction records are randomly divided in a ratio of 4:1 to generate training sets and test sets. Observed user-item interactions are regarded as positive samples, while unobserved interactions are regarded as negative samples.

[0018] Step 2: Hyperbolic space interest propagation:

[0019] First, the representations of users and items {E u ,E i} is mapped to the hyperbolic space {H u ,H i}, and establish the corresponding interest propagation layer as a self-teacher network to naturally capture the hierarchical structure. The interest propagation mechanism in the hyperbolic space is defined as follows:

[0020]

[0021] Among them, N u represents the set of neighbor nodes of user u, N i represents the set of adjacent nodes of item i, and Represent the hyperbolic embedding of user u and item i in the lth interest propagation layer. B (·) represents the d-dimensional hyperbolic space B d The interest propagation operation in , which extracts and aggregates the information of the target node and its neighbors in the hyperbolic manifold.

[0022] Aggregation operation g B The specific process of (·) is as follows:

[0023]

[0024] Where exp0 and log0 are the exponential mapping and logarithmic mapping functions mentioned above. In order to avoid complex averaging operations in hyperbolic space, the hyperbolic embeddings of users and items are first mapped to tangent space, which is approximately Euclidean space. Then, traditional GCN aggregation is performed in tangent space and projected back to hyperbolic space using exponential mapping to obtain the hyperbolic representation of interest propagation at the lth layer. and Therefore, this aggregation operation can integrate the neighborhood information into the tangent representation in the hyperbolic space through the spatial mapping function, avoiding complex hyperbolic average pooling.

[0025] Step 3: Layer combination and prediction:

[0026] After L layers of interest propagation, the embeddings of each layer can be obtained in different spaces, and these embeddings contain different structural information. In order to obtain a better quality representation, average pooling is used to fuse collaborative filtering signals from multiple layers. When combining layers in the hyperbolic space, the same mapping strategy as the interest propagation module is used to extract information. In addition, this method removes the feature vectors of the initial input layer and retains the skip connection structure from all intermediate layers to the final layer to obtain better recommendation performance. The layer combination operation in different spaces is expressed as:

[0027]

[0028]

[0029] in, and Represents the final embedding of user u and item i in Euclidean space and hyperbolic space respectively.

[0030] Step 4: Recommended prediction:

[0031] For Euclidean embedding, the inner product operation is used as the prediction function to measure the similarity between users and items, which is widely used in collaborative filtering tasks. For hyperbolic embedding, the hyperbolic distance is used to evaluate the similarity between user-item pairs. Specifically, the similarity between two embeddings is the inverse of the geodesic distance connecting the two points:

[0032]

[0033]

[0034] Among them, d B is the geodesic distance function mentioned above. and Represent the prediction scores in Euclidean space and hyperbolic space, respectively.

[0035] Step 5: Self-distillation and model training:

[0036] In order to combine the recommendation task with the structured knowledge transfer task, a multi-task training strategy is used to optimize the entire model. For the recommendation task, pairwise Bayesian personalized ranking (BPR) is adopted as the main loss function, which encourages the score of user u and the positive sample item i interacted with it to be higher than the unobserved negative sample item j.

[0037]

[0038] Among them, O={(u,i,j)|(u,i)∈O + ,(u,i)∈O -},O + represents the set of observed interactions between user u and item i, O - represents the set of unobserved interactions between user u and item j. σ is the sigmoid function.

[0039] Intuitively, the predicted score directly reflects the user's preference for items, regardless of spatial inconsistency. Therefore, inspired by response-based knowledge distillation, an adaptive self-distillation loss is innovatively designed as an auxiliary task, whose purpose is to use the predicted score in the hyperbolic space as a soft label and migrate the missing hierarchical information to the Euclidean space, so that the score in the Euclidean space can not only reflect the hierarchical structure in the data, but also strengthen the score distribution in the Euclidean space through joint learning with the main function. Specifically, the geometrically enhanced self-distillation based on score response is achieved by minimizing the following loss function:

[0040]

[0041] Among them, σ is the sigmoid function. τ is the temperature factor, which is used to adjust the output of the sigmoid function, thereby affecting the final recommendation result. By combining the adaptive self-distillation loss with the BPR loss, the following objective function is minimized to optimize the model parameters:

[0042] loss=loss Rec +λloss SD

[0043] λ controls the self-distillation loss and adjusts this parameter to minimize the loss.

[0044] Step 6: Verify the effectiveness of the method:

[0045] After the model training is completed, in order to verify the effectiveness of the method of the present invention, the method of the present invention is applied to the public data sets: 1) social network: Gowalla, 2) e-commerce platform: Amazon-book, 3) review website: Yelp2018). After obtaining the interactive prediction scores of the target users for the recommended items, for each target user, the top 20, 40, 60, 80, 100 items with the highest scores are selected to form a Top-N personalized recommendation list. The evaluation indicators used in the experiment are Recall (the accuracy relative to all positive samples), Precision (the accuracy relative to the Top-N list), Hit Rate (testing the probability of hitting any positive sample in the Top-N list) and Normalized Discounted Cumulative Gain (NDCG focuses on the position of the positive sample in the Top-N recommendation list. The higher the position, the greater the NDCG).

Claims

1. A recommendation method based on user's implicit hierarchical interests; characterized by: The following steps are involved: S1. Data preprocessing: Obtain a data set, including user ID, item ID, and labels of user-item interactions; divide into training set and test set; S2. Interest propagation: First, the representations of users and items are mapped to hyperbolic space through exponential mapping, and the corresponding interest propagation layer is established as a self-teacher network to naturally capture the hierarchical structure; secondly, the hyperbolic embeddings of users and items are mapped to tangent space, which is approximately Euclidean space; Then, traditional GCN aggregation is performed in the tangent space and projected back to the hyperbolic space using exponential mapping to obtain the hyperbolic representation of interest propagation; S3, inter-layer aggregation: After L layers of interest propagation, L = 3, the embeddings of each layer are obtained in different spaces, and these embeddings contain different structural information; average pooling is used to fuse collaborative filtering signals from multiple layers; when combining layers in hyperbolic space, the same mapping strategy as the interest propagation module is used to extract information; the feature vectors of the initial input layer are removed, and the skip connection structure from all intermediate layers to the final layer is retained; S4, recommendation prediction: For hyperbolic embedding, the hyperbolic distance is used to evaluate the similarity between user-item pairs; the similarity between two embeddings is the inverse of the geodesic distance connecting the two points; S5. Optimization method and loss function: The loss function consists of two parts: one is the BPR loss function used for recommendation prediction and the L2 regularization term composed of initialization parameters; the other is the adaptive self-distillation loss, and the two are jointly optimized. S6. Generate a recommendation list: After obtaining the predicted user-item interaction score, for each user, all items are sorted in descending order according to the score, and a list of the top N items is generated and recommended to the user.

2. A recommendation method based on user implicit hierarchical interests according to claim 1, characterized in that: The user-item interaction records are recorded in the form of implicit feedback: that is, if the user and the item have interacted, the label is recorded as 1, which is a positive sample, otherwise it is recorded as 0, which is a negative sample; for each data set, the ratio of training set to test set is 4:

1.

3. The recommendation method based on user implicit hierarchical interests according to claim 1, characterized in that: The representation of users and items {E u ,E i } is mapped to the hyperbolic space {H u ,H i }, and establish the corresponding interest propagation layer as a self-teacher network to naturally capture the hierarchical structure; the interest propagation mechanism in the hyperbolic space is defined as follows: in, and Represent the hyperbolic embedding of user u and item i in the lth interest propagation layer respectively; g B (·) represents the d-dimensional hyperbolic space B d The interest propagation operation in N extracts and aggregates the information of the target node and its neighborhood in the hyperbolic manifold; u Represents the neighbor node set of user u, which is composed of the ID information of neighbor users, N i Represents the adjacent node set of item i, which is composed of the ID information of neighboring items; Aggregation operation g B The specific process of (·) is as follows: Among them, exp0 and log0 are the exponential mapping and logarithmic mapping functions mentioned above; in order to avoid complex averaging operations in hyperbolic space, the hyperbolic embeddings of users and items are first mapped to the tangent space, which is approximately Euclidean space; then, traditional GCN aggregation is performed in the tangent space and projected back to the hyperbolic space using the exponential mapping to obtain the hyperbolic representation of the interest propagation at the lth layer and 4. The recommendation method based on user implicit hierarchical interests according to claim 1, characterized in that: The layer combination operation in different spaces is expressed as: in, and Represents the final embedding of user u and item i in Euclidean space and hyperbolic space respectively.

5. The recommendation method based on user implicit hierarchical interests according to claim 1, characterized in that: For Euclidean embedding, the inner product operation is used as the prediction function to measure the similarity between users and items, which is widely used in collaborative filtering tasks; for hyperbolic embedding, the hyperbolic distance is used to evaluate the similarity between user-item pairs; specifically, the similarity between two embeddings is the inverse of the geodesic distance connecting the two points: Among them, d B is the geodesic distance function mentioned above; and Represent the prediction scores in Euclidean space and hyperbolic space, respectively.

6. The recommendation method based on user implicit hierarchical interests according to claim 1, characterized in that: For the recommendation task, paired Bayesian personalized ranking (BPR) is used as the main loss function, which encourages the scores of user u and the positive sample item i interacting with it to be higher than the unobserved negative sample item j; Among them, O={(u,i,j)|(u,i)∈O + ,(u,i)∈O - },O + represents the set of observed interactions between user u and item i, P - represents the set of unobserved interactions between user u and item j; represents the interactive prediction score of the model for positive examples in Euclidean space, represents the interactive prediction score of the model for negative examples in Euclidean space; σ is the sigmoid function.

7. The recommendation method based on user implicit hierarchical interests according to claim 1, characterized in that: The score-response-based geometry-enhanced self-distillation is implemented by minimizing the following loss function: Where σ is the sigmoid function; represents the interactive prediction score of the model for positive examples in hyperbolic space, represents the interactive prediction score of the model for negative examples in hyperbolic space; represents the interactive prediction score of the model for positive examples in Euclidean space, Represents the interactive prediction score of the model for negative examples in Euclidean space; τ is the temperature factor, which is used to adjust the output of the sigmoid function, thereby affecting the final recommendation result; the sigmoid function converts the original prediction score of the model into a probability distribution, and the temperature factor τ is used to control this conversion process; By combining the adaptive self-distillation loss with the BPR loss, the following objective function is minimized to optimize the model parameters: loss=loss Rec +λloss SD Among them, λ controls the self-distillation loss and adjusts this parameter to minimize the loss; in order to systematically find the optimal λ and τ values, a grid search algorithm is used; within a predefined range, different combinations of λ and τ values ​​are tried, and the combination of λ and τ with the lowest verification loss is selected.

8. The recommendation method based on user implicit hierarchical interests according to claim 1, characterized in that: After the interactive prediction scores of all recommended items are calculated, for each user, all items are sorted in descending order according to the scores, and a list of the top N items is generated and recommended to the user.

Images