A knowledge-aware recommendation method for multi-hyperbolic space

Abstract

The present application belongs to the technical field of big data mining, and particularly relates to a knowledge perception recommendation method of multiple hyperbolic spaces. The present application proposes a network model based on multiple hyperbolic spaces, and designs feature interaction between different hyperbolic spaces, thereby effectively learning data of different distributions. Moreover, because the knowledge perception recommendation method is based on hyperbolic space, it inherits the related advantages of hyperbolic embedding, such as avoiding embedding distortion problems, obtaining more hierarchical embedding results, and the like, which are not possessed by existing knowledge perception recommendation methods based on Euclidean space. The present application breaks through the shackles of the existing knowledge perception recommendation method, which cannot judge the sensitivity of the knowledge attribute of the user to the commodity, integrates hyperbolic distance information into embedding learning, mines the potential hierarchy of the user commodity bipartite graph, and scores the sensitivity of the knowledge attribute of each user to the commodity.

Description

Technical Field

[0001] This invention belongs to the field of big data mining technology, specifically relating to a knowledge perception recommendation method for multi-hyperbolic spaces. Background Technology

[0002] Recommendation, as a crucial component of internet data mining systems, effectively helps users extract valuable information from massive amounts of internet data, thereby enhancing user experience. However, traditional recommendation methods suffer from challenges such as sparsity and cold start. Knowledge graphs, as static databases, can be used to enhance product representation and improve recommendation performance. But clearly, user interaction with products doesn't entirely depend on the product's knowledge attributes. Therefore, we attempt to use multiple hyperbolic spaces as embedding spaces to obtain each user's score on their dependence on product knowledge attributes while simultaneously utilizing knowledge graphs for recommendation tasks. This is the motivation behind this invention. The following section details the relevant background technologies in this field.

[0003] (I) Knowledge-based recommendation methods

[0004] Let G be the historical interaction between the user and the product. b = {(u,i)|u∈U,i∈I} represents the interaction history between user u and product i, where U and I represent the sets of users and products; the knowledge graph is denoted as G. kg ={(h,r,t)|h,t∈E,r∈R} exists as a triple, where h and t represent the head and tail nodes of the triple, r represents the directed relation between them, and E and R represent the sets of entities and relations in the knowledge graph. Generally, we have... This means that products exist as partial entity nodes in the knowledge graph. Knowledge-aware recommendation methods aim to find a mapping function f that can predict the probability of a user interacting with other products, and rank them, using the knowledge graph and the user's historical interactions with products as input. The above can be formalized as follows:

[0005] {s1,s2,…,s n}=f u (G b G kg )

[0006] Where s kThis refers to user u's rating of the k-th item. Based on the message passing mechanism between nodes in the graph, existing knowledge-aware recommendation methods can be divided into path-based methods and graph neural network-based methods. Among them, the graph neural network-based methods are the main ones, such as KGAT and KGIN. However, the existing methods have two main problems: (1) They all use Euclidean space as the embedding space, without considering that the natural growth rate of nodes will exceed the representation capacity of the space, which will lead to embedding distortion and affect the final recommendation effect. At the same time, these methods do not analyze the degree of user dependence on knowledge attributes, and directly assume that all user interaction behaviors will be linked to the knowledge attributes of the items.

[0007] (II) Hyperbolic Space

[0008] Hyperbolic spaces are non-Euclidean spaces whose curvature is set to a constant negative value. Compared to Euclidean spaces, hyperbolic spaces have superior representational power. The representational power of hyperbolic spaces increases exponentially with the spatial radius, making them adept at capturing the latent hierarchical properties of data, while the representational power of Euclidean spaces only increases polynomially with the spatial radius. Common hyperbolic spaces include Lorentz spaces, etc. An n-dimensional Lorentz manifold with a constant negative curvature of -1 is defined as a Riemannian manifold (L... n ,g L ):

[0009] L n ={x∈R n+1 |<x,x> L =-1, x0>0},

[0010] g L = diag{-1,1,…,1}∈R n+1 ,

[0011] <x,y> L =x T g L y,

[0012] here<x,x> L The inner product of the Lorentz space, g L It is the upper half-plane of the Riemannian metric of Lorentz space, which resembles a hyperboloid, where x0 refers to the first dimension of x. An n-dimensional Poincaré manifold with constant negative curvature of -1 is defined as a Riemannian manifold.

[0013] D n ={x∈R n |<x,x> 2<1},

[0014]

[0015] Here gE It is an n-dimensional standard identity matrix. Note that arithmetic operations in Euclidean space do not satisfy closure in hyperbolic space. To apply hyperbolic space to deep learning, we will utilize the tangent space of hyperbolic space. The tangent space is a local Euclidean space. We assume that the properties of this locality hold globally, so that after performing operations in the tangent space, the results can be projected back into hyperbolic space. The function that projects data from hyperbolic space to the tangent space is called the logarithmic function, and the function that reprojects data back into hyperbolic space is called the exponential function. This pair of functions is clearly defined in Lorentz space and Poincaré space, respectively denoted as:

[0016]

[0017]

[0018]

[0019]

[0020] Where d L (x,y) and These represent distance in Lorentz space and Möbius striping, respectively.

[0021] (III) Knowledge-Aware Recommendation Method Based on Hyperbolic Space

[0022] Influenced by the hyperbolic space-based graph neural network proposed by Chami et al. in 2019, hyperbolic space has been gradually applied to graph neural networks and has attracted attention due to its excellent representational capabilities. When hyperbolic space is used as the embedding space to replace the original Euclidean space, the problem of embedding distortion is solved. Furthermore, experiments at Facebook's AI lab have found that because the definition of distance in hyperbolic space is related to its position in the space, hyperbolic distance imposes a stricter penalty on data far from the origin. Using hyperbolic space as the embedding space can reveal hierarchical relationships between data to some extent. The distance settings in hyperbolic space are as follows:

[0023] d L (x,y)=arcosh(-<x,y> L ),

[0024]

[0025] Previously, both LKGR and HAKG have attempted to apply hyperbolic spaces to knowledge-aware recommendation tasks, and both have improved recommendation performance. LKGR implements the KGAT method in Lorentz space, using tangent space to calculate the weights between different nodes and performing iterative calculations. Its main iterative formula is:

[0026]

[0027]

[0028] HAKG used the Poincaré space to learn the hierarchical relationships between data. The main equation in the paper is:

[0029]

[0030] However, existing knowledge-aware recommendation methods based on hyperbolic space use cosine similarity between user and item embeddings to measure user interest in items, without establishing a loss based on hyperbolic distance. This will compromise the hierarchical properties of the modeling to some extent. Furthermore, they all use the same hyperbolic space, failing to consider the differences in data distribution between knowledge graphs and interaction graphs.

[0031] Furthermore, current methods are based entirely on the assumption that user interactions are linked to the knowledge attributes of products, without taking into account the different levels of acceptance of product knowledge attributes among different users. Summary of the Invention

[0032] To overcome the shortcomings of existing knowledge-aware recommendation methods, this invention improves the model's predictive performance by using multiple hyperbolic spaces to handle different data distributions. Simultaneously, an attention mechanism based on hyperbolic distance is established to further enhance the hierarchical attributes of the model. This invention proposes a model called DHN, a knowledge-aware recommendation method based on multiple hyperbolic spaces. Furthermore, DHN also designs a scheme to obtain an index of each user's sensitivity to knowledge attributes.

[0033] The technical solution of the present invention:

[0034] A knowledge-aware recommendation method for multi-hyperbolic spaces, comprising the following steps:

[0035] Step 1: Preprocessing historical interaction and knowledge graph data

[0036] 1.1 Data Cleaning: In raw data, especially in knowledge graphs, some nodes appear too infrequently. In most cases, these nodes can interfere with the final learning process. To avoid this, we should first clean the knowledge graph to ensure that each entity appears in at least 10 triples. For the user's product history interaction graph, the same processing method used in neural collaborative filtering can be applied.

[0037] 1.2 Dataset partitioning: The historical interaction data between users and products is divided into training set, validation set and test set according to the proportion, which are used for model training, validation and testing respectively.

[0038] The second step is to construct a knowledge-aware recommendation system DHN model based on multi-hyperbolic space. This model consists of four parts: a knowledge graph encoder, a user-item bipartite graph encoder, an encoding fusion module, and a prediction module.

[0039] 2.1 Knowledge Graph Encoder

[0040] Knowledge graph encoders can efficiently parameterize entities and relations into vector representations while preserving graph structure. In DHN, we utilize a Poincaré space version of the classic knowledge graph embedding model TransE to learn embeddings. TransE, as a classic knowledge graph embedding method, ensures that entities with similar knowledge attributes are sufficiently close in the embedding space. In HAKG, we use a Möbius-based addition approach. This approach aggregates information about relationships and nodes, but it ignores the impact of different head nodes on model learning. We use parallel translation in hyperbolic space to distinguish the influence of different head nodes on relationships and tail nodes. For the product node h, the specific formula is as follows:

[0041]

[0042] In this context, we use underlined letters to represent the representation in hyperbolic space and ununderlined letters to represent the representation in the corresponding tangent space at the origin. We use... This represents the intermediate results of the tangent space graph of the l-th layer. and These refer to the logarithmic and exponential operations at corresponding points in the Poincaré space, respectively. Parallel translation refers to shifting the tangent space corresponding to a point in the hyperbolic space to the tangent space corresponding to another point. In the formula... This refers to translating a point in the tangent space of the Poincaré origin to the tangent space corresponding to the head node. We use this intermediate result to represent the product node in the knowledge graph. For a non-product node h, then... This is the final representation at level l.

[0043] 2.2 User Product Bipartite Map Encoder

[0044] Previous research has found that data in user-item bipartite graphs often exhibit a power-law distribution. To handle the power-law distribution in bipartite graphs and obtain hierarchical information, this invention learns user-item bipartite graph embeddings in Lorentz space. LightGCN has previously demonstrated that self-information aggregation and feature transformation are unnecessary for recommendation tasks. Therefore, DHN uses first-order neighborhoods for feature aggregation and obtains higher-order representations of users and items through recursive aggregation. For users and items in the bipartite graph, we have the following representations:

[0045]

[0046]

[0047] in, We designed attention weights related to hyperbolic distance. Unlike previous methods, DHN introduces a constraint related to hyperbolic distance. It requires minimizing the sum of hyperbolic distances between users and items. We assume that items with the same knowledge attributes are relatively close in the embedding space. For users who are not sensitive to the knowledge attributes of items, the items they interact with are mostly scattered in many different clusters. The embeddings of these users should be located near the origin, because the hyperbolic distance penalty is greater for points far from the origin. For other users, the embeddings of the items they interact with form spatially close clusters, causing these users' embeddings to be mainly located near the center of these clusters. Therefore, we use the distance of the user's embedding from the origin to determine whether the user is sensitive to the knowledge attributes of the items, and we designed an attention score based on the distance of the embedding from the origin. The calculation formula is:

[0048]

[0049] Users who are closer to the origin will receive higher weight in the bipartite graph embedding.

[0050] 2.3 Encoding Fusion Module

[0051] During the learning node embedding process, the knowledge from the knowledge graph and the user-item bipartite graph should reinforce each other to achieve higher-quality embeddings. To achieve this, our goal is to construct a mapping between the Poincaré space and the Lorentz space, enabling information interaction between different hyperbolic spaces. Specifically, DHN achieves interaction between the KG and the bipartite graph information by increasing the dimension. The formula can be viewed as... Information transfer was accomplished by leveraging the similarity in construction between Poincaré and Lorentz spaces on the tangent space at the origin. We designed a learnable gating module to control the updates of item embeddings in the knowledge graph:

[0052]

[0053]

[0054] here W0 and W1∈R n×n These are all learnable parameters, and σ represents the activation function.

[0055] 2.4 Prediction Module

[0056] After layer L, we obtain representations of users and items at different scales, and use summation to obtain the final representations of users and items:

[0057]

[0058]

[0059] DHN calculates the matching score by weighted summation of hyperbolic distance and cosine similarity:

[0060]

[0061] Where t is a hyperparameter and σ is the sigmoid activation function.

[0062] Step 3: Model Training and Performance Evaluation

[0063] 3.1 Model Training: After inputting batches of user interaction information and knowledge graphs into the DHN model, the model prediction results are obtained. An appropriate loss function is selected, and the loss value is calculated based on the model prediction results and label values. A suitable optimization algorithm is then selected to calculate the optimal solution for the model parameters. During model training, the model hyperparameters can be adjusted based on the validation set fit.

[0064] 3.2 Performance Evaluation: Determine the evaluation metrics to measure the fit between the model's prediction results and the label values ​​in the test set, and thus evaluate the overall prediction performance of the model.

[0065] The beneficial effects of this invention are:

[0066] (1) This invention proposes a network model based on multiple hyperbolic spaces and designs feature interactions between different hyperbolic spaces to effectively learn data with different distributions. Furthermore, because it is a knowledge-aware recommendation method based on hyperbolic space, it inherits the advantages of hyperbolic embedding, such as avoiding embedding distortion and obtaining more hierarchical embedding results, which are not available in existing knowledge-aware recommendation methods based on Euclidean space.

[0067] (2) This invention breaks through the limitation of existing knowledge-aware recommendation methods that cannot determine the sensitivity of users to the knowledge attributes of products. It integrates hyperbolic distance information into embedding learning, mines the potential hierarchy of the user-product bipartite graph, and scores each user’s sensitivity to the knowledge attributes of products. Attached Figure Description

[0068] Figure 1 This paper encodes common tree-like structures for common hyperbolic spaces, Poincaré spaces, and Lorentz spaces, demonstrating the embedding distortion in Euclidean spaces.

[0069] Figure 2 This provides the overall framework for the DHN model.

[0070] Figure 3 This is a schematic diagram illustrating the calculation of attention weights.

[0071] Figure 4 This is a graph showing the DHN ratings for users with different embedding locations. Detailed Implementation

[0072] like Figure 1 As shown, tree structures are one of the most common graph structures. As the depth of a tree increases, the number of nodes grows exponentially. Taking the two-dimensional space in the graph as an example, the representable area of ​​Euclidean space grows quadratically, which is far slower than the growth rate of nodes. This results in very close embeddings in Euclidean space, making it impossible to distinguish between positive and negative examples, leading to embedding distortion. Hyperbolic space, due to its stronger representational power, does not exhibit this problem. Furthermore, using different hyperbolic spaces allows for the application of different spatial metrics to constrain data with varying distributions.

[0073] The specific embodiments of the present invention will be described in detail below with reference to the design intent, accompanying drawings, and specific examples.

[0074] A knowledge-aware recommendation method for multi-hyperbolic spaces, comprising the following steps:

[0075] Step 1: Preprocess and divide the data.

[0076] (1) Dataset: The Alibaba-iFashion dataset is used to further illustrate the content of this invention. This dataset is widely used in existing research on knowledge-aware recommendation methods. It is a fashion dataset collected from Alibaba's online shopping system. The data in the knowledge graph is cleaned using the 10Core strategy.

[0077] (2) Data partitioning: The data is divided into training set, validation set and test set in a ratio of 8:1:1.

[0078] Step 2: Forward Learning Process

[0079] (1) Initial embedding representation: All nodes are assigned a unique number and a random 64-dimensional embedding representation for final optimization.

[0080] (2) Batch input model: Input the obtained representations in batches Figure 2 In the DHN model shown, in order to make the model more robust and avoid the risk of overfitting, random edge dropout is performed on the knowledge graph in different batches of calculation. That is, the input of different batches corresponds to the knowledge graph with half of the edges randomly filtered out.

[0081] (3) Attention mechanisms: such as Figure 3 As shown, we use a two-dimensional Poincaré disk as an example to further illustrate the rationale for the proposed attention mechanism. Unlike previous methods, DHN introduces a constraint related to hyperbolic distance. It requires minimizing the sum of the hyperbolic distances between the user and the item. When this condition is satisfied, the item embedding should lie within a hyperbolic circle centered on the user embedding. Due to the geometric properties of hyperbolic distance, such as... Figure 3 As shown, the item embedding should be located within a droplet-shaped region centered on the user. Compared to the hyperbolic embedding of the user, the cosine similarity between the item and the user tends to be higher when the item embedding is farther from the origin (as shown by i in the figure). o In this way, we can estimate the cosine similarity between users and products at different scales with minimal computational cost.

[0082] (4) Prediction module: After all forward propagation, the prediction result is finally obtained. Figure 4 The DHN scores users at different embedding positions, demonstrating that the hierarchical performance of the embedding results obtained by DHN is more pronounced.

[0083] Step 3: Model Training and Performance Evaluation

[0084] (1) Model Training: After inputting batches of user historical interaction data into the model, the model prediction results are obtained. Hyperbolic distance and cosine similarity in the tangent space are selected as loss functions, and the loss value is calculated based on the model prediction results and label values. The Adam optimization algorithm is selected to calculate the optimal solution of the model parameters. During model training, the model hyperparameters are adjusted according to the fitting of the validation set.

[0085] This includes a hyperbolic distance loss function embedded in the user-product bipartite graph:

[0086]

[0087] Where M u It is a randomly selected set of negative samples. The hyperbolic loss function in the knowledge graph is roughly similar to the loss function of TransE, denoted as L. dis-kg There is also the loss function for cosine similarity in bipartite graphs:

[0088]

[0089] Here, 'm' is a threshold, presented as a hyperparameter. Only when the value exceeds this hyperparameter will it be treated as a negative example, thus avoiding the situation where potential positive examples are mistakenly identified as negative examples. Finally, the above loss functions are weighted to ensure that their orders of magnitude are roughly the same, and then summed to obtain the final loss function.

[0090] (2) Performance evaluation: The evaluation metrics for recommended methods are generally the recall rate (Recall@20) and normalized depreciation cumulative gain (NDCG@20) of the top 20 scores.

[0091] The recall calculation formula is as follows:

[0092]

[0093] Among them, True Positive (TP) is an example where the actual class is positive and the predicted class is positive; False Negative (FN) is an example where the actual class is positive and the predicted class is negative. Recall@20 mainly detects the recall rate of the top 20 rated products.

[0094] The formula for calculating NDCG is as follows:

[0095]

[0096] NDCG@20 is mainly used to check whether the order of the first 20 sorted items is reasonable.

Claims

1. A knowledge-aware recommendation method for multi-hyperbolic spaces, characterized in that, The steps are as follows: Step 1: Preprocessing historical interaction and knowledge graph data Step 2: Construct a knowledge-aware recommendation system DHN model based on multi-hyperbolic space. This model includes a knowledge graph encoder, a user-item bipartite graph encoder, an encoding fusion module, and a prediction module. 2.1 Knowledge Graph Encoder In DHN, a Poincaré space version of the classic knowledge graph embedding model TransE is used to learn embeddings; parallel displacement in hyperbolic space is used to distinguish the influence of different head nodes on relations and tail nodes; for the product node h, the specific formula is as follows: In this context, underlined letters represent representations in hyperbolic space, while ununderlined letters represent representations in the tangent space at the corresponding origin. For non-commodity nodes h, then... That is, this is the final representation at layer l; 2.2 User Product Bipartite Map Encoder DHN uses first-order neighborhoods for feature aggregation and obtains higher-order representations of users and items through recursive aggregation; for users and items in a bipartite graph, the representations are as follows: This study uses the user's embedding position from the origin to determine whether the user is sensitive to the product's knowledge attributes, and designs an attention score based on the embedding's distance from the origin; here... The calculation formula is: Users who are closer to the origin will receive higher weight in the bipartite graph embedding; 2.3 Encoding Fusion Module DHN enables interaction between KG and bipartite graph information by increasing dimensionality, and designs a learnable gating module to control the updates of item embeddings in the knowledge graph: 2.4 Prediction Module After layer L, representations of users and items at different scales are obtained. Summation is used to obtain the final representations of users and items: DHN calculates the matching score by weighted summation of hyperbolic distance and cosine similarity: Where t is a hyperparameter and σ is the sigmoid activation function; Step 3: Model training and performance evaluation.

2. The knowledge-aware recommendation method for multiple hyperbolic spaces as described in claim 1, characterized in that, The third step, as described above, is performed as follows: 3.1 Model Training: After inputting batches of user interaction information and knowledge graphs into the DHN model, the model prediction results are obtained. A loss function is selected and the loss value is calculated based on the model prediction results and label values. An optimization algorithm is selected to calculate the optimal solution of the model parameters. During model training, the model hyperparameters are adjusted based on the fitting of the validation set. 3.2 Performance Evaluation: Determine the evaluation metrics to measure the fit between the model's prediction results and the label values ​​in the test set, and thus evaluate the overall prediction performance of the model.

3. The knowledge-aware recommendation method for multiple hyperbolic spaces as described in claim 2, characterized in that, Step 3.1 is specifically operated as follows: After inputting batch user historical interaction data into the model, the model prediction results are obtained. Hyperbolic distance and cosine similarity in tangent space are selected as loss functions, and the loss value is calculated based on the model prediction results and label values. The Adam optimization algorithm is selected to calculate the optimal solution of model parameters. During model training, the model hyperparameters are adjusted according to the fitting of the validation set. This includes a hyperbolic distance loss function embedded in the user-product bipartite graph:

4. A knowledge-aware recommendation method for multiple hyperbolic spaces as described in claim 1, 2, or 3, characterized in that, The first step, as described above, is performed as follows: 1.1 Data Cleaning: First, clean the knowledge graph to ensure that each entity appears in at least 10 triples; for the user's product history interaction graph, the same processing method as in the neural collaborative filtering method can be used. 1.2 Dataset partitioning: The historical interaction data between users and products is divided into training set, validation set and test set according to the proportion, which are used for model training, validation and testing respectively.

Images