Recommendation method based on attention knowledge network combining explicit and implicit information
By combining explicit and implicit information in an attention-based knowledge network, the problems of data sparsity and changing user interests in recommendation systems are solved, enabling more accurate personalized recommendations. Advanced feature cross-fusion using entity relationships and user preferences in knowledge graphs improves the performance of the recommendation system.
Patent Information
- Application Number
- CN202310017654.4
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2023-01-06
- Publication Date
- 2026-02-17
- Estimated Expiration
- 2043-01-06
AI Technical Summary
In existing recommendation systems, collaborative filtering methods suffer from data sparsity and cold start problems, and existing knowledge graph embedding methods have difficulty accurately distinguishing changes in user interests at different levels in personalized recommendations, and cannot effectively integrate knowledge graphs in recommendation scenarios.
We employ an attention-based knowledge network that combines explicit and implicit information. Through an information sharing layer, an attention ripple propagation layer, and a cross-feature fusion network, we automatically transmit interactive information. By utilizing entity relationships and user preferences in the knowledge graph, we perform advanced feature cross-fusion, reduce the number of parameters, and accurately capture changes in user interests.
It improves the accuracy of recommendations, effectively alleviates the problem of data sparsity, and can better represent knowledge graphs in low-dimensional space, thus achieving more accurate personalized recommendations.
Smart Images
Figure CN116010716B_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of computer application technology, specifically a recommendation method based on an attention knowledge network that combines explicit and implicit information. Background Technology
[0002] In today's era of widespread internet access, information and services are exploding, often overwhelming users with massive amounts of data. Recommender systems (RS) can address this information overload by helping users find information of interest. Collaborative filtering (CF) is one of the most classic and widely used techniques in recommender systems. It leverages users' historical information and interactions to make recommendations based on their potential shared preferences. However, collaborative filtering suffers from data sparsity and cold-start problems. Common solutions to these issues include using information related to users and items to compensate for sparsity, such as social networks, user-item attributes, and contextual information, thereby improving recommendation performance.
[0003] Simply adding relevant information to collaborative filtering for recommendations can improve performance to some extent, but it ignores the connections between these pieces of information. Relevant information is interconnected; linking these pieces of information forms a knowledge graph (KG). Generally, a knowledge graph is a directed heterogeneous graph where nodes represent entities (such as users or items), and edges represent the relationships between entities. Knowledge graphs typically offer advantages over traditional methods in recommendation systems, such as interpretability and scalability.
[0004] Among the research on improving recommendation performance using knowledge graphs, methods based on knowledge graph embedding (KGE) are one type. For example, Deep Knowledge-Aware Networks (DKN) is a model combining entity embedding and convolutional neural networks (CNNs), which has shown good performance in news recommendation. Embedding-based methods offer high flexibility, allowing different types of information to be embedded into the model. However, these methods perform well within graphs such as link prediction and are not suitable for personalized recommendations. Another approach is path-based methods, such as Personalized Entity Recommendation (PER), which treats the knowledge graph as a heterogeneous information network. This model uses latent features of the original path to represent the connectivity between users and items on different types of relationship graphs. However, the graph in the PER model relies on manual design, limiting its application in different recommendation scenarios. RippleNet is a memory-like network model where user preferences propagate through the KG, involving different levels of user interest. However, users' interests differ across levels, and this model cannot accurately distinguish changes in user interest at different levels. In practical recommendation tasks, challenges remain, such as how to better integrate knowledge graphs into recommendation scenarios and how to design KGE methods to better represent KG in low-dimensional space. These are pressing issues that need to be addressed. Summary of the Invention
[0005] The purpose of this invention is to solve the problems existing in the prior art and provide a recommendation method based on an attention knowledge network that combines explicit and implicit information. This method makes full use of the rich semantic information in the knowledge graph to obtain the implicit semantics of entities and items, and automatically transmits interaction information during the training process, thereby improving the accuracy of recommendations and effectively mitigating the problem of data sparsity.
[0006] To achieve the above objectives, the present invention employs the following technical solution:
[0007] Recommendation methods based on attention-based knowledge networks that combine explicit and implicit information include the following steps:
[0008] S1. Collect user historical behavior data, process the data, and match the items in the recommendation system with the entities in the knowledge graph. Utilize the entities in the knowledge graph that correspond to the items in the user's historical interactions to extend outwards in the knowledge graph, and use a multi-head attention mechanism to assign different weights to the entities.
[0009] S2. Use an information sharing layer to enable projects and entities to interact with each other in terms of features and complement each other's available information;
[0010] S3. In the attention ripple propagation layer, the user interaction item representation is input, and then the user preference is propagated in the knowledge graph by using the correspondence between the item and the entity in the knowledge graph and the connection relationship between the entity in the knowledge graph. Finally, the entity representation obtained by the diffusion is aggregated into the user representation.
[0011] S4. In cross-feature fusion networks, explicit interaction of features is performed at the vector level to reduce the number of parameters;
[0012] S5. Perform a dot product on the enhanced embedding representations of users and items to obtain the prediction results of users on items. Finally, in the prediction layer, combine the explicit and implicit information to output the dot product prediction results.
[0013] Preferably, in step S2, the potential embedding representation of the item in the recommendation system and the corresponding entity representation in the knowledge graph are input into the information sharing layer. The information sharing layer first increases the dimensionality of the embedding representation of the item and the entity representation in the knowledge graph, and then performs information fusion and complementation in the high dimension. Finally, the fused and shared high-dimensional information is processed and output.
[0014] Preferably, in step S3, an attention mechanism is used to assign different weights to each layer of the ripple network.
[0015] Preferably, in step S4, in the cross-feature fusion network, an outer product operation is performed on the entity representations aggregated in each layer of the previous item representation and the ripple propagation layer diffusion process to form a tensor-like form, and then the input of the next level network is obtained by dimensional compression of the tensor.
[0016] Preferably, in step S5, a negative sampling strategy is used to train the model, and the loss function is as follows:
[0017]
[0018] Where U represents user, V represents item, E represents entity embedding matrix, R is relation embedding matrix, J() is cross-entropy loss function, and λ * These are the model parameters.
[0019] Preferably, in step S5, the objective function is optimized using stochastic gradient descent, and the learning rate and batch size are determined using grid search.
[0020] Compared with the prior art, the beneficial effects of the present invention are as follows:
[0021] 1. This invention combines KGE as an auxiliary module to supplement information in recommendation tasks. An information sharing layer (ISL) is designed between the recommendation task and KGE. The ISL can fully connect KGE and recommendation tasks to share information and automatically transmit interactive information during training to obtain the implicit semantics of entities and items.
[0022] 2. This invention enables advanced explicit feature cross-fusion, where feature interactions occur at the vector level, compared to the element-level interactions in traditional neural network models, thus reducing the number of parameters. The network can retain key information from each layer during propagation, preventing the loss of crucial information.
[0023] 3. Building upon the complementary information at the lower level of the ISL layer, this invention innovatively designs a multi-layered ripple network based on multi-head attention. This network can fully perceive changes in user interests across similar categories of items, achieving more accurate recommendations. Finally, it combines a cross-feature fusion layer for high-order feature interaction, fully utilizing the shared information between KG entities and items. Attached Figure Description
[0024] Figure 1 This is the overall architecture diagram of the present invention;
[0025] Figure 2 This is the information sharing layer of the present invention;
[0026] Figure 3 This invention relates to a cross-feature fusion network;
[0027] Figure 4 These are the experimental results of the present invention and the comparative algorithm;
[0028] Figure 5 This is the experimental result of the present invention and the comparison algorithm Top-K. Detailed Implementation
[0029] The present invention will be further illustrated below with reference to specific embodiments. It should be understood that these embodiments are for illustrative purposes only and are not intended to limit the scope of the invention. Furthermore, it should be understood that after reading the teachings of this invention, those skilled in the art can make various alterations or modifications to the invention, and these equivalent forms also fall within the scope defined in this application.
[0030] Example: As attached Figure 1-5 The present invention describes a recommendation method based on an attention knowledge network that combines explicit and implicit information, comprising:
[0031] 1. Data Preprocessing
[0032] First, user historical behavior data is collected and processed. Then, items in the recommendation system are mapped to entities in the knowledge graph, allowing each item to connect with other items through certain attributes. Next, some user interaction samples are randomly selected as negative samples.
[0033] 2. Information Sharing Layer
[0034] This invention inputs the latent embedding representation of items in the recommendation system and the corresponding entity representation in the knowledge graph into the information sharing layer (see appendix). Figure 2 In this approach, the information sharing layer first increases the dimensionality of the embedded representations of projects and the entity representations in the knowledge graph. Then, it fuses and complements the information at this higher dimension, and finally outputs the fused and shared high-dimensional information. This operation can automatically complement and share information between entities and projects, which can, to some extent, compensate for the sparsity in the data.
[0035] 3. Attention Ripple Propagation Layer
[0036] In the ripple propagation layer, this invention inputs a user interaction item representation, which inherently possesses certain attributes. Then, utilizing the correspondence between items and entities in the knowledge graph, and the connections between entities within the knowledge graph, this invention propagates user preferences within the knowledge graph. Finally, this invention aggregates the propagated entity representations into the user representation. The process of user preference propagation is analogous to water droplets forming outward-spreading ripples on the surface of water. It is important to note that as user preferences propagate outward, the correlation between entities in the knowledge graph gradually decreases, and user preferences are negatively influenced by higher-order entities. To address this issue, this invention uses an attention mechanism to assign different weights to each layer of the ripple network. This allows for more accurate capture of user preferences while reducing the impact of negative information.
[0037] 4. Cross-feature fusion network
[0038] As attached Figure 3 In the cross-feature fusion network, this invention performs an outer product operation on the entity representations aggregated at each layer during the previous item representation and the ripple propagation layer diffusion process, forming a tensor-like form. This operation can fully utilize the high-order features in the input vector, and then the input to the next level of the network is obtained by dimensionality compression of this tensor. Here, each level of propagation corresponds to a different level of the ripple propagation layer. Finally, this invention fully fuses the information extracted from the ripple propagation layer to better capture the user's personalized preferences. It is important to note that the feature interactions in the cross-feature fusion network are explicit, and their interaction occurs at the vector level rather than the element level, which reduces the number of parameters to some extent.
[0039] 5. Training
[0040] Through the above steps, the present invention performs a dot product on the enhanced embedding representations of users and items to obtain the prediction results of users for items.
[0041] 5.1 Loss Function
[0042] The training phase involves parameter estimation and iterative optimization. To improve efficiency, we employed a negative sampling strategy to train the model. The loss function is as follows:
[0043]
[0044] Where U represents user, V represents item, E represents entity embedding matrix, R is relation embedding matrix, J() is cross-entropy loss function, and λ * These are the model parameters.
[0045] 5.2 Optimization Methods
[0046] This invention uses stochastic gradient descent (SGD) to optimize the objective function. The learning rate and batch size are determined using grid search.
[0047] 6. Experiments prove
[0048] As attached Figure 4 , 5 We used four datasets: movielens-20M, movielens-1M, Book-Crossing, and Last.FM, and compared them with the following nine algorithms: LibFM, PER, Wide&Deep, DKN, RippleNet, MKR, KGCN, Ripp-MKR, and CAKR, to verify that our model outperforms other comparison recommendation algorithms.
Claims
1. A recommendation method based on attention knowledge network combining explicit and implicit information, characterized in that, The method comprises the steps of: S1, collecting user historical behavior data, processing the data, and corresponding items in the recommendation system and entities in the knowledge graph, using the corresponding entities of the user historical interaction items in the knowledge graph to extend outward in the knowledge graph, and using a multi-head attention mechanism to assign different weights to the entities; S2, using an information sharing layer to make the items and entities interact with each other and complement the available information; The latent embedding representation of the item in the recommendation system and the corresponding entity representation in the knowledge graph are input into the information sharing layer, the information sharing layer first performs dimensionality elevation on the embedding representation of the item and the entity representation in the knowledge graph, then performs information fusion and complementation in the high dimension, and finally processes and outputs the high-dimensional information after fusion and sharing; S3, in the attention wave propagation layer, input the item representation of user interaction, then use the corresponding relationship between the item and the entity in the knowledge graph and the connection relationship between the entities in the knowledge graph to propagate the user preference in the knowledge graph, and finally aggregate the entity representation obtained by diffusion to the user representation; wherein, an attention mechanism is used to assign different weights to each layer of the wave network; S4, in the cross-feature fusion network, the features are explicitly interacted at the vector level to reduce the number of parameters; In the cross-feature fusion network, the outer product operation is performed on the previous item representation and the entity representation aggregated in each layer in the wave propagation layer diffusion process to form a tensor-like form, and then the dimension compression is performed on the tensor to obtain the input of the next level network; S5, the obtained enhanced embedding representation of the user and the item is dot multiplied to obtain the prediction result of the user to the item, and finally in the prediction layer, the explicit and implicit information is combined to output the dot product prediction result. 2.The recommendation method based on the attention knowledge network combined with explicit and implicit information according to claim 1, wherein, In step S5, the model is trained by using a negative sampling strategy, and the loss function is as follows: wherein U denotes users, V denotes items, E denotes an entity embedding matrix, R is a relation embedding matrix, J( ) is a cross-entropy loss function, and λ * are model parameters. 3.The recommendation method based on the attention knowledge network combined with explicit and implicit information according to claim 1, characterized in that, In step S5, the random gradient descent method is used to optimize the objective function, and the learning rate and batch_size are determined by using the grid search method respectively.