A knowledge-aware graph convolutional network recommendation method based on denoising
By calculating the similarity score of user preferences and neighbor entities in the knowledge-aware graph convolution network and normalizing it, the problem of noise information propagation in the graph neural network recommendation method is solved, and recommendation performance and accuracy are improved.
Patent Information
- Application Number
- CN202210266896.2
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2022-03-17
- Publication Date
- 2025-03-18
- Estimated Expiration
- 2042-03-17
AI Technical Summary
The existing graph-based graph neural network recommendation method introduces noise when aggregating the neighborhood information of an entity, resulting in negative impact on recommendation performance.
The knowledge-aware graph convolution network recommendation method based on denoising is adopted. By calculating the similarity score between user preferences and neighbor entities, it is normalized as the weight of neighbor information transmission of entities in the graph convolution network, thereby destroying the propagation of noise information.
Effectively remove noise information, improve recommendation performance, and enhance recommendation accuracy and interpretability.
Smart Images
Figure CN114637857B_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of personalized recommendation technology, and in particular to a knowledge-aware graph convolutional network recommendation method based on denoising. Background Art
[0002] With the rapid development of the Internet, data and information are growing explosively. Faced with such a huge amount of data, the information that people really want is drowned in the ocean of information. Recommendation systems can capture the historical interaction records between users and items, explore users' interest preferences, and actively provide users with content of interest. Recommendations based on collaborative filtering use the similarities of users or items from interaction data to model user preferences and have been widely used. However, recommendation algorithms based on collaborative filtering have the problems of sparsity and cold start, which greatly affect the recommendation effect.
[0003] In order to solve these problems, people introduce auxiliary information such as project attributes, project comments, and social networks to obtain better recommendation performance. Knowledge graphs describe concepts, entities, and their relationships in the objective world in a structured form, providing a large amount of semantic information. By introducing knowledge graphs as auxiliary information into recommendation systems, the recommendation effect can be greatly improved and the interpretability of recommendations can be enhanced. Among the existing recommendation methods combined with knowledge graphs, the commonly used representation learning-based algorithms mostly process the triples (head entity, relationship, tail entity) in the knowledge graph independently, fail to introduce multi-hop relationships, and ignore the hidden information in the local neighbors around the triples. Path-based recommendation algorithms use the multi-hop knowledge of knowledge graphs and effectively utilize the semantic network information of knowledge graphs, but their implementation is relatively complex and generally requires manual definition and selection of meta-paths.
[0004] Recently, it has become a trend to develop end-to-end models based on graph neural networks. The key idea is to use information aggregation schemes, which can effectively integrate multi-hop neighbors into vector representations. Thanks to the integration of structural modeling and representation learning, these graph neural network-based models achieve good recommendation performance. However, it is worth noting that in recommendation scenarios, users select items based on their own preferences. Therefore, when using graph neural networks to aggregate information in knowledge graphs, some of them may be noisy information that cannot reflect user preferences. Existing studies assign different weights according to the relevance of user-relationships or the relevance between entities. However, these methods do not explicitly model user preferences and propose the importance of removing noise, and fail to delicately express the relevance of user preferences and entities, resulting in poor representation and performance. This means that blindly aggregating neighborhood information will introduce more noise, which will negatively affect the recommendation performance. Summary of the invention
[0005] The present invention aims to solve the problem that the existing graph neural network recommendation method based on knowledge graph introduces noise when aggregating the neighborhood information of entities, thereby negatively affecting the recommendation performance, and provides a knowledge-aware graph convolutional network recommendation method based on denoising.
[0006] To solve the above problems, the present invention is achieved through the following technical solutions:
[0007] A knowledge-aware graph convolutional network recommendation method based on denoising includes the following steps:
[0008] Step 1: Collect the user's historical data, find all the items that the user has interacted with in the historical data, and align the items with the entities in the knowledge graph; send the user ID, entity ID and relationship ID in the knowledge graph to the embedding layer to obtain the initial embedding vector representation of the user, the initial embedding vector representation of the entity and the initial embedding vector representation of the relationship;
[0009] Step 2: Using the attention mechanism, first calculate the user's preference vector representation for a single interacted item, and then calculate the user's preference vector representation for all interacted items, that is, the user preference vector representation;
[0010]
[0011]
[0012] Step 3: Calculate and normalize the similarity score between the user preference vector representation and the neighbor entity vector representation through the noise recognition operation. The normalized similarity score is used to control the neighbor information transmission of each entity belonging to the recommended type in the graph convolutional network, destroy the propagation of noise information, and obtain the initial neighbor structure vector representation of each entity belonging to the recommended type.
[0013]
[0014] Step 4: Capture the neighbor structure information of each entity of the recommended type in the knowledge graph through graph convolution operations, aggregate the initial embedding vector representation of each entity of the recommended type and the initial neighbor structure vector representation of each entity of the recommended type, and obtain the first-order vector representation of the entity of the recommended type. By stacking multiple layers of graph convolution to collect influence signals from high-order neighbors, more information related to the recommended type entity is obtained, and each order of embedding vector representation of each entity of the recommended type is obtained;
[0015]
[0016] Step 5: Aggregate the user preference vector representation through graph convolution operations to represent the vector representation of each item or entity that the user has interacted with. By stacking multiple layers of graph convolution, the user's potential interests are mined in a more in-depth manner to obtain the user's embedding vector representation of each order;
[0017]
[0018] Step 6: Concatenate the 0-L order embedding vector representations of the user to obtain the final vector representation of the user; at the same time, concatenate the 0-L order embedding vector representations of the entities belonging to the recommended type to obtain the final vector representation of the entities belonging to the recommended type;
[0019] Step 7: The inner product of the final vector representation of the user and the final vector representation of each entity of the recommended type is used as the user's score for the entity, and the entities with the top k scores are recommended to the user; k is a set value;
[0020] In the above formula, e u represents the initial embedding vector representation of user u, Indicates the items that the user has interacted with p j Related relationships The initial embedding vector representation of Represents the user's interaction with a single item p j The preference vector representation of represents the user preference vector representation, Indicates the items that the user has interacted with p j Related relationships The collection of N u is the set of items that the user has interacted with; e t represents the initial embedding vector representation of the neighbor entity t of entity h belonging to the recommended type, represents the initial neighbor structure vector representation of entity h belonging to the recommended type, N h represents the set of neighbor entities t of entity h belonging to the recommended type; represents the l-th order embedding vector representation of entity h belonging to the recommended type, represents the l-1th order embedding vector representation of entity h belonging to the recommended type, The l-1 order neighbor structure vector representation of entity h belonging to the recommended type, l = 0, 1, ..., L, L is the number of graph convolution layers, and f represents the aggregation function; represents the l-order embedding vector representation of user u, |N u | represents the number of items the user has interacted with, ⊙ represents the Hadamard product, and the superscript T represents the transpose.
[0021] In step 4 above, the aggregation function f is:
[0022]
[0023] In the formula, represents the l-th order embedding vector representation of entity h belonging to the recommended type, represents the l-1th order embedding vector representation of entity h belonging to the recommended type, Represents the l-1 order neighbor structure vector representation of entity h belonging to the recommended type, W represents the weight parameter of the graph convolutional network, b represents the bias parameter of the graph convolutional network, l = 0, 1, …, L, L is the number of graph convolution layers.
[0024] In the above method, the 0-order embedding vector of entity h belonging to the recommended type is represented by That is, the initial embedding vector representation e of entity h belonging to the recommended type h ; The 0-order neighbor structure vector representation of entity h belonging to the recommended type That is, the initial neighbor structure vector representation of entity h belonging to the recommended type The 0th-order embedding vector representation of user u That is, the initial embedding vector representation e of user u u .
[0025] Compared with the prior art, the present invention utilizes the rich relationship information between knowledge graph entities to achieve modeling of user preferences. In order to clarify whether the neighbor entities are noise that cannot reflect user preferences, the similarity scores between user preferences and neighbor entities are calculated. The score reflects the confidence of whether the entity is noise. The score is normalized and used as the weight when aggregating the neighbor information of the entity under the graph convolutional network. This can destroy the propagation of noise information while making full use of the high-order structural information of the knowledge graph, thereby enhancing the recommendation performance. BRIEF DESCRIPTION OF THE DRAWINGS
[0026] Figure 1 This is a model framework diagram of a knowledge-aware graph convolutional network recommendation method based on denoising.
[0027] Figure 2 Schematic diagram of the knowledge graph in the field of movie recommendations. DETAILED DESCRIPTION
[0028] In order to make the objectives, technical solutions and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with specific examples.
[0029] See also Figure 1 , a knowledge-aware graph convolutional network recommendation method based on denoising, the specific implementation process is as follows:
[0030] Step 1: Collect the user's historical data, find all the items that the user has interacted with in the historical data, and align the items with the entities in the given knowledge graph. Send the user ID, entity ID and relationship ID in the knowledge graph to the embedding layer to obtain the initial embedding vector representation of the user, the initial embedding vector representation of the entity, and the initial embedding vector representation of the relationship.
[0031] Knowledge graph is a graph-based data structure consisting of nodes and edges. Each node represents an "entity" and each edge is a "relationship" between entities. Knowledge graph is essentially a semantic network. Entities can refer to things in the real world, such as people, places, companies, telephones, animals, etc.; relationships are used to express a certain connection between different entities.
[0032] like Figure 2 As shown in the figure, in the field of movie recommendation, the entities in its knowledge graph include movie names, director names, actor names, theme names, etc., and the relationships include director, starring actor, theme, etc. The item that the user has interacted with, that is, the movie he has watched, is "Movie B", and the item "Movie B" that the user has interacted with is aligned with the entity "Movie B" in the knowledge graph. In the knowledge graph, the entities related to the entity "Movie B" include "Actor A", "Director A", and "Inspirational Film", etc. The relationship between the head entity "Actor A" and the tail entity "Movie B" is "Starring actor", the relationship between the head entity "Director A" and the tail entity "Movie B" is "Director", and the relationship between the head entity "Movie B" and the tail entity "Inspirational Film" is theme.
[0033] The word embedding method can be used to convert the ID into a vector. The input data such as user ID, entity ID, and relationship ID are converted into a one-hot vector. The one-hot vector and the weight of the embedding layer are matrix multiplied to obtain the initial embedding vector representation of the user, the initial embedding vector representation of the entity, and the initial embedding vector representation of the relationship.
[0034] Step 2: Use the attention mechanism to first calculate the user's preference vector representation for a single interacted item Then calculate the user's preference vector representation for all interacted items, that is, the user preference vector representation
[0035] 1) User preference vector representation for a single interacted item
[0036] Users interact with an item, i.e. an entity, based on their own preferences. The relationship connected by the entity can be regarded as the user's preference for interacting with the entity. For example, if a user chooses to watch "Movie B", "Movie B" is the item that the user has interacted with. According to the knowledge graph, there is a triple (director A, director, movie B), i.e. (head entity, relationship, tail entity). The reason for the user's choice can be attributed to director A, i.e. the relationship "director" is the user's preference.
[0037] Since an entity may have multiple relationship connections and each relationship has different importance to the user, the attention mechanism can be used to calculate the importance of the relationship to the user and assign different attention weights to each relationship.
[0038]
[0039] In the formula, e u represents the initial embedding vector representation of user u, Indicates the items that the user has interacted with p j Related relationships The initial embedding vector representation of Indicates the items that the user has interacted with p j Related relationships A collection of items that the user has interacted with. j The relevant relations include the items that the user has interacted with. j All relations as head entities and items p that the user has interacted with j All relations as tail entities.
[0040] For items that the user has interacted with, j Related relationships Perform weighted summation to obtain the user's preference vector representation for a single interacted item
[0041]
[0042] 2) User preference vector representation for all interacted items
[0043] Since users often interact with more than one item and have multiple preferences, the attention mechanism is used to calculate the importance of each preference to the user and assign different attention weights β(u, p j ):
[0044]
[0045] Among them, e u represents the initial embedding vector representation of user u, Represents the user's interaction with a single item p j The preference vector representation of N u Is the collection of items that the user has interacted with.
[0046] The user's interaction with all items is the user's interaction with entity p j The preference vector representation of the user is weighted summed to obtain the user's preference vector representation for all interacted items, that is, the user preference vector representation
[0047]
[0048] Step 3: Calculate and normalize the similarity score between the user preference vector representation and the neighbor entity vector representation through the noise recognition operation. The normalized similarity score is used to control the neighbor information transmission of each entity belonging to the recommended type in the graph convolutional network, destroy the propagation of noise information, and obtain the initial neighbor structure vector representation of each entity belonging to the recommended type.
[0049] A given knowledge graph includes entities that belong to the recommended type and entities that do not belong to the recommended type. For example, in the field of movie recommendations, movie names such as "Movie B", "Movie A", and "Movie C" belong to the recommended type, while director names such as "Director A" and actor names such as "Actor A" do not belong to the recommended type. When making recommendations, only entities that belong to the recommended type are considered, and entities that do not belong to the recommended type are not considered.
[0050] In knowledge graph-based recommendations, graph convolutional networks are used to capture high-order structures between entities in the knowledge graph and aggregate neighbor information. When graph convolution aggregates neighbor information of entities of the recommended type in the knowledge graph, the neighbor information does not necessarily reflect user preferences. Therefore, noise information exists in the neighbor entity t of entity h of the recommended type. Neighbor entity t refers to the entity connected to entity h of the recommended type.
[0051] To identify noise, the similarity score between user preference and neighbor entity is calculated to measure the confidence that the neighbor entity is noise. for:
[0052]
[0053] In the formula, e t The initial embedding vector representation of the neighbor entity t of entity h belonging to the recommended type.
[0054] The similarity score between user preference p and neighbor entity t is Perform normalization operation to obtain the normalized score s(p,t) of the similarity between user preferences and neighbor entities:
[0055]
[0056] Where N h Represents the set of neighbor entities t of entity h belonging to the recommended type
[0057] When the graph convolutional network performs information transmission, the normalized score s(p, t) of the similarity between user preferences and neighbor entities is used to control the transmission of neighbor information through the above noise recognition operation, destroy the propagation of noise information, and obtain the initial neighbor structure vector representation of each entity belonging to the recommended type.
[0058]
[0059] In the formula, represents the initial neighborhood structure vector representation of entity h belonging to the recommended type, e t Represents the initial embedding vector representation of the neighbor entity t of entity h belonging to the recommended type, N h represents the set of neighbor entities t of entity h that belongs to the recommended type.
[0060] Step 4: Capture the neighbor structure information of each entity of the recommended type in the knowledge graph through graph convolution operations, aggregate the initial embedding vector representation of each entity of the recommended type and the initial neighbor structure vector representation of each entity of the recommended type, and obtain the first-order vector representation of the entity of the recommended type. By stacking multiple layers of graph convolution to collect influence signals from high-order neighbors, more information related to the recommended type entities is obtained, and the embedding vector representations of each order of each entity of the recommended type are obtained.
[0061] The initial embedding vector representation of each entity belonging to the recommended type is propagated to its neighboring entities to obtain a first-order entity vector representation, and then this process can be repeated, i.e., further propagating and aggregating l-1-order vector representations to obtain l-order vector representations.
[0062] After a single graph convolutional layer, the final vector representation of an entity belonging to the recommended type depends on itself and its neighbors. The initial embedding vector representation of an entity belonging to the recommended type is propagated to its neighboring entities and aggregated into a first-order vector representation of an entity belonging to the recommended type.
[0063]
[0064] Among them, the 0-order embedding vector of entity h belonging to the recommended type is represented by That is, the initial embedding vector representation e of entity h belonging to the recommended type h ; The 0-order neighbor structure vector representation of entity h belonging to the recommended type That is, the initial neighbor structure vector representation of entity h belonging to the recommended type f is an aggregation function, which takes the sum of two vector representations and then performs a nonlinear transformation. f can be expressed as:
[0065]
[0066] Among them, ReLU is the activation function, W is the weight parameter of the graph convolutional network, and b is the bias parameter of the graph convolutional network.
[0067] By running multiple graph convolution layers in a stack, the noise recognition operation is also used to control the transmission of neighbor information. While destroying the propagation of noise information to achieve denoising, high-order information is captured and aggregated to obtain a high-order entity vector representation of the recommendation type.
[0068]
[0069] Among them, l is the current aggregation iteration number, l = 0, 1, ..., L, and L is the number of graph convolution layers.
[0070] Step 5: Aggregate the user's preference vector representation through graph convolution operations to obtain the vector representation of each item or entity that the user has interacted with. By stacking multiple layers of graph convolution, the user's potential interests can be mined in a more in-depth manner to obtain the user's embedded vector representation of each order.
[0071] In this invention, the items that the user has interacted with, i.e., entities, must be entities of the recommended type. In order to collect more information that can reflect user preferences, the user preference vector representation and the entity vector representation of the user's interaction are aggregated through graph convolution operations, and the vector representation of the user is obtained by stacking multiple layers of graph convolution.
[0072]
[0073] Where ⊙ is the Hadamard product, N u is the set of entities that the user interacts with, |N u | is the number of entities in the collection, represents the user preference vector representation, and the 0th-order embedding vector representation of user u That is, the initial embedding vector representation e of user u u .
[0074] Step 6: Concatenate the user's embedding vector representations of each order to obtain the user's final vector representation At the same time, the embedding vector representations of entities of the recommended type are concatenated to obtain the final vector representation of entities of the recommended type.
[0075] After stacking l layers of graph convolutional layers, we obtain vector representations of entities belonging to the recommended type at different layers. The output vectors of different layers represent the connection information of different layers. The connection operation is used to connect the vector representations of each layer into a vector. Not only can the initial embedding be enriched by performing the embedding propagation operation, but the intensity of the propagation can also be controlled by adjusting the number of layers.
[0076] Final vector representation of the user for:
[0077]
[0078] Final vector representation of entities belonging to the recommended type for:
[0079]
[0080] Among them, || is a connection operation.
[0081] Step 7: Take the inner product of the user's final vector representation and the final vector representation of each entity belonging to the recommended type as the user's score for the entity And recommend the entities with the top k scores to the users.
[0082] Calculate the inner product of the user vector representation and the entity vector representation to predict the probability of the user interacting with the entity:
[0083]
[0084] in, is the final vector representation of the user, is the final vector representation of the entity belonging to the recommended type.
[0085] The present invention provides a knowledge-aware graph convolutional network recommendation method based on denoising, which explicitly proposes the modeling of user preferences and the elimination of noise information in the aggregation process. According to the user's historical interaction information, the rich relationship information between knowledge graph entities is used to capture the vector representation of user preferences. In order to identify noise that cannot reflect user preferences, the similarity score between the user preference and the corresponding neighbor entity is calculated to measure the confidence of whether the neighbor entity is noise. Multiple graph convolutional layers are stacked, and the transmission of neighbor information is controlled by using the normalized similarity score. While obtaining richer embedding, the propagation of noise information is destroyed, thereby achieving better recommendation effect. The technical advantage of the present invention is that it explicitly proposes the importance of modeling user preferences and denoising neighbor entities in the knowledge graph in the knowledge graph recommendation scenario under the graph convolutional network method, and realizes the identification of noise information in the aggregation process of entity information transmission, destroying the propagation of its noise information, thereby improving the recommendation performance.
[0086] It should be noted that although the embodiments of the present invention described above are illustrative, they are not intended to limit the present invention, and therefore the present invention is not limited to the above specific embodiments. Without departing from the principles of the present invention, any other embodiments obtained by those skilled in the art under the guidance of the present invention are deemed to be within the protection of the present invention.
Claims
1. A knowledge-aware graph convolutional network recommendation method based on denoising, applied in the field of movie recommendation, characterized by: The steps include: Step 1: Collect the user's historical data, find all the items that the user has interacted with in the historical data, and align the items with the entities in the knowledge graph; send the user ID, entity ID and relationship ID in the knowledge graph to the embedding layer to obtain the initial embedding vector representation of the user, the initial embedding vector representation of the entity and the initial embedding vector representation of the relationship; Step 2: Using the attention mechanism, first calculate the user's preference vector representation for a single interacted item, and then calculate the user's preference vector representation for all interacted items, that is, the user preference vector representation; Step 3: Calculate and normalize the similarity score between the user preference vector representation and the neighbor entity vector representation through the noise recognition operation. The normalized similarity score is used to control the neighbor information transmission of each entity belonging to the recommended type in the graph convolutional network, destroy the propagation of noise information, and obtain the initial neighbor structure vector representation of each entity belonging to the recommended type. Step 4: Capture the neighbor structure information of each entity of the recommended type in the knowledge graph through graph convolution operations, aggregate the initial embedding vector representation of each entity of the recommended type and the initial neighbor structure vector representation of each entity of the recommended type, and obtain the first-order vector representation of the entity of the recommended type. By stacking multiple layers of graph convolution to collect influence signals from high-order neighbors, more information related to the recommended type entity is obtained, and each order of embedding vector representation of each entity of the recommended type is obtained; Step 5: Aggregate the user preference vector representation through graph convolution operations to represent the vector representation of each item or entity that the user has interacted with. By stacking multiple layers of graph convolution, the user's potential interests are mined in a more in-depth manner to obtain the user's embedding vector representation of each order; Step 6: Concatenate the 0-L order embedding vector representations of the user to obtain the final vector representation of the user; at the same time, concatenate the 0-L order embedding vector representations of the entities belonging to the recommended type to obtain the final vector representation of the entities belonging to the recommended type; Step 7: The inner product of the final vector representation of the user and the final vector representation of each entity of the recommended type is used as the user's score for the entity, and the entities with the top k scores are recommended to the user; k is a set value; In the above formula, e u represents the initial embedding vector representation of user u, Indicates the items that the user has interacted with p j Related relationships The initial embedding vector representation of Represents the user's interaction with a single item p j The preference vector representation of represents the user preference vector representation, Indicates the items that the user has interacted with p j Related relationships The collection of N u is the set of items that the user has interacted with; e t represents the initial embedding vector representation of the neighbor entity t of entity h belonging to the recommended type, represents the initial neighbor structure vector representation of entity h belonging to the recommended type, N h represents the set of neighbor entities t of entity h belonging to the recommended type; represents the l-th order embedding vector representation of entity h belonging to the recommended type, represents the l-1th order embedding vector representation of entity h belonging to the recommended type, The l-1 order neighbor structure vector representation of entity h belonging to the recommended type, l = 0, 1, ..., L, L is the number of graph convolution layers, and f represents the aggregation function; represents the l-order embedding vector representation of user u, |N u | represents the number of items the user has interacted with, ⊙ represents the Hadamard product, and the superscript T represents the transpose.
2. According to claim 1, the method for recommending knowledge-aware graph convolutional networks based on denoising is characterized in that: In step 4, the aggregate function f is: In the formula, represents the l-th order embedding vector representation of entity h belonging to the recommended type, represents the l-1th order embedding vector representation of entity h belonging to the recommended type, Represents the l-1 order neighbor structure vector representation of entity h belonging to the recommended type, W represents the weight parameter of the graph convolutional network, b represents the bias parameter of the graph convolutional network, l = 0, 1, …, L, L is the number of graph convolution layers.
3. The method for recommending knowledge-aware graph convolutional networks based on denoising according to claim 1 or 2, characterized in that: The 0-order embedding vector representation of entity h belonging to the recommended type That is, the initial embedding vector representation e of entity h belonging to the recommended type h ; The 0-order neighbor structure vector representation of entity h belonging to the recommended type That is, the initial neighbor structure vector representation of entity h belonging to the recommended type The 0th-order embedding vector representation of user u That is, the initial embedding vector representation e of user u u .