Self-adaptive recommendation method for fusing graph neural network and knowledge graph

Abstract

The invention discloses a self-adaptive recommendation method for fusing a graph neural network and a knowledge graph. Considering that the knowledge graph is used as emerging auxiliary information and has remarkable semantic expression ability, the method maps the items in the recommendation scene and related attributes thereof into the knowledge graph, and models the similarity between the items by using rich information in the knowledge graph, thereby obtaining item representation with higher expression ability. Information in the knowledge graph is spread through graph convolution calculation, and item representation with rich information is obtained. Thirdly, inputting the item sequence with the time relation into a Transform model for modeling, capturing time sequence information, and forming user interest representation; and finally, by performing attention calculation on the candidate items, generating interest representation of the user and performing recommendation.

Description

Technical Field

[0001] The present invention belongs to the field of recommendation algorithms, and is specifically an adaptive recommendation method that integrates graph neural networks and knowledge graphs, involving knowledge graph embedding, graph neural networks, neighborhood information extraction, user interest extraction, and user interest prediction. Background Art

[0002] With the rapid development of Internet technology, the amount of information that users can obtain is growing exponentially, and the problem of information overload is becoming increasingly serious. When faced with massive amounts of data, it is difficult for users to quickly and efficiently identify and filter out valuable information for themselves. This not only increases the cost of information acquisition for users, but may also lead to decision-making difficulties, information anxiety and other problems, greatly reducing user experience. In this context, how to accurately recommend personalized content has become an important challenge that all platforms need to solve. As an effective information retrieval tool, the recommendation system can help users efficiently obtain the information they need, and its core is the recommendation algorithm. The recommendation process is usually divided into two stages: recall and sorting. In the recall stage, the system selects a small number of candidate instances that users may be interested in from a large number of candidate instances. This stage requires predicting all potential interactions between users and entities, so the amount of calculation is huge. In order to reduce the computational complexity, the recall stage usually uses simplified user and instance representations, such as models based on embedding or sparse features. The sorting stage further refines these instances based on the recall results and finally recommends items with higher scores. Since the number of candidates processed in the ranking stage is relatively small and the computational cost is low, more features can be introduced for modeling, such as user and item attributes, contextual information, behavior sequences, and even social network characteristics to more accurately capture users' preferences for items.

[0003] Traditional deep learning usually deals with structured data, such as text and images; however, data in the real world is often unstructured, such as social networks and knowledge graphs. For this reason, graph neural networks came into being. Graph neural networks are a deep learning model for processing graph structured data, which are widely used in fields such as knowledge graphs and recommendation systems. Its core idea is to learn node representations through information propagation. As a typical graph neural network model, graph convolutional networks generalize convolution operations to graphs. It takes nodes and edges as input, and uses convolution operations to learn the feature representation of nodes, so that each node can update its own representation based on the information of neighboring nodes, thereby deepening the information representation of the current node, and ultimately used to achieve tasks such as data classification, clustering, and prediction.

[0004] The attention mechanism originates from human visual perception, that is, when humans observe an object, they usually do not pay attention to the entire object evenly, but focus on certain key parts. For example, when looking at a person, we usually look at the face first and then pay attention to other details. In deep learning, the attention mechanism is used to help the model extract key information more effectively by assigning different weights to features, thereby making more accurate judgments. Specifically, the attention mechanism is implemented by calculating the attention weight vector. For the pixels in the picture or the words in the sentence, their attention scores are calculated, thereby weighting their feature vectors. In this way, the model can capture key information based on the importance of each feature.

[0005] A knowledge graph is a network diagram that organizes a large amount of knowledge in a structured way. It is generally composed of instances, relationships, and semantic descriptions. Instances can be objects in the real world or some abstract concepts, and relationships represent the associations between instances. Due to its excellent semantic expression ability, knowledge graphs have received widespread attention in recent years. They are presented in the form of heterogeneous graphs, where nodes represent instances and edges represent relationships. In recommendation systems, items and their attributes can be mapped to knowledge in knowledge graphs, making it possible to use the rich information in knowledge graphs to model similarities between items. The heterogeneity of knowledge graphs allows the system to integrate information from different fields, which helps to obtain more expressive item representations.

[0006] In order to effectively utilize the information in the knowledge graph, this paper proposes an adaptive recommendation algorithm that integrates graph neural network and knowledge graph. First, the information in the knowledge graph is propagated through edges using graph convolution calculation to generate item representations containing rich knowledge. Then, Transformer is used to model the item sequence with temporal relationship, capture the temporal features therein, and thus generate the user's interest representation. Finally, by performing attention mechanism calculation on the candidate items, the user's final interest representation is generated, and recommendations are made based on it. Summary of the invention

[0007] In view of this, the technical problem to be solved by the present invention is to provide an adaptive recommendation method integrating graph neural network and knowledge graph, which is an end-to-end deep learning method, consisting of a knowledge graph embedding layer, a neighborhood interest extraction layer, an interest evolution capture layer and a prediction layer. It aims to model the user behavior sequence, obtain the user interest representation, and recommend project instances.

[0008] The technical solution adopted by the present invention to solve the technical problem is as follows:

[0009] An adaptive recommendation method integrating graph neural network and knowledge graph, including:

[0010] Step 1) parameterize instances and relations into embedding vectors;

[0011] Step 2) Learn the neighborhood representation of the item knowledge subgraph and aggregate it to itself;

[0012] Step 3) Capture the dynamic evolution of user behavior history over time, and use the attention mechanism to control the influence of each historical item in the final prediction, and aggregate it into the user's interest representation;

[0013] Step 4) Aggregate the user interest representation and candidate item representation and output the prediction score.

[0014] Further, the step 1) specifically includes the following steps:

[0015] Step 11) Optimize the translation distance to learn the embedding of relations and entities, and construct the mapping vectors of entities and relations respectively.

[0016] Step 12) transform the mapping vector into the relational space and distinguish the head entity from the tail entity.

[0017] Further, the step 2) specifically includes the following steps:

[0018] Step 21) Calculate the knowledge-aware knowledge graph attention to obtain the neighborhood information representation.

[0019] Step 22) Aggregate the neighborhood representation into the item representation to obtain a new item representation with neighbor information.

[0020] Further, the step 3) specifically includes the following steps:

[0021] Step 31) Use Transformer encoding to extract temporal features in the user interest sequence.

[0022] Step 32) uses the attention mechanism to fuse multiple interests into the final user interest representation.

[0023] Further, the step 4) specifically includes the following steps:

[0024] Step 41) Model the concatenated user interest representation and candidate item representation and make prediction results. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] In order to make the purpose, technical solutions and advantages of the present invention more clear, the present invention will be further described in detail below with reference to the accompanying drawings, in which:

[0026] Figure 1 It is a flow chart of the recommendation algorithm provided in the present invention;

[0027] Figure 2This is a diagram of the overall construction framework of the recommendation algorithm provided in the present invention. DETAILED DESCRIPTION

[0028] The preferred embodiments of the present invention will be described in detail below with reference to the accompanying drawings.

[0029] The present invention provides an adaptive recommendation method that integrates graph neural network and knowledge graph, and the method flow chart is as follows: Figure 1 As shown in the figure, the overall model framework diagram proposed by this method is as follows Figure 2 As shown, the method comprises the following steps:

[0030] Step 1) data processing;

[0031] Step 2) Build a knowledge graph embedding layer;

[0032] Step 3) construct a neighborhood interest extraction layer;

[0033] Step 4) constructing a layer to capture interest evolution;

[0034] Step 5) construct the prediction layer;

[0035] Step 6) training the recommendation algorithm model;

[0036] Step 7) Test the recommendation algorithm model.

[0037] Further, step 1) specifically includes the following steps:

[0038] Step 11) Obtain open source data sets and collect corresponding knowledge graphs.

[0039] Step 12) Associate entities in the knowledge graph with items in the user-item interaction.

[0040] Step 13) Sort the user behavior sequence by timestamp to generate training samples for sequence recommendation.

[0041] Further, step 2) specifically includes the following steps:

[0042] Step 21) In addition to the representation vector of each element in the given triple (head entity, relation, tail entity),

[0043] Separately construct entity mapping vectors and relationship mapping vectors. The mapping process is as shown in formula (1)-formula (2):

[0044]

[0045] where a e is the entity mapping vector, a r is the relationship mapping vector, and It is the relationship mapping vector between the head and tail entities.

[0046] Step 22) For the head and tail entities, given the entity representation vector h and the entity projection vector a e There is a one-to-one correspondence between the two entities. This method uses the same projection vector, so there is no need to define an embedding matrix for the head and tail entities separately. The projection matrix is ​​used to complete the projection of the entity to the relationship space. The process is shown in formula (3)-formula (4):

[0047] h ⊥ =A rh ·h (3)

[0048] t ⊥ =A rh ·t (4)

[0049] Step 23) Calculate the corresponding score through formula (5), complete the process of jointly determining the mapping matrix by the entity and the relationship, and enhance the memory of the projection vector for specific knowledge.

[0050]

[0051] Further, step 3) specifically includes the following steps:

[0052] Step 31) After projecting the head and tail entities into the relational space, calculate h using formula (6) ⊥ +r and t ⊥ The attention score between .

[0053]

[0054] Step 32) Calculate the attention of all knowledge triples in the subgraph, obtain the attention score of each item and candidate item, and normalize the attention score through SoftMax.

[0055] Step 33) The normalized attention score is weighted to the tail entity, and then the results are summed to obtain the neighborhood representation. Through the attention operation, the attention of the tail entity (attribute value) that is more closely related to the head entity (item) is increased. The process is shown in formula (7):

[0056]

[0057] Step 34) Aggregate the neighborhood representation and the project representation to obtain the final representation of the project, wherein the aggregation method is the mean pooling aggregation shown in formula (8).

[0058] mean=σ(mean(e h ,e nei )) (8)

[0059] eh represents the embedding vector of the project entity, and mean(·) represents the mean calculation.

[0060] Further, step 4) specifically includes the following steps:

[0061] Step 41) Use Transformer technology to capture the dynamic changes of interests from the user's behavior sequence, so as to better simulate the evolution of the user's interests.

[0062] Step 42) Calculate the similarity between the candidate item and each item in the behavior sequence, and use this as the attention score. After SoftMax, perform weighted summation on the behavior sequence to obtain the user's final interest representation.

[0063] Further, step 5) specifically includes the following steps:

[0064] Step 51) Build a 3-layer perceptron network, the input is the concatenated vector of the user interest representation and the candidate item representation, and model the concatenated vector.

[0065] Further, step 6) specifically includes the following steps:

[0066] Step 61) configure the corresponding configuration, set the training parameters, and carry out the training task;

[0067] Step 62) The training environment configured is: Ubuntu 20.04 LTS, Python 3.8 and Pytorch 2.0.0. The hardware equipment includes CPU: Intel (R) Xeon (R) CPU E5-2680v4 @ 2.40 GHz, memory is 32 GB; GPU: NVIDIA RTX 3090, memory is 24 GB.

[0068] Step 63) The training parameters are configured as follows: 300 training rounds and 0.3 dropout.

[0069] Further, step 7) specifically includes the following steps:

[0070] Step 71) Load the trained weight file into the recommendation algorithm model, select data from the test data set for testing, and return the test results.

[0071] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention rather than to limit it. Although the present invention has been described in detail through the above embodiments, those skilled in the art should understand that various changes can be made in form and details without departing from the scope defined by the claims of the present invention.

Claims

1. An adaptive recommendation method integrating graph neural network and knowledge graph, characterized in that: The following steps are involved: Step 1, collect and process the data set: download the open source data set from the public website, collect information for each subset and generate the corresponding knowledge graph, then associate the entities in it with the items in the user-item interaction record, and filter out the interaction records corresponding to the items that do not exist in the knowledge graph. Finally, sort the user's behavior sequence according to the timestamp to generate training samples for sequence recommendation. And divide it into training set, validation set and test set in the ratio of 8:1:

1. Step 2, construct the knowledge graph embedding layer: In this layer, a knowledge graph embedding method is proposed, which mainly embeds the knowledge graph based on relationship-aware dynamic mapping projection, and generates an embedding vector that can represent the project. This vector is used to capture the characteristics of user preferences affected by project attributes, so as to effectively deal with the influence of project-related attributes on user preferences, thereby enhancing the importance of relationships in knowledge representation. Step 3, construct the neighborhood information extraction layer: The operation of this layer is based on the graph attention network. By calculating the attention score, the information of the node neighborhood can be transferred to the node according to the different weights, so as to propagate the neighborhood information of the project entity along the relationship in the subgraph and obtain a new project representation containing auxiliary information related features. Step 4: Construct a user interest extraction layer: The goal of this layer is to capture the evolution of user interests, so as to obtain a more accurate representation of user interests. The correlation between the user's next behavior and his recent behavior is analyzed based on the item sequence that aggregates auxiliary information in the knowledge graph to improve the subsequent prediction probability. Step 5: Build a multi-layer perceptron network based on step 4 to predict the user's next step. Step 6, train the adaptive recommendation method that integrates graph neural network and knowledge graph: combine the cross entropy loss, L2 loss and regularization items to prevent overfitting while using supervised training methods, thereby obtaining a high-quality prediction model. Step 7, test the adaptive recommendation model that integrates graph neural network and knowledge graph: load the trained model file into the recommendation method network and evaluate the prediction accuracy with the Acc indicator.

2. According to the adaptive recommendation method integrating graph neural network and knowledge graph in claim 1, the method is characterized in that, in the knowledge graph embedding layer constructed in step 2, through this operation, the mapping matrix is ​​jointly determined by the entity and the relationship, which enhances the memory of the projection vector for specific knowledge. At the same time, by distinguishing the relationship projection vectors of the head and tail entities, the model's ability to distinguish the head and tail entities in the relationship space is improved, which makes the method perform better in the many-to-many entity-relationship space in the recommendation scenario.

3. The adaptive recommendation method integrating graph neural network and knowledge graph according to claim 1, characterized in that: The neighborhood information extraction layer constructed in step 3. For a given item, first extract its knowledge subgraph, which contains entities and relations directly related to the item. Then, calculate the similarity between the head entity (item embedding) and the tail entity (attribute embedding) to obtain the attention score. These scores reflect the importance of each attribute. Next, use the attention score to weight the tail entity to generate a neighborhood embedding that integrates all relevant attribute information. Finally, aggregate the neighborhood embedding with the item embedding to form the final neighborhood information representation.

4. The adaptive recommendation method integrating graph neural network and knowledge graph according to claim 1, characterized in that: The user feature extraction layer constructed in step 4. The overall model consists of an interest evolution capture layer and a user interest aggregation layer. The interest evolution capture layer uses a Transformer encoder to capture the temporal features in the user interest sequence, and uses a self-attention mechanism to contextually encode each interest point to generate a rich interest representation. The user interest aggregation layer uses an attention mechanism to aggregate multiple interest representations into the final user interest representation. The attention mechanism weights each interest point according to its importance, dynamically adjusts the weight, and forms a unified interest representation. This method can not only capture the temporal changes of user interests, but also effectively aggregate diverse interest information, providing a more accurate description of user interests for personalized recommendations.

Images