Object recommendation method and device based on path reasoning, electronic device

Abstract

The application discloses a path reasoning-based object recommendation method, and belongs to the technical field of computers, which helps to improve the matching degree of recommended objects and users. The method comprises the following steps: learning a first knowledge graph expressing the relationship between preset entities by a first graph convolutional neural network to obtain a first vector representation of the entities; wherein the entities comprise users and recommended objects; determining a plurality of paths between the entities based on the first vector representation of the entities by executing a path reasoning model of reinforcement learning; learning the relationship between the entities expressed by the plurality of paths, updating the first vector representation of the entities according to the learning result, and iteratively executing the path reasoning model until the execution result of the path reasoning model meets a preset iteration termination condition, and screening a path from the users to the recommended objects from the plurality of paths obtained through the final reasoning; and recommending the recommended objects to the users according to the path from the users to the recommended objects screened.

Description

Technical Field

[0001] This application relates to the field of computer technology, and in particular to an object recommendation method, apparatus, electronic device, and computer-readable storage medium based on path reasoning. Background Technology

[0002] In existing technologies, a path-based reasoning recommendation method is used when recommending goods, information, and services to users. This method involves constructing a static knowledge graph based on user-to-product decision records, reasoning multi-hop paths from the user to the product based on the structure of the static knowledge graph, and then recommending products based on these inferred multi-hop paths. This existing path-based reasoning recommendation method models the path-walking problem as a Markov decision process and uses an action-policy network from reinforcement learning to solve for the multi-hop paths from the user to the product in the graph. The action-policy network in this method relies on pre-trained vector representations of nodes or entities on the static graph, and the quality of these pre-trained vector representations significantly impacts the performance of the final model. Low-quality pre-trained vector representations can lead to a decrease in the matching degree between recommended products and users.

[0003] Existing path-reasoning-based object recommendation methods still need improvement. Summary of the Invention

[0004] This application provides an object recommendation method based on path reasoning, which helps to improve the matching degree between recommended objects and users.

[0005] In a first aspect, embodiments of this application provide an object recommendation method based on path reasoning, including:

[0006] A first knowledge graph expressing the relationships between preset entities is learned through a first graph convolutional neural network to obtain a first vector representation of the entity; wherein, the entity includes a first entity and a second entity, the first entity including: a user; the second entity including: a recommendation object;

[0007] Based on the first vector representation of the entity, several paths between the entities are determined by executing a path reasoning model of reinforcement learning;

[0008] In response to the fact that the execution result of the path reasoning model of the reinforcement learning does not meet the preset iteration termination condition, the relationship between the entities expressed by the several paths is learned, and the first vector representation of the entity is updated according to the learning result. Then, the process jumps to the step of determining the several paths between the entities by executing the path reasoning model of the reinforcement learning based on the first vector representation of the entity.

[0009] In response to the execution result of the path reasoning model that performs reinforcement learning satisfying a preset iteration termination condition, the path from the user to the recommended object is selected from the plurality of paths;

[0010] Based on the path from the user to the recommended object obtained through filtering, the recommended object is recommended to the user.

[0011] Secondly, embodiments of this application provide an object recommendation device based on path reasoning, including:

[0012] The first vector representation learning module is used to learn from a first knowledge graph expressing the relationships between preset entities through a first graph convolutional neural network to obtain a first vector representation of the entity; wherein, the entity includes a first entity and a second entity, the first entity including: a user; the second entity including: a recommendation object;

[0013] The path reasoning module is used to determine several paths between the entities based on the first vector representation of the entities by executing a path reasoning model of reinforcement learning;

[0014] The vector update module is used to learn the relationship between the entities expressed by the several paths in response to the execution result of the path reasoning module not meeting the preset iteration termination condition, and update the first vector representation of the entity according to the learning result. Then, it jumps to execute the path reasoning module.

[0015] A path filtering module is used to filter the path from the user to the recommended object from the plurality of paths in response to the execution result of the path reasoning module satisfying a preset iteration termination condition;

[0016] The object recommendation module is used to recommend the recommended object to the user based on the path from the user to the recommended object obtained through filtering.

[0017] Thirdly, embodiments of this application also disclose an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, it implements the object recommendation method based on path reasoning described in embodiments of this application.

[0018] Fourthly, embodiments of this application provide a computer-readable storage medium storing a computer program thereon, which, when executed by a processor, represents the steps of the object recommendation method based on path reasoning disclosed in embodiments of this application.

[0019] The object recommendation method based on path reasoning disclosed in this application uses a first graph convolutional neural network to learn a first knowledge graph expressing the relationships between preset entities, obtaining a first vector representation of the entity. The entity includes a user and a recommended object. Based on the first vector representation of the entity, a path reasoning model based on reinforcement learning is executed to determine several paths between the entities. In response to the path reasoning model's execution result not satisfying a preset iteration termination condition, the relationships between the entities expressed by the several paths are learned, and the first vector representation of the entity is updated according to the learning result. The path reasoning model is iteratively executed until its execution result satisfies the preset iteration termination condition. Then, the path from the user to the recommended object is selected from the several paths. Based on the selected path from the user to the recommended object, the recommended object is recommended to the user, which helps improve the matching degree between the recommended object and the user.

[0020] The above description is only an overview of the technical solution of this application. In order to better understand the technical means of this application and to implement it in accordance with the contents of the specification, and to make the above and other objects, features and advantages of this application more obvious and understandable, the following are specific embodiments of this application. Attached Figure Description

[0021] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0022] Figure 1 This is a flowchart of the object recommendation method based on path reasoning in Embodiment 1 of this application;

[0023] Figure 2 This is a schematic diagram of the first knowledge graph in Embodiment 1 of this application;

[0024] Figure 3 This is a schematic diagram of the object recommendation device based on path reasoning according to Embodiment 2 of this application;

[0025] Figure 4 A block diagram schematically illustrates an electronic device for performing the method according to this application; and

[0026] Figure 5 A storage unit for holding or carrying program code that implements the method according to this application is schematically shown. Detailed Implementation

[0027] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0028] Example 1

[0029] This application discloses an object recommendation method based on path reasoning, such as... Figure 1 As shown, the method includes steps 100 to 160.

[0030] Step 100: The first knowledge graph expressing the relationship between preset entities is learned through the first graph convolutional neural network to obtain the first vector representation of the entity.

[0031] The entity includes a first entity and a second entity, wherein the first entity further includes a user; and the second entity further includes one or more of the following: a recommendation object, user attributes, and recommendation object attributes.

[0032] The path-reasoning-based object recommendation method described in this application is applicable to recommending browsing and clicking objects such as goods, services, and information to users on application pages or web pages. For example, the objects can be dishes, books, clothing, tourist attractions, entertainment projects, etc. The objects can be recommended to users through images or text displayed on the page, or through other channels or carriers. This application does not limit the specific object recommendation channels and forms.

[0033] The preset entity described in this application embodiment includes: a first entity, which refers to a user. The preset entity also includes: a second entity, which is distinct from the first entity. The second entity includes: a recommendation object, user attributes, and recommendation object attributes. The second entity varies depending on the recommendation scenario. For example, in an application scenario recommending dishes to a user, the recommendation object includes: the dishes; the user attributes include: taste preferences, preferred shopping areas, etc.; and the recommendation object attributes include: cuisine, flavor, ingredients, merchants, etc. As another example, in an application scenario recommending clothing to a user, the recommendation object includes: clothing; the user attributes may include: gender, age; and the recommendation object attributes may include: merchants, brands, styles, colors, fabrics, etc.

[0034] In different application scenarios, the relationships between entities are defined as different actions. For example, in an application scenario that recommends dishes to users, the relationships between entities include one or more of the following: purchase, browsing, etc.; in an application scenario that recommends clothing to users, the relationships between entities include one or more of the following: purchase, browsing, consultation, etc.

[0035] In specific implementation of this application, it is first necessary to obtain a first knowledge graph based on the relationships between entities in a specified application scenario. In some embodiments of this application, the first knowledge graph can be represented as... Here, each triple (e,r,e′) represents the fact that from the head entity e to the tail entity e′ under relation r. Let ξ represent a set of relationships (e.g., including click behavior, purchase behavior), and let ξ represent a set of entities. The first knowledge graph includes: a first entity set and a second entity set, and a relationship r connecting the first and second entities. ui The first knowledge graph can be a directed graph, with vertices constructed based on entities and edges constructed based on relationships between entities.

[0036] For example, in a menu recommendation scenario, a user-menu knowledge graph is constructed based on users' historical browsing and purchase data. This user-menu knowledge graph includes: a user set u (i.e., the aforementioned first entity set) and a menu set. (i.e., the aforementioned second set of entities), entities with interactive relationships constitute the user-menu knowledge graph. And the relationship between users and dishes. ui In practice, for example, a triple can be obtained based on the fact that user A browsed dish 1.

[0037] In some embodiments of this application, for a pre-constructed first knowledge graph, unsupervised learning can be performed on the first knowledge graph using a first graph convolutional neural network to obtain the first vector representation of each entity in the first knowledge graph. The first graph convolutional neural network described in the embodiments of this application is a graph convolutional neural network. The specific training process of the first graph convolutional neural network is described in the prior art and will not be repeated in the embodiments of this application.

[0038] For a detailed implementation of unsupervised learning of the first knowledge graph by using a first graph convolutional neural network to obtain the first vector representation of each entity in the first knowledge graph, please refer to the prior art, and it will not be repeated in the embodiments of this application.

[0039] By learning the relationships between entities in the first knowledge graph through a first graph convolutional neural network, a first vector representation of the first entity included in the first knowledge graph and a first vector representation of the second entity included in the first knowledge graph can be obtained.

[0040] Step 120: Based on the first vector representation of the entity, determine several paths between the entities by executing a path reasoning model of reinforcement learning.

[0041] In order to generate a set of recommended objects for a given user u, a specified number of objects, and a defined relationship density, this application constructs a first knowledge graph based on the user's historical interaction data with the objects. Then, the set of recommended objects generated for the user is further determined by analyzing the paths from the user to the objects in the first knowledge graph.

[0042] To infer effective paths between users and objects, we model the problem of finding effective paths as a Markov decision process and use reinforcement learning methods to solve for effective paths. In some embodiments of this application, based on the first vector representation of the entities, several paths between the entities are determined by executing a path reasoning model of reinforcement learning, including: executing a pre-built Markov decision process model to solve for several paths between the entities; the elements of the Markov decision process include: state, action, policy, and reward, wherein the state of the current step is represented as: a triple consisting of the first vector representation of the starting entity of the solved multi-hop path, the first vector representation of the entity reached by the current step of the multi-hop path, and the historical state corresponding to the current step; the action of the current step is represented as: executing a preset number of actions with the highest preference in the action space corresponding to the state of the current step; the policy is represented as: the probability distribution of executing the next action in the state of the current step; the reward is represented as: obtaining a positive reward if the multi-hop path reaches the specified second entity, otherwise, no reward is obtained. The path is a sequence of entities connecting entities. For example, a sequence of recommendation objects connecting users to recommendation objects.

[0043] In some embodiments of this application, the problem of solving the multi-hop path problem from the first entity to the second entity is modeled as a Markov decision process model. Based on the entity vectors in the first knowledge graph and the action relationships between entities represented in the first knowledge graph, the elements of the Markov decision process model are represented. Then, by executing the Markov decision process model, the effective path between entities can be solved. In the embodiments of this application, a path refers to a sequence of entities connected by a series of actions from one entity to another. The Markov decision process model includes four elements: state, action, policy, and reward. The following examples illustrate how the four elements of the Markov decision process model are implemented in the scenario of solving multi-hop paths between entities, to help readers understand the process of modeling the problem of solving the multi-hop path problem from the first entity to the second entity as a Markov decision process model.

[0044] One of the elements: State

[0045] In scenarios involving solving multi-hop paths between entities, the state refers to the state at step t of the action performed from one entity to another, where t is an integer greater than or equal to 1. In some embodiments of this application, taking the current step as step t as an example, the state s at step t... t It can be defined as a triple (u, e) t ,h t ), where u is the given starting entity (such as a user), e t h represents the entity reached in the current step (selected from the second entity, such as a dish). t To record the historical state corresponding to step t, given a step size T, the final state s is... T It can be represented as s T =(u,e T ,h T ).

[0046] Element Two: Action

[0047] If state s t The action space is defined as A t Then state s t Action space A t It can be represented as: in, For the first knowledge graph, e t Let s represent the entity reached in step t, and state s in step t. t The action space is entity e t Excluding historical relationships and entities, consider all possible out-degree edges, i.e., the actions corresponding to all entities that can be reached in the next step of step t. Then, from state s... t A preset number of actions are selected from the action space for path reasoning.

[0048] As the foregoing analysis shows, the action space is affected by the out-degree of entities. In order to dynamically capture the current state's preference for the next action and prune the action space, some embodiments of this application introduce a multi-head attention mechanism based on the vector representations of the current state and actions to calculate an attention score α for each action. i Actions are selected from the action space of the current state based on attention scores. For example, a preset number of actions with the highest execution preference in the action space corresponding to the current state are determined by the following method: the action space of the current state is determined based on the first knowledge graph; an attention score for each action is calculated using a preset multi-head attention mechanism network based on the vector representation of the current state and the vector representation of each action in the action space; and a preset number of actions are selected in descending order of attention scores as the preset number of actions with the highest execution preference in the action space corresponding to the current state.

[0049] The input to the multi-head attention mechanism network is the vector representation s of the current state. t Vector representations of actions in the action space of the current state The multi-head attention mechanism network can be represented as: Where i is the action index. α represents the vector representation of the i-th action in the action space of the t-th state. i Let represent the attention score of the i-th action in the action space of state t. Then, select a preset number of actions with the highest attention scores in descending order as the actions most likely to be executed in the next step of state t.

[0050] In some embodiments of this application, the vector representations of different actions can be determined by vector encoding methods in the prior art (e.g., encoding actions such as clicking and purchasing separately using word2vec), and this application does not limit this.

[0051] Element Three: Strategy

[0052] In some embodiments of this application, the decision network of the Markov decision process model is a probability distribution network for calculating the next action in the current state. In this embodiment, the decision network of the Markov decision process model is defined as: π(a t |s t A t ),a t ∈A t , where a t s represents the next action at step t. tLet A represent the state at step t (i.e., the current state). t Let represent the action space of the state at step t. The decision network calculates the probability of executing each action in the action space in the next step based on the state at step t and the action space.

[0053] In some embodiments of this application, the decision network may be a classification network, and the decision network may be represented as: π(a t |s t A t =Softmax(α), where α is the attention score vector composed of the attention scores of all actions in the action space of the current state.

[0054] In some embodiments of this application, to prevent model overfitting, the probability distribution of the current step's state executing the next action is determined by the following method: randomly masking the attention score vector composed of the attention scores of all actions in the action space using a binary mask vector of the action space; and calculating the probability distribution of the current step's state executing the next action based on the masked attention score vector. For example, the decision network can be represented as: in, For state s t Action space A t The binary mask vector, through the binary mask vector Some actions in the action space can be randomly blocked.

[0055] Element Four: Rewards

[0056] Given any user, during path reasoning, the model does not know what entity the final state will reach. To incentivize the policy network to follow a reasonable or correct path, in some embodiments of this application, a binary reward function is used to incentivize the policy network to follow a reasonable or correct path during the execution of the Markov decision process model. For example, a positive reward is given when the path reaches the target entity; otherwise, no reward is given. The reward can be expressed as: Among them, e T Target represents the entity reached by the final state obtained from path reasoning, and Target represents the tail entity in the triple of the multi-hop path.

[0057] By executing the Markov decision process model constructed using the above method, for a given entity and a first knowledge graph, the Markov decision process model will infer one or more paths with that entity as the head entity. Taking the application scenario of recommending dishes to users as an example, by executing the Markov decision process model constructed using the above method, for a given user, the Markov decision process model will infer one or more paths from the user to second entities such as dishes, merchants, or flavors.

[0058] Step 130: Determine whether the execution result of the path reasoning model that performs reinforcement learning meets the preset iteration termination condition. If yes, proceed to step 140; otherwise, proceed to step 150.

[0059] To improve the accuracy of path reasoning, this application sets up an iterative execution flow for the Markov Decision Process (MDP) model. For example, an iteration termination condition can be pre-set based on the execution results of the path reasoning steps. During model execution, if the result of path reasoning using the MDP model meets the pre-set iteration termination condition, the model iteration execution stops, and the data processing step based on the result reasoned from the MDP model continues. If the pre-set iteration termination condition is not met, the MDP reasoning process is repeated. In some embodiments of this application, the iteration termination condition can be that the number of iterations reaches a preset number, or that the loss value of the MDP model is less than a preset loss value threshold.

[0060] Step 140: In response to the fact that the execution result of the path reasoning model performing reinforcement learning does not meet the preset iteration termination condition, the relationship between the entities expressed by the several paths is learned, and the first vector representation of the entity is updated according to the learning result. Then, the process jumps to step 120.

[0061] When repeatedly executing the Markov Decision Process (MDP) model, this embodiment of the application fine-tunes the vector representation of entities in the first knowledge graph, enabling the MDP model to infer more efficient or reasonable paths when performing path reasoning based on the vector representation of entities in the first knowledge graph again. In some embodiments of this application, the relationships between entities expressed in several paths obtained by the currently executed MDP model are further learned to obtain the second vector representation of each entity included in the path, and the first vector representation of the corresponding entity in the first knowledge graph is fine-tuned using the second vector representation of each entity.

[0062] In some embodiments of this application, learning the relationships between the entities expressed by the plurality of paths and updating the first vector representation of the entities based on the learning results includes: constructing a second knowledge graph based on the plurality of paths, wherein the second knowledge graph is used to express the relationships between the entities included in the entity sequences corresponding to the plurality of paths; learning the structural features in the second knowledge graph through a second graph convolutional neural network to obtain a second vector representation of the entity; and updating the first vector representation of the corresponding entity through the second vector representation of the entity.

[0063] As described above, given entity e0 and path length k, a path from e0 to e can be obtained through the path reasoning process described above. k k-hop path Therefore, by reasoning through the first knowledge graph, a set of several paths can be obtained. Taking the application scenario of recommending dishes to users as an example, by reasoning through the user-dish knowledge graph, for all users u, a set containing N paths can be obtained. ε represents the set of entities corresponding to the user-dish knowledge graph.

[0064] Then, a second knowledge graph is constructed based on all paths obtained through path reasoning from the currently executed Markov decision process model. Specific implementation methods for constructing the second knowledge graph based on all paths obtained through path reasoning from the currently executed Markov decision process model are described in existing technologies and will not be repeated in the embodiments of this application. In some embodiments of this application, the second knowledge graph can be represented as: Where ε represents the set of entities in the first knowledge graph, V represents the set of relations in the first knowledge graph, V represents the vertex of the second knowledge graph, and E represents the edge of the second knowledge graph. The second knowledge graph is used to express the entities included in the several paths determined in the preceding steps and the relationships between the entities.

[0065] Taking the application scenario of recommending dishes to users as an example, assuming that the user's interaction data with the dishes includes: user A clicked on dish 1 and then purchased dish 2, then the first knowledge graph will include the vertex corresponding to user A (e.g., ...). Figure 2 Vertex A in the table), the vertex corresponding to dish 1 (e.g., vertex A in the table), and the ver Figure 2 Vertex N1 in the table), and the vertex corresponding to dish 2 (e.g., ... Figure 2 Using vertex N11, an edge connecting user A and dish 1, and an edge connecting dish 1 and dish 2, a Markov decision process model is executed for path reasoning. This yields the path from user A to dish 1 and then to dish 2. A second knowledge graph is then constructed based on these paths obtained through the Markov decision process model. At that time, based on the path from user A to dish 1 to dish 2, vertices corresponding to user A, dish 1 and dish 2 will be constructed respectively, as well as edges connecting user A to dish 1 and edges connecting dish 1 to dish 2.

[0066] Next, the second knowledge graph is processed using a graph convolutional neural network. The second knowledge graph is obtained by learning the structural features in the graph. The second vector representation of each entity in the graph. Graph convolutional neural networks (GNNs) generalize from grid data of images to graph data of arbitrary structures through neighbor aggregation. By aggregating neighbor nodes and their own features, GNNs can learn a second knowledge graph. The local structural features, i.e., the vector representations of directly connected entities, are similar. Furthermore, as the number of convolutional layers increases, graph convolutional neural networks can capture higher-order neighbor features, such as structural features between non-adjacent nodes and features within the nodes themselves. Let's continue with the second knowledge graph constructed in the aforementioned application scenario of recommending dishes to users. For example, the graph convolutional neural network can learn not only the similarity between user A and dish 1, but also the similarity between user A and dish 2. Therefore, learning the second knowledge graph through the graph convolutional neural network is equivalent to learning the effective interaction paths between entities, and the resulting vector representations of the entities (i.e., the second vector representations) will be able to more accurately express the relationships between entities.

[0067] Furthermore, the first vector representation of the corresponding entity in the first knowledge graph is updated by learning the second vector representation of the entity from the second knowledge graph. For example, based on the second knowledge graph... The vector representation of user A learned from the second knowledge graph is used to update the first vector representation of user A in the first knowledge graph; based on the vector representation of user A learned from the second knowledge graph... The vector representation of dish 1 learned from the first knowledge graph is used to update the first vector representation of dish 1 in the first knowledge graph. By updating the vector, the Markov decision process model can be executed again in the next time, and path reasoning can be performed based on the entity vector representation learned from the effective path, thereby improving the accuracy of the reasoned path.

[0068] Step 150: In response to the execution result of the path reasoning model performing reinforcement learning satisfying the preset iteration termination condition, the path from the user to the recommended object is selected from the plurality of paths.

[0069] If the result of path reasoning using the Markov decision process model meets a pre-set iteration termination condition, then the model iteration execution stops, and the data processing step based on the result obtained from the Markov decision process model continues. For example, the loss value generated by path reasoning is less than a preset loss value threshold, or the number of times the Markov decision process model is iterated reaches a preset number threshold.

[0070] The purpose of path reasoning in this embodiment is to generate a set of recommendation objects {i} for user u, given a user u, a path hop count K, and a number of recommendation objects N. n} n∈[N] In each user-object pair (u, i) n ) through a reasoning path p k (u,i n (2≤k≤K) associations. To learn more comprehensive association information, in this embodiment, when constructing the first knowledge graph, the entities corresponding to the vertices of the first knowledge graph include not only users and recommended objects, but also entities such as user attributes and recommended objects. Therefore, it is necessary to filter the paths obtained by performing path reasoning on the Markov decision process model, retaining only the path from user to recommended object.

[0071] Step 160: Recommend the recommended object to the user based on the path from the user to the recommended object obtained through filtering.

[0072] During the recommendation process, when it is necessary to determine the recommended object for the user, the entity in the path from the user to the recommended object can be matched with the object currently reached by the user. The next entity to be reached on the path of the successfully matched entity is then used as the recommended object and recommended to the user.

[0073] In other embodiments of this application, other methods may be used to recommend the recommended object to the user based on the path from the filtered user to the recommended object. These methods will not be listed individually in this application. This application does not limit the specific implementation method for recommending the recommended object to the user based on the path from the filtered user to the recommended object.

[0074] The object recommendation method based on path reasoning disclosed in this application uses a first graph convolutional neural network to learn a first knowledge graph expressing the relationships between preset entities, obtaining a first vector representation of the entity. The entity includes a user and a recommended object. Based on the first vector representation of the entity, a path reasoning model based on reinforcement learning is executed to determine several paths between the entities. In response to the path reasoning model's execution result not satisfying a preset iteration termination condition, the relationships between the entities expressed by the several paths are learned, and the first vector representation of the entity is updated according to the learning result. The path reasoning model is iteratively executed until its execution result satisfies the preset iteration termination condition. Then, the path from the user to the recommended object is selected from the several paths. Based on the selected path from the user to the recommended object, the recommended object is recommended to the user, which helps improve the matching degree between the recommended object and the user.

[0075] The path-reasoning-based object recommendation method disclosed in this application combines a graph convolutional neural network (GCNN) and a path-reasoning interpretability recommendation model. First, reinforcement learning is used to infer multi-hop paths from the user to recommended objects. These paths are then used in the learning of the GCNN. Finally, the entity vector representations learned by the GCNN are further applied to path reasoning. Dynamically learning entity vectors for object recommendation improves the matching degree between recommended objects and users. Furthermore, since the path-reasoning method learns relationships between entities without relying on manually defined paths or rules, it can more accurately reflect the association between users and recommended objects, further improving the matching degree. On the other hand, using paths learned from the knowledge graph updated by the aforementioned method for object recommendation can improve the interpretability of recommended objects.

[0076] Example 2

[0077] This application discloses an object recommendation device based on path reasoning, such as... Figure 3 As shown, the device includes:

[0078] The first vector representation learning module 310 is used to learn from a first knowledge graph expressing the relationships between preset entities through a first graph convolutional neural network to obtain a first vector representation of the entity; wherein, the entity includes a first entity and a second entity, the first entity including: a user; the second entity including: a recommendation object;

[0079] The path reasoning module 320 is used to determine several paths between the entities based on the first vector representation of the entities by executing a path reasoning model of reinforcement learning;

[0080] The vector update module 330 is used to learn the relationship between the entities expressed by the plurality of paths in response to the execution result of the path reasoning module not meeting the preset iteration termination condition, and update the first vector representation of the entity according to the learning result, and then jump to execute the path reasoning module 320.

[0081] The path filtering module 340 is used to filter the path from the user to the recommended object from the plurality of paths in response to the execution result of the path reasoning module 320 satisfying the preset iteration termination condition;

[0082] The object recommendation module 350 is used to recommend the recommended object to the user based on the path from the user to the recommended object obtained through filtering.

[0083] In some embodiments of this application, the vector update module 330 is further configured to:

[0084] A second knowledge graph is constructed based on the plurality of paths, wherein the second knowledge graph is used to express the relationships between the entities included in the entity sequences corresponding to the plurality of paths;

[0085] The second convolutional neural network is used to learn the structural features in the second knowledge graph to obtain the second vector representation of the entity;

[0086] The first vector representation of the corresponding entity is updated by the second vector representation of the entity.

[0087] In some embodiments of this application, the path reasoning module 320 is further configured to:

[0088] A pre-built Markov decision process model is executed to solve for several paths between the entities; the elements of the Markov decision process include: state, action, policy, and reward, wherein,

[0089] The state of the current step is represented as a triple consisting of the first vector representation of the starting entity of the solved multi-hop path, the first vector representation of the entity reached by the current step of the multi-hop path, and the historical state corresponding to the current step.

[0090] The action in the current step is represented as: executing the preset number of actions with the highest preference in the action space corresponding to the state of the current step;

[0091] The strategy is represented as: the probability distribution of executing the next action based on the current state;

[0092] The reward is defined as follows: if the multi-hop path reaches the designated second entity, a positive reward is obtained; otherwise, no reward is obtained.

[0093] In some embodiments of this application, the preset number of actions with the highest execution preference in the action space corresponding to the state of the current step are determined by the following method:

[0094] The action space of the current step's state is determined based on the first knowledge graph.

[0095] By using a pre-defined multi-head attention mechanism network, the attention score of each action is calculated based on the vector representation of the state of the current step and the vector representation of each action in the action space.

[0096] Select a preset number of actions according to the attention scores from high to low, and execute the preset number of actions with the highest preference in the action space corresponding to the state of the current step.

[0097] In some embodiments of this application, the probability distribution of the current step's state executing the next action is determined by the following methods, including:

[0098] The attention score vector, composed of the attention scores of all actions in the action space, is randomly masked using the binary mask vector of the action space.

[0099] Based on the attention score vector after masking, calculate the probability distribution of executing the next action in the current step's state.

[0100] In some embodiments of this application, the second entity further includes: user attributes and / or recommendation object attributes.

[0101] The object recommendation device based on path reasoning disclosed in this application is used to implement the object recommendation method based on path reasoning described in Embodiment 1 of this application. The specific implementation methods of each module of the device will not be repeated here, but can be found in the specific implementation methods of the corresponding steps in the method embodiment.

[0102] The object recommendation device based on path reasoning disclosed in this application learns from a first knowledge graph expressing the relationships between preset entities using a first graph convolutional neural network to obtain a first vector representation of the entity. The entity includes a user and a recommended object. Based on the first vector representation of the entity, a path reasoning model based on reinforcement learning is executed to determine several paths between the entities. In response to the path reasoning model's execution result not satisfying a preset iteration termination condition, the relationships between the entities expressed by the several paths are learned, and the first vector representation of the entity is updated according to the learning result. The path reasoning model is iteratively executed until its execution result satisfies the preset iteration termination condition. Then, the path from the user to the recommended object is selected from the several paths. Based on the selected path from the user to the recommended object, the recommended object is recommended to the user, which helps improve the matching degree between the recommended object and the user.

[0103] The object recommendation device based on path reasoning disclosed in this application combines a graph convolutional neural network (GCNN) and a path reasoning-based interpretable recommendation model. First, reinforcement learning is used to infer multi-hop paths from the user to recommended objects. These paths are then used in the learning of the GCNN. Finally, the entity vector representations learned by the GCNN are further applied to path reasoning. Dynamically learning entity vectors for object recommendation improves the matching degree between recommended objects and users. Furthermore, since the path reasoning method learns relationships between entities without relying on manually defined paths or rules, it can more accurately reflect the association between users and recommended objects, further improving the matching degree. On the other hand, object recommendation based on paths learned from the knowledge graph updated by the aforementioned method can improve the interpretability of recommended objects.

[0104] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on its differences from other embodiments. Similar or identical parts between embodiments can be referred to interchangeably. For the apparatus embodiments, since they are fundamentally similar to the method embodiments, the description is relatively simple; relevant parts can be referred to the descriptions in the method embodiments.

[0105] The above provides a detailed description of the object recommendation method and apparatus based on path reasoning provided in this application. Specific examples have been used to illustrate the principles and implementation methods of this application. The description of the above embodiments is only for the purpose of helping to understand the method of this application and its core idea. At the same time, for those skilled in the art, there will be changes in the specific implementation methods and application scope based on the idea of ​​this application. Therefore, the content of this specification should not be construed as a limitation of this application.

[0106] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without any creative effort.

[0107] The various component embodiments of this application can be implemented in hardware, or as software modules running on one or more processors, or a combination thereof. Those skilled in the art will understand that microprocessors or digital signal processors (DSPs) can be used in practice to implement some or all of the functions of some or all of the components in the electronic device according to the embodiments of this application. This application can also be implemented as a device or apparatus program (e.g., a computer program and computer program product) for performing part or all of the methods described herein. Such a program implementing this application can be stored on a computer-readable medium, or can be in the form of one or more signals. Such signals can be downloaded from an Internet website, provided on a carrier signal, or provided in any other form.

[0108] For example, Figure 4 An electronic device is shown that can implement the methods according to this application. The electronic device may be a PC, mobile terminal, personal digital assistant, tablet computer, etc. The electronic device conventionally includes a processor 410 and a memory 420, and program code 430 stored on the memory 420 and executable on the processor 410, which, when executing the program code 430, implements the methods described in the above embodiments. The memory 420 may be a computer program product or a computer-readable medium. The memory 420 may be an electronic memory such as flash memory, EEPROM (Electrically Erasable Programmable Read-Only Memory), EPROM, hard disk, or ROM. The memory 420 has a storage space 4201 for the program code 430 of a computer program for performing any of the method steps described above. For example, the storage space 4201 for the program code 430 may include various computer programs for implementing the various steps in the above methods. The program code 430 is computer-readable code. These computer programs can be read from or written to one or more computer program products. These computer program products include program code carriers such as hard disks, CDs, memory cards, or floppy disks. The computer program includes computer-readable code that, when executed on an electronic device, causes the electronic device to perform the method according to the above embodiments.

[0109] This application also discloses a computer-readable storage medium storing a computer program thereon, which, when executed by a processor, implements the steps of the object recommendation method based on path reasoning as described in Embodiment 1 of this application.

[0110] Such a computer program product can be a computer-readable storage medium, which can have the same characteristics as... Figure 4 The memory 420 in the illustrated electronic device is similarly arranged with storage segments, storage spaces, etc. Program code can be stored, for example, in a compressed form on the computer-readable storage medium. The computer-readable storage medium is typically as shown in the reference... Figure 5 The portable or fixed storage unit is described above. Typically, the storage unit includes computer-readable code 430', which is code read by a processor and, when executed by the processor, implements the various steps in the method described above.

[0111] The terms "an embodiment," "embodiment," or "one or more embodiments" as used herein mean that a particular feature, structure, or characteristic described in connection with an embodiment is included in at least one embodiment of this application. Furthermore, please note that the examples of the phrase "in one embodiment" do not necessarily all refer to the same embodiment.

[0112] Numerous specific details are set forth in the specification provided herein. However, it will be understood that embodiments of this application may be practiced without these specific details. In some instances, well-known methods, structures, and techniques have not been shown in detail so as not to obscure the understanding of this specification.

[0113] In the claims, any reference signs placed between parentheses should not be construed as limiting the claims. The word "comprising" does not exclude the presence of elements or steps not listed in the claims. The word "a" or "an" preceding an element does not exclude the presence of a plurality of such elements. This application can be implemented by means of hardware comprising several different elements and by means of a suitably programmed computer. In a unit claim enumerating several means, several of these means may be embodied by the same item of hardware. The use of the words first, second, and third, etc., does not indicate any order. These words can be interpreted as names.

[0114] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application.

Claims

1. An object recommendation method based on path reasoning, characterized in that, include: By using a first graph convolutional neural network, a first knowledge graph expressing the relationships between preset entities is learned to obtain a first vector representation of the entity; The entity includes a first entity and a second entity, wherein the first entity includes a user; The second entity includes the recommended object; Based on the first vector representation of the entity, several paths between the entities are determined by executing a path reasoning model of reinforcement learning; In response to the fact that the execution result of the path reasoning model for performing reinforcement learning does not meet the preset iteration termination condition, a second knowledge graph is constructed based on the several paths; The second knowledge graph is used to express the relationships between the entities included in the entity sequences corresponding to the plurality of paths; The second convolutional neural network is used to learn the structural features in the second knowledge graph to obtain the second vector representation of the entity; The process involves updating the first vector representation of the corresponding entity using the second vector representation of the entity, and then jumping to the step of determining several paths between the entities by executing a path reasoning model based on the first vector representation of the entity and performing reinforcement learning. In response to the execution result of the path reasoning model that performs reinforcement learning satisfying a preset iteration termination condition, the path from the user to the recommended object is selected from the plurality of paths; Based on the path from the user to the recommended object obtained through filtering, the recommended object is recommended to the user.

2. The method according to claim 1, characterized in that, The step of determining several paths between entities based on the first vector representation of the entities by executing a path reasoning model of reinforcement learning includes: Execute a pre-built Markov decision process model to solve for multi-hop paths between the entities; The elements of the Markov decision process include: Status, action, strategy, and reward, among which, The state in the current step is represented as follows: The triple is composed of the first vector representation of the starting entity of the solved multi-hop path, the first vector representation of the entity reached by the current step of the multi-hop path, and the historical state corresponding to the current step. The action described in the current step is represented as: Execute the preset number of actions with the highest preference in the action space corresponding to the state of the current step; The strategy is expressed as follows: The probability distribution of executing the next action based on the current state; The reward is represented as follows: If the multi-hop path reaches the designated second entity, it receives a positive reward; otherwise, it receives no reward.

3. The method according to claim 2, characterized in that, The preset number of actions with the highest execution preference in the action space corresponding to the state of the current step are determined by the following method: The action space of the current step's state is determined based on the first knowledge graph; By using a pre-defined multi-head attention mechanism network, the attention score of each action is calculated based on the vector representation of the state of the current step and the vector representation of each action in the action space. Select a preset number of actions according to the attention scores from high to low, and execute the preset number of actions with the highest preference in the action space corresponding to the state of the current step.

4. The method according to claim 2, characterized in that, The probability distribution of executing the next action from the current state is determined by the following methods, including: The attention score vector, composed of the attention scores of all actions in the action space, is randomly masked using the binary mask vector of the action space. Based on the attention score vector after masking, calculate the probability distribution of executing the next action in the current step's state.

5. The method according to any one of claims 1 to 4, characterized in that, The second entity also includes: User attributes and / or recommender attributes.

6. An object recommendation device based on path reasoning, characterized in that, include: The first vector representation learning module is used to learn the first knowledge graph that expresses the relationship between preset entities through the first graph convolutional neural network to obtain the first vector representation of the entity; The entity includes a first entity and a second entity, wherein the first entity includes a user; The second entity includes the recommended object; The path reasoning module is used to determine several paths between the entities based on the first vector representation of the entities by executing a path reasoning model of reinforcement learning; The vector update module is used to construct a second knowledge graph based on the several paths in response to the execution result of the path reasoning module not meeting the preset iteration termination condition. The second knowledge graph is used to express the relationships between the entities included in the entity sequences corresponding to the plurality of paths; The second convolutional neural network is used to learn the structural features in the second knowledge graph to obtain the second vector representation of the entity; The first vector representation of the corresponding entity is updated by the second vector representation of the entity, and then the process jumps to execute the path reasoning module. A path filtering module is used to filter the path from the user to the recommended object from the plurality of paths in response to the execution result of the path reasoning module satisfying a preset iteration termination condition; The object recommendation module is used to recommend the recommended object to the user based on the path from the user to the recommended object obtained through filtering.

7. An electronic device, comprising a memory, a processor, and program code stored in the memory and executable on the processor, characterized in that, When the processor executes the program code, it implements the object recommendation method based on path reasoning as described in any one of claims 1 to 5.

8. A computer-readable storage medium having program code stored thereon, characterized in that, When the program code is executed by the processor, it implements the steps of the object recommendation method based on path reasoning as described in any one of claims 1 to 5.

Images