Knowledge graph reasoning method, device and storage medium

Abstract

The present disclosure relates to the field of data processing, and in particular to a knowledge graph reasoning method, device and storage medium. The method includes: obtaining initial knowledge graph data to be completed, the initial knowledge graph data includes multiple groups of initial data groups, and the initial data groups only include head entities and entity relationships; according to the initial knowledge graph data, calling the knowledge graph reasoning model to obtain the completed target knowledge graph data, the knowledge graph reasoning model is a model pre-trained based on reinforcement learning, and each group of target data groups in the target knowledge graph data includes head entities, entity relationships and tail entities. The embodiment of the present disclosure performs knowledge reasoning on the initial knowledge graph by calling the knowledge graph reasoning model obtained by reinforcement learning training. Compared with the current knowledge graph reasoning method, it has good scalability and faster running speed, improves the accuracy of the reasoning results, and ensures the effect of knowledge graph completion.

Description

Technical Field

[0001] The present disclosure relates to the field of data processing, and in particular to a knowledge graph reasoning method, device and storage medium. Background Art

[0002] The data in the knowledge graph is generally obtained from public information on the Internet through knowledge extraction and saved in the form of (head entity, entity relationship, tail entity) triples. In the process of constructing the knowledge graph, there are often a large number of triples that need to be completed. They cannot be directly extracted but can be obtained by certain reasoning methods.

[0003] The knowledge graph reasoning method is the process of obtaining the missing element of any two elements in a given triple through a certain method. In related technologies, the knowledge graph reasoning method is often based on relatively simple logical rules or statistical features, or on frequent patterns, constraints or paths at the ontology level for knowledge reasoning.

[0004] The current knowledge graph reasoning method has poor inference results and poor effect on knowledge graph completion. A reasonable and effective knowledge graph reasoning method has not yet been provided in the relevant technology. Summary of the invention

[0005] In view of this, the present disclosure proposes a knowledge graph reasoning method, device and storage medium. The technical solution includes:

[0006] According to one aspect of the present disclosure, a knowledge graph reasoning method is provided, characterized in that it is used in a computer device, and the method includes:

[0007] Acquire initial knowledge graph data to be completed, wherein the initial knowledge graph data includes multiple groups of initial data groups, and the initial data groups only include head entities and entity relationships;

[0008] According to the initial knowledge graph data, the knowledge graph reasoning model is called to obtain the completed target knowledge graph data, wherein the knowledge graph reasoning model is a model pre-trained based on reinforcement learning, and each target data group in the target knowledge graph data includes the head entity, the entity relationship and the tail entity.

[0009] In a possible implementation, the step of calling a knowledge graph reasoning model to obtain completed target knowledge graph data based on the initial knowledge graph data includes:

[0010] Obtaining the trained knowledge graph reasoning model, where the knowledge graph reasoning model is a model based on a partially observable Markov decision process;

[0011] For each of the initial data groups in the initial knowledge graph data, the initial data group is input into the knowledge graph reasoning model to obtain the completed target data group.

[0012] In another possible implementation, the policy network in the knowledge graph reasoning model includes a long short-term memory network LSTM and a feedforward network, and for each group of the initial data groups in the initial knowledge graph data, the initial data group is input into the knowledge graph reasoning model to obtain the completed target data group, including:

[0013] Obtaining an action parameter corresponding to a current node, wherein the action parameter is the initial data group associated with the current node;

[0014] According to the action parameters corresponding to the previous node and the observation parameters corresponding to the current node, the historical state parameters corresponding to the current node are obtained using LSTM, and the observation parameters are used to indicate the currently observed environment content;

[0015] According to the historical state parameters, the tail entity corresponding to the current node is obtained by using the feedforward network output;

[0016] The target data group corresponding to the current node is determined according to the tail entity corresponding to the current node.

[0017] In another possible implementation, the feedforward network is a two-layer feedforward network including an activation function and a classifier; and according to the historical state parameter, the tail entity corresponding to the current node is obtained by using the output of the feedforward network, including:

[0018] According to the historical state parameter, the observation parameter and the initial query relationship, the activation function is used to obtain a probability distribution estimate, wherein the probability distribution estimate is used to indicate probability values ​​corresponding to each of the plurality of candidate action parameters;

[0019] According to the probability distribution estimation, the target probability value corresponding to the target action parameter is obtained using the classifier;

[0020] The tail entity corresponding to the current node is determined according to the target probability value.

[0021] In another possible implementation, before the method of invoking the knowledge graph reasoning model to obtain the completed target knowledge graph data according to the initial knowledge graph data, further includes:

[0022] Acquire a training data set, the training data set comprising a plurality of sample data groups and respective corresponding standard tail entities, each sample data group comprising a sample head entity and a sample entity relationship, the standard tail entity being pre-calibrated;

[0023] According to the training data set, reinforcement learning training is performed to obtain the knowledge graph reasoning model.

[0024] In another possible implementation, after the knowledge graph inference model is called according to the initial knowledge graph data to obtain the completed target knowledge graph data, the method further includes:

[0025] Adding the target knowledge graph data to the training data set to obtain an updated training data set;

[0026] The knowledge graph reasoning model is trained according to the updated training data set to obtain an updated knowledge graph reasoning model.

[0027] In another possible implementation, the knowledge graph data is object relationship knowledge graph data, the head entity is a first object, the tail entity is a second object, the entity relationship is a relationship between the first object and the second object, and the second object is different from the first object.

[0028] According to another aspect of the present disclosure, a knowledge graph reasoning apparatus is provided, which is used in a computer device, and the apparatus includes:

[0029] An acquisition module, used to acquire initial knowledge graph data to be completed, wherein the initial knowledge graph data includes multiple groups of initial data groups, and the initial data groups only include head entities and entity relationships;

[0030] A calling module is used to call the knowledge graph reasoning model to obtain the completed target knowledge graph data based on the initial knowledge graph data. The knowledge graph reasoning model is a model pre-trained based on reinforcement learning. Each target data group in the target knowledge graph data includes the head entity, the entity relationship and the tail entity.

[0031] In a possible implementation, the calling module is further used to:

[0032] Obtaining the trained knowledge graph reasoning model, where the knowledge graph reasoning model is a model based on a partially observable Markov decision process;

[0033] For each of the initial data groups in the initial knowledge graph data, the initial data group is input into the knowledge graph reasoning model to obtain the completed target data group.

[0034] In another possible implementation, the policy network in the knowledge graph reasoning model includes a long short-term memory network LSTM and a feedforward network, and the calling module is further used to:

[0035] Obtaining an action parameter corresponding to a current node, wherein the action parameter is the initial data group associated with the current node;

[0036] According to the action parameters corresponding to the previous node and the observation parameters corresponding to the current node, the historical state parameters corresponding to the current node are obtained using LSTM, and the observation parameters are used to indicate the currently observed environment content;

[0037] According to the historical state parameters, the tail entity corresponding to the current node is obtained by using the feedforward network output;

[0038] The target data group corresponding to the current node is determined according to the tail entity corresponding to the current node.

[0039] In another possible implementation, the feedforward network is a two-layer feedforward network including an activation function and a classifier; and the calling module is further used to:

[0040] According to the historical state parameter, the observation parameter and the initial query relationship, the activation function is used to obtain a probability distribution estimate, wherein the probability distribution estimate is used to indicate probability values ​​corresponding to each of the plurality of candidate action parameters;

[0041] According to the probability distribution estimation, the target probability value corresponding to the target action parameter is obtained using the classifier;

[0042] The tail entity corresponding to the current node is determined according to the target probability value.

[0043] In another possible implementation, the device further includes: a training module; the training module is used to:

[0044] Acquire a training data set, the training data set comprising a plurality of sample data groups and respective corresponding standard tail entities, each sample data group comprising a sample head entity and a sample entity relationship, the standard tail entity being pre-calibrated;

[0045] According to the training data set, reinforcement learning training is performed to obtain the knowledge graph reasoning model.

[0046] In another possible implementation, the device further includes: an updating module; the updating module is configured to:

[0047] Adding the target knowledge graph data to the training data set to obtain an updated training data set;

[0048] The knowledge graph reasoning model is trained according to the updated training data set to obtain an updated knowledge graph reasoning model.

[0049] In another possible implementation, the knowledge graph data is object relationship knowledge graph data, the head entity is a first object, the tail entity is a second object, the entity relationship is a relationship between the first object and the second object, and the second object is different from the first object.

[0050] According to another aspect of the present disclosure, a computer device is provided, the computer device comprising: a processor; a memory for storing processor executable instructions;

[0051] Wherein, the processor is configured to:

[0052] A knowledge graph reasoning method, characterized in that it is used in a computer device, and the method comprises:

[0053] Acquire initial knowledge graph data to be completed, wherein the initial knowledge graph data includes multiple groups of initial data groups, and the initial data groups only include head entities and entity relationships;

[0054] According to the initial knowledge graph data, the knowledge graph reasoning model is called to obtain the completed target knowledge graph data, wherein the knowledge graph reasoning model is a model pre-trained based on reinforcement learning, and each target data group in the target knowledge graph data includes the head entity, the entity relationship and the tail entity.

[0055] According to another aspect of the present disclosure, a non-volatile computer-readable storage medium is provided, on which computer program instructions are stored, characterized in that the computer program instructions implement the above method when executed by a processor.

[0056] The disclosed embodiment obtains initial knowledge graph data to be completed through a computer device, the initial knowledge graph data includes multiple groups of initial data groups, and the initial data groups only include head entities and entity relationships; according to the initial knowledge graph data, the knowledge graph reasoning model is called to obtain the completed target knowledge graph data, and the knowledge graph reasoning model is a model pre-trained based on reinforcement learning, and each group of target data groups in the target knowledge graph data includes head entities, entity relationships and tail entities; that is, the knowledge graph reasoning model trained based on reinforcement learning is called to perform knowledge reasoning on the initial knowledge graph. Compared with the current knowledge graph reasoning method, it has good scalability and faster running speed, and can complete the reasoning task of large-scale sparse knowledge graphs with higher performance, thereby improving the accuracy of the reasoning results and ensuring the effect of knowledge graph completion. BRIEF DESCRIPTION OF THE DRAWINGS

[0057] The accompanying drawings, which are incorporated in and constitute a part of the specification, illustrate exemplary embodiments, features, and aspects of the disclosure and, together with the description, serve to explain the principles of the disclosure.

[0058] Figure 1 A schematic diagram showing the structure of a computer device provided by an exemplary embodiment of the present disclosure is shown;

[0059] Figure 2 A flowchart of a knowledge graph reasoning method provided by an exemplary embodiment of the present disclosure is shown;

[0060] Figure 3 A flowchart of a knowledge graph reasoning method provided by another exemplary embodiment of the present disclosure is shown;

[0061] Figure 4 A schematic diagram of the structure of a feedforward network involved in a knowledge graph reasoning method provided by an exemplary embodiment of the present disclosure is shown;

[0062] Figure 5 A schematic diagram showing the principles involved in a knowledge graph reasoning method provided by an exemplary embodiment of the present disclosure is shown;

[0063] Figure 6 A schematic diagram showing the principles involved in a knowledge graph reasoning method provided by an exemplary embodiment of the present disclosure is shown;

[0064] Figure 7 A schematic diagram of the structure of a knowledge graph reasoning device provided by an embodiment of the present disclosure is shown;

[0065] Figure 8 is a block diagram of a device for executing a knowledge graph reasoning method according to an exemplary embodiment;

[0066] Fig. 9 It is a block diagram of a device for executing a knowledge graph reasoning method according to another exemplary embodiment. DETAILED DESCRIPTION

[0067] Various exemplary embodiments, features and aspects of the present disclosure will be described in detail below with reference to the accompanying drawings. The same reference numerals in the accompanying drawings represent elements with the same or similar functions. Although various aspects of the embodiments are shown in the accompanying drawings, the drawings are not necessarily drawn to scale unless otherwise specified.

[0068] The word “exemplary” is used exclusively herein to mean “serving as an example, example, or illustration.” Any embodiment described herein as “exemplary” is not necessarily to be construed as preferred or advantageous over other embodiments.

[0069] In addition, in order to better illustrate the present disclosure, numerous specific details are given in the following specific embodiments. It should be understood by those skilled in the art that the present disclosure can also be implemented without certain specific details. In some examples, methods, means, components and circuits well known to those skilled in the art are not described in detail in order to highlight the subject matter of the present disclosure.

[0070] The knowledge graph reasoning method is a process of obtaining the missing element of any two elements in a given triple through a certain method. It is an important means to complete the knowledge graph and an important supplement to knowledge extraction.

[0071] In related technologies, knowledge graph reasoning methods are often based on relatively simple logical rules or statistical features, or on frequent patterns, constraints or paths at the ontology level for knowledge reasoning. They have the disadvantages of cumbersome operations, manual setting of reasoning rules, low performance, lack of universality, and inability to process large knowledge graphs. The current knowledge graph reasoning methods have poor inference results and poor results in completing knowledge graphs.

[0072] The disclosed embodiments provide a knowledge graph reasoning method, device and storage medium, which obtain initial knowledge graph data to be completed through a computer device, wherein the initial knowledge graph data includes multiple groups of initial data groups, and the initial data groups only include head entities and entity relationships; based on the initial knowledge graph data, a knowledge graph reasoning model is called to obtain the completed target knowledge graph data, and the knowledge graph reasoning model is a model pre-trained based on reinforcement learning, and each group of target data groups in the target knowledge graph data includes head entities, entity relationships and tail entities; that is, the knowledge graph reasoning model trained based on reinforcement learning is called to perform knowledge reasoning on the initial knowledge graph. Compared with the current knowledge graph reasoning method, it has good scalability and faster running speed, and can complete the reasoning task of large-scale sparse knowledge graphs with higher performance, thereby improving the accuracy of the reasoning results and ensuring the effect of knowledge graph completion.

[0073] First, some nouns involved in the embodiments of the present disclosure are introduced.

[0074] Knowledge graph: also known as scientific knowledge graph, in the library and information industry it is called knowledge domain visualization, or knowledge domain mapping map, a series of various graphics used to display the knowledge development process and structural relationship, using visualization technology to describe knowledge resources and carriers, to mine, analyze, construct, draw and display knowledge and their relationships.

[0075] The knowledge graph aims to describe various entities or concepts and their relationships in the real world, which constitute a huge semantic network graph. The knowledge graph is a directed graph composed of entities and relationships. The nodes in the graph represent entities, and the edges represent semantic relationships between entities. Relationships are used to describe the association between two entities, such as the relationship between Beijing and China; for the attributes of an entity, "attribute-value pairs" are used to characterize its intrinsic characteristics, such as a person has age, height, weight, etc. At present, knowledge graphs have been used to refer to various large-scale knowledge bases.

[0076] Entity (head entity or tail entity): refers to something that is distinguishable and exists independently. For example, a person, a city, a plant, a commodity, etc. Everything in the world is made up of specific things, which refers to entities, such as "China", "United States", "Japan", etc. Entities are the most basic elements in the knowledge graph, and different entities have different relationships.

[0077] Relationship (also called entity relationship): formalized as a function that maps multiple points to a Boolean value. In the knowledge graph, a relationship is a function that maps multiple graph nodes to Boolean values.

[0078] Based on the above definition, in order to facilitate computer processing and understanding, knowledge can be represented in a more formalized and concise way, that is, triples, which are a general representation of knowledge graphs. The basic form of triples mainly includes (entity 1-relationship-entity 2). Each entity can be identified by a globally unique ID, and the relationship can be used to connect two entities and characterize the association between them. For example, China is an entity, Beijing is an entity, and (China-capital-Beijing) is a triple example of (entity-relationship-entity). In the embodiment of the present application, the knowledge represented by triples is used to indicate the connection between two entities, wherein the connection between the two entities can be the relationship between the two entities (for example, (entity 1-relationship-entity 2)), or the connection between the two entities can be an attribute of one of the entities, and the other entity is the attribute value of the attribute (for example, (entity-attribute-attribute value)). The knowledge represented by triples in the embodiment of the present application can also be called structured knowledge. It should also be understood that the representation of triples is not limited to the above-mentioned (entity 1-relationship-entity 2) form, for example, it can also be expressed as (entity 1-entity 2-relationship) and the like.

[0079] Next, the application scenarios involved in the present disclosure are introduced.

[0080] Please refer to Figure 1 , which shows a schematic diagram of the structure of a computer device provided by an exemplary embodiment of the present disclosure.

[0081] The computer device 10 may be a terminal or a server. The terminal includes a tablet computer, a laptop computer, a desktop computer, etc. The server may be a server, or a server cluster consisting of several servers, or a cloud computing service center.

[0082] The computer device 10 is used to obtain the initial knowledge graph data 14 to be completed; based on the initial knowledge graph data 14, the knowledge graph reasoning model 12 is called to obtain the completed target knowledge graph data 16.

[0083] The knowledge graph reasoning model 12 is a model for performing knowledge reasoning on the initial knowledge graph data 14. The knowledge graph reasoning model 12 includes elements such as an agent, an environment, a strategy, an action, a state, and a reward value. For details, please refer to the description in the following embodiment, which will not be introduced here.

[0084] The knowledge graph reasoning model 12 is a model pre-trained based on reinforcement learning.

[0085] Optionally, the knowledge graph reasoning model 12 is a model based on a partially observable Markov decision process (POMDP), which is a generalized Markov decision process (MDP).

[0086] Optionally, the computer device 10 further includes: a storage device 18. The storage device 18 is a device with a data storage function. The storage device 18 is used to store data used in the knowledge graph reasoning process, such as relevant parameters of the knowledge graph reasoning model 12, etc. This embodiment of the disclosure is not limited to this.

[0087] Below, the knowledge graph reasoning method provided by the embodiment of the present disclosure is introduced using an illustrative embodiment.

[0088] Please refer to Figure 2 , which shows a flowchart of a knowledge graph reasoning method provided by an exemplary embodiment of the present disclosure. This embodiment applies the knowledge graph reasoning method to Figure 1 The computer device shown is used as an example. The knowledge graph reasoning method includes:

[0089] Step 201, obtaining the initial knowledge graph data to be completed, the initial knowledge graph data includes multiple groups of initial data groups, and the initial data groups only include head entities and entity relationships.

[0090] The computer device obtains the initial knowledge graph data to be completed, and the initial knowledge graph data includes multiple groups of initial data groups.

[0091] Optionally, the initial knowledge graph data includes multiple initial data groups and complete data groups other than the multiple initial data groups. Usually, each data group is in the form of a triple. The initial data group is also called a missing data group. The initial data group is a data group with a single missing element, that is, the initial data group only includes the head entity and entity relationship; the complete data group is a data group with three complete elements, that is, the complete data group includes the head entity, entity relationship and tail entity.

[0092] Optionally, header entities in at least two of the multiple groups of initial data groups are the same, or header entities in the multiple groups of initial data groups are all different.

[0093] Optionally, the entity relationships in at least two of the multiple initial data groups are the same, or the entity relationships in the multiple initial data groups are all different.

[0094] Optionally, the tail entities in at least two of the multiple initial data groups are the same, or the tail entities in the multiple initial data groups are all different. This embodiment of the present disclosure does not limit this.

[0095] Optionally, the knowledge graph data in the disclosed embodiments is object relationship knowledge graph data or object attribute knowledge graph data, the head entity is the first object, the tail entity is the second object, the entity relationship is the relationship between the first object and the second object, and the second object is different from the first object.

[0096] Schematically, taking the object as a person as an example, the knowledge graph data is the person relationship knowledge graph data. For example, one complete data group includes the head entity "Lao Zhang", the entity relationship "Who is the son" and the tail entity "Xiao Zhang", and an initial data group only includes the head entity "Xiao Li" and the entity relationship "Who is the father".

[0097] For example, taking a person as an object, the knowledge graph data is the person attribute knowledge graph data. For example, a complete data set includes the head entity "Xiao Ming", the entity relationship "what sport is played", and the tail entity "basketball", and an initial data set only includes the head entity "Xiao Zhao" and the entity relationship "what is the nationality". The embodiments of the present disclosure are not limited to this.

[0098] Step 202: Based on the initial knowledge graph data, call the knowledge graph reasoning model to obtain the completed target knowledge graph data. The knowledge graph reasoning model is a model pre-trained based on reinforcement learning. Each target data group in the target knowledge graph data includes a head entity, an entity relationship, and a tail entity.

[0099] The computer device calls the knowledge graph reasoning model based on the initial knowledge graph data to obtain the completed target knowledge graph data.

[0100] Optionally, the computer device obtains a pre-trained knowledge graph reasoning model, inputs the initial knowledge graph data into the knowledge graph reasoning model, and outputs the target knowledge graph data.

[0101] The knowledge graph reasoning model is a model pre-trained based on reinforcement learning. The knowledge graph reasoning model is a model obtained by reinforcement learning based on the training data set.

[0102] The knowledge graph reasoning model is a model obtained by training the original graph reasoning model using a training dataset.

[0103] The knowledge graph reasoning model is a reinforcement learning model that performs data reasoning on the initial knowledge graph data to be completed.

[0104] Optionally, the knowledge graph reasoning model is used to represent the correlation between the initial knowledge graph data to be completed and the target knowledge graph data after completion.

[0105] Optionally, the knowledge graph reasoning model is a model based on a partially observable Markov decision process.

[0106] Optionally, before the computer device calls the knowledge graph reasoning model to obtain the completed target knowledge graph data based on the initial knowledge graph data, it also includes: using the training data set to perform reinforcement learning training to obtain the target knowledge graph data.

[0107] The target knowledge graph data is the knowledge graph data completed after knowledge reasoning is performed on the initial knowledge graph data. That is, the target knowledge graph data includes multiple target data groups, and each of the multiple target data groups includes a head entity, an entity relationship, and a tail entity.

[0108] There is a one-to-one correspondence between the multiple target data groups in the target knowledge graph data and the multiple initial data groups in the initial knowledge graph data. The target data group corresponding to the initial data group is the data group after the initial data group is completed, that is, the head entity in the initial data group is the same as the head entity in the corresponding target data group, and the entity relationship in the initial data group is the same as the entity relationship in the corresponding target data group.

[0109] In summary, the disclosed embodiments provide a knowledge graph reasoning method, which obtains initial knowledge graph data to be completed through a computer device, wherein the initial knowledge graph data includes multiple data groups, and there is at least one data group in the multiple data groups that only includes a head entity and an entity relationship; based on the initial knowledge graph data, a knowledge graph reasoning model is called to obtain the completed target knowledge graph data, and the knowledge graph reasoning model is a model pre-trained based on reinforcement learning, and each data group in the target knowledge graph data includes a head entity, an entity relationship, and a tail entity; that is, with the help of a reasoning method based on graph features, a computer device calls a knowledge graph reasoning model trained based on reinforcement learning to perform knowledge reasoning on the initial knowledge graph. Compared with the current knowledge graph reasoning method, it has good scalability and a faster running speed, and can complete the reasoning task of large-scale sparse knowledge graphs with higher performance, thereby improving the accuracy of the reasoning results and ensuring the effect of knowledge graph completion.

[0110] It should be noted that the principles of reinforcement learning include: the agent executes the determined actions through the policy network, and each action taken by the agent will cause changes in the environment. At the same time, the environment transmits the changes in the environment after the agent takes action to the agent through state variables for the agent to use in the next decision. At the same time, the environment also feeds back the reward for the action to the agent. The goal of reinforcement learning is to allow the agent to iteratively update its own strategy through the collected rewards and state variables during the above-mentioned interaction with the environment, so that the agent can get the maximum reward in the next interaction with the environment.

[0111] In order to realize the training of the knowledge graph reasoning model by adopting reinforcement learning, it is necessary to map the knowledge graph reasoning scheme to the appropriate operation in reinforcement learning when the technology is implemented. The reinforcement learning system in the scheme provided by the embodiment of the present disclosure consists of two parts, namely the environment and the agent. The environment can be understood as the entire knowledge graph data and is modeled as a POMDP. The strategy implementation process of the agent in the POMDP process determines that the system state is set by the Markov decision process, but in the strategy implementation process, the agent cannot directly observe the overall state, but must determine the state distribution through the prior cognition of the model and the local observation in the actual process. Among them, the environment of reinforcement learning includes five important components, namely state, action, conditional transfer function, reward function and observation. Optionally, POMDP is formally a quintuple (S, A, T, R, Ω), where S is a set of states, A is a set of actions, T is a conditional transfer function between states, R is a reward function, and Ω is a set of observations. It should be noted that, in the embodiments of the present disclosure, the state is also referred to as the state parameter, the action is also referred to as the action parameter, and the observation is also referred to as the observation parameter, which is not limited in the embodiments of the present disclosure.

[0112] The knowledge graph reasoning method provided by the embodiment of the present disclosure introduces a question-answer search graph through a reinforcement learning method, so that the agent learns a strategy: for a given knowledge graph completion task, the agent will start from the head node and learn to move to the tail node by selecting labeled relationship edges. The decision conditions are the query, relationship, and historical path. The decision goal is to take the best relationship edge to maximize the expected reward, so as to reach the correct tail node. The meanings of the various elements in reinforcement learning in the scheme provided by the embodiment of the present disclosure are as follows:

[0113] State S: The element s in the state S set consists of a four-tuple, where the four elements are the current node, the starting node, the initial query relation, and the tail node.

[0114] For example, s=(h t ,h q , r, t e), h t is the current node, h q is the starting node, r is the initial query relationship, t e is the tail node, where h t ,h q , r∈E,t e ∈R. The above definition ensures that the elements in the state S contain all feasible combinations of the entity set E and the relationship set R.

[0115] Action A: It is the triple associated with the current node, used to indicate going to the next node.

[0116] The actions that can be executed in state S are determined by the current node h t From the perspective of the graph, the action that can be taken when staying at a node is to go to any node associated with the node, so the action set includes the current node h t All associated nodes and a stay action.

[0117] For example, action A = (h t , r t ,h tn ), where r t Indicates h t The relationship between any associated node, h tn is the associated node of the current node. Since each node in the knowledge graph has a considerable degree, that is, the action set A is often a very large set, whether the agent can take the required action a by making decisions on historical state parameters is a very important issue and will have a key impact on the accuracy and efficiency of knowledge reasoning, which is also the core of reinforcement learning.

[0118] The action of staying is very important, because in reasoning, a length L needs to be determined for the obtained reasoning path to prevent the generation of too long reasoning paths, because too long reasoning paths are not direct and often have a very low probability of being established, but at the same time they will consume a lot of computing resources, resulting in inefficiency of the algorithm. The actual reasoning paths obtained are often of different lengths. For example, for reasoning about nationality, the agent may be able to obtain two paths, "place of birth-located in the country" and "graduate school-located in the city-located in the country". The lengths of these two paths are inconsistent. After introducing the action of staying, the reasoning paths shorter than the length L can be uniformly expanded to the path of length L, that is, "place of birth-located in the country" can be changed to "place of birth-located in the country-stay" (for example, define L=3), so that subsequent processing can make it more unified.

[0119] State transition probability function T: When the agent takes an action a, the state of the environment changes from s to s′. Based on the above-defined state S and action A, the state transition function T is defined as: T(S, A) = (h tn ,h q , r, t e ), that is, the current node h t After action a, it changes to the new current node h tn , the state transition probability P is P(S′=s′|S=s, A=a), where S′ represents the current state, s′ represents the previous state, and A represents the action taken.

[0120] Reward function R: Also called the activation function. The setting of the reward function R can be different depending on the data set.

[0121] In one possible implementation, the reward function is used to indicate if the agent's current node and the tail node t e The same means that the agent has successfully completed the correct knowledge reasoning, then the value of the reward function is 1, that is, a reward of +1 is given; if the agent's current node is the same as the tail node t e If the reward function is different, the value of the reward function is 0, and a reward of 0 is given. Schematically, the reward function is defined as:

[0122]

[0123] Among them, h t , t e ∈s.

[0124] In another possible implementation, the reward function R includes two reward functions, which are a reward function for path length and a reward function for path diversity. The embodiment of the present disclosure does not limit the setting method of the reward function R.

[0125] Observation Ω: It is used to indicate the currently observed environment content. For a given state s, the only information that the agent can access is the current node (h t , starting node h q As well as the initial query relation r, the agent cannot observe the tail node, which is the answer node t e Therefore, the above state s cannot be fully observed, and the tail node is unknown, so the definition of observation Ω is given as: Ω(s=(h t ,h q , r, h e ))=(h t ,h q , r).

[0126] Agent: Also known as a tool, it is mainly composed of a knowledge graph reasoning model. The knowledge graph reasoning model can be implemented by a deep neural network. The input parameter of the knowledge graph reasoning model is the initial knowledge graph data to be completed, and the output parameter is the completed target knowledge graph data. The knowledge graph reasoning model is used to provide the completed target knowledge graph data according to the initial knowledge graph data. The training process of the model is actually the process of adjusting the action parameters. For example, by updating the policy network (that is, adjusting the parameters of the model) to obtain a higher reward value (that is, higher performance).

[0127] In the disclosed embodiment, the agent and the environment interact with each other, and the agent takes the best action through decision making. The environment changes and changes its state because of this action, and the environment will feed back information about the agent's reward function. After receiving the feedback information from the environment, the agent's next decision will refer to this information and all previous historical state parameters to make the next best decision. The agent is a policy network, which gives the best decision for the next step by analyzing the feedback information from the environment.

[0128] Based on the meaning of the various elements in the reinforcement learning introduced above in the solution provided in the embodiment of this disclosure, please refer to Figure 3 , which shows a flow chart of a knowledge graph reasoning method provided by another exemplary embodiment of the present disclosure. This embodiment applies the knowledge graph reasoning method to Figure 1 The computer device shown is used as an example. The knowledge graph reasoning method includes:

[0129] Step 301, obtaining the initial knowledge graph data to be completed, the initial knowledge graph data includes multiple groups of initial data groups, and the initial data groups only include head entities and entity relationships.

[0130] In the process of using the knowledge graph reasoning model, the computer device obtains the initial knowledge graph data to be completed. For relevant details, please refer to the relevant content in the above embodiment, which will not be repeated here.

[0131] Step 302: Obtain a trained knowledge graph reasoning model, where the knowledge graph reasoning model is a model based on a partially observable Markov decision process.

[0132] Before the computer device obtains the trained knowledge graph reasoning model, the knowledge graph reasoning model is trained. Optionally, the computer device obtains a training data set, the training data set includes multiple groups of sample data groups and their corresponding standard tail entities, each group of sample data groups includes a sample head entity and a sample entity relationship, and the standard tail entity is pre-calibrated; according to the training data set, reinforcement learning training is performed to obtain the knowledge graph reasoning model.

[0133] Optionally, the computer device trains the knowledge graph inference model based on the training data set using a first expected parameter and a second expected parameter; wherein the first expected parameter is the empirical average of the training data set, and the second expected parameter is an estimated value obtained after multiple training of each training sample.

[0134] Indicatively, for the above random non-stationary history-dependent strategy, in order to maximize the reward function, the model parameter is set to θ, and the following training function is used for training:

[0135] [R(S T )|S 1 =(h t ,h q , r, t e )]

[0136] Among them, J(θ) is the training function, E (h,r,t)～D is the first expected parameter, is the second expected parameter, R(S T ) is the current state S T The reward function value, S 1 The complete set of states.

[0137] The computer device obtains the knowledge graph reasoning model trained as above.

[0138] Step 303: For each initial data group in the initial knowledge graph data, the initial data group is input into the knowledge graph reasoning model to obtain a completed target data group.

[0139] Optionally, the policy network in the knowledge graph reasoning model includes a long short-term memory network (Long Short-Term Memory, LSTM) and a feedforward network, and the computer device obtains the action parameters corresponding to the current node, and the action parameters are the initial data group associated with the current node; according to the action parameters corresponding to the previous node and the observation parameters corresponding to the current node, the historical state parameters corresponding to the current node are obtained using LSTM, and the observation parameters are used to indicate the currently observed environmental content; according to the historical state parameters, the feedforward network output is used to obtain the tail entity corresponding to the current node; according to the tail entity corresponding to the current node, the target data group corresponding to the current node is determined.

[0140] Instructively, the probability distribution of all actions in the set of possible actions A is given, and the action with the largest probability value is selected. In order to solve the above POMDP problem, we design a random non-stationary history-dependent strategy π=(d 1 , d 2 , ...d T-1 ), d tis the probability value, which is determined by the historical state parameter H t Determined by H t =(H t-1 , A t-1 ,Ω t ), where H t-1 is the historical state parameter of the previous step, A t-1 is the action of the previous step, Ω t For the current observation, LSTM is used to encode the historical behavior. That is, the input parameters of LSTM are the action parameters of the previous step and the current observation parameters, and the output parameters are the historical state parameters.

[0141] The agent encodes historical behavior H through LSTM t =(H t-1 , A t-1 ,Ω t ), and obtain the vector representation h of the historical state parameters t ∈R, and according to the relationship set R and entity set E obtained above, the historical state parameter H can be updated t .

[0142] h t =LSTM(h t-1 ,[a t-1 ;ω t ])

[0143] Among them, a t-1 ∈R represents the vector representation of the action in the previous step, that is, the embedded representation of the corresponding edge selected by the agent in the previous step. t ∈R represents the vector representation of the observation at this step, that is, the embedding representation of the agent’s current node, all associated edges, and the initial node.

[0144] The input parameter of the feedforward network is the historical state parameter, and the output parameter is the target probability value corresponding to the target action parameter. Optionally, the feedforward network is a two-layer feedforward network including an activation function.

[0145] Optionally, the feedforward network is a two-layer feedforward network including an activation function and a classifier; the computer device uses the feedforward network output to obtain the tail entity corresponding to the current node based on the historical state parameters, including: the computer device uses the activation function to obtain a probability distribution estimate based on the historical state parameters, observation parameters and initial query relationship, and the probability distribution estimate is used to indicate the probability values ​​corresponding to each of multiple candidate action parameters; the computer device uses the classifier to obtain a target probability value corresponding to the target action parameter based on the probability distribution estimate; and determines the tail entity corresponding to the current node based on the target probability value.

[0146] Schematically, the vector representation h based on the historical state parameters t, the policy network will select an action from multiple candidate actions in the action set A as the decision for this step. In one possible implementation, multiple candidate actions are outgoing edges of the current node with edge relationship labels l and target entity d information. Therefore, for each action A, it can be expressed as follows A st =[r l :e d ], these vectors can form A t Matrix, where r l For the connection relationship, e d is the target entity. The policy network takes these as input and is parameterized as a two-layer feed-forward network with an activation function.

[0147] The label l and target entity d are manually annotated before training. The meaning of outgoing edge: the next node connected to the corresponding current node in the knowledge graph. The relationship between the two nodes is represented by a line in the graph, which is the outgoing edge.

[0148] Activation function: The input parameters are historical state parameters, observation parameters and initial query relationship, and the output parameter is probability distribution estimation. The probability distribution estimation is the probability distribution estimation of executable actions, which is used to indicate the probability values ​​corresponding to multiple candidate action parameters, that is, dt:H t →P(Ast).

[0149] Optionally, the activation function is a Rectified Linear Unit (ReLU) function.

[0150] Schematically, the input parameter of the ReLU function is the vector representation of the historical state parameter h t , observation parameter ω t and the initial query relation r, the output parameter is the probability distribution estimate.

[0151] d t =softmax(A t (W 2 ReLU(W 1 [h t ;ω t ; r])))

[0152] Among them, W 1 and W 2 The weight matrix to be determined for training.

[0153] In one possible implementation, Figure 4 As shown, the feedforward network is a two-layer feedforward network 41 including an activation function, for example, the activation function is a ReLU function, and the classifier is a SoftMax classifier. t, the computer device represents the vector of the current historical state parameters h t , the currently observed environment content is the observation parameter ω t , and the initial query relation r is used as the input parameter of the ReLU function, and the output parameter of the ReLU function is used as the input parameter of the next level classifier. The classifier is used to select the target probability value from the probability values ​​corresponding to multiple candidate action parameters, that is, the target probability value d is obtained by the classifier output t , as a strategy for this step.

[0154] Optionally, after the computer device calls the knowledge graph reasoning model to obtain the completed target knowledge graph data based on the initial knowledge graph data, the method also includes: the computer device adds the target knowledge graph data to the training data set to obtain an updated training data set; and trains the knowledge graph reasoning model based on the updated training data set to obtain an updated knowledge graph reasoning model.

[0155] It should be noted that the relevant details can be referred to the relevant description of the model training process in the above embodiment, and will not be repeated here.

[0156] Optionally, the knowledge graph data is object relationship knowledge graph data, the head entity is the first object, the tail entity is the second object, the entity relationship is the relationship between the first object and the second object, and the second object is different from the first object. For example, the objects are characters, that is, the knowledge graph data is character relationship knowledge graph data, the head entity is the first character, the tail entity is the second character, the entity relationship is the relationship between the first character and the second character, and the second character is different from the first character. This is not limited in the embodiments of the present disclosure.

[0157] The agent will give different action space probability distributions for different reasoning tasks, so that the agent can act in the most correct direction. In an illustrative example, Figure 5 As shown, the computer device obtains the initial knowledge graph data to be completed, wherein a set of initial data groups only includes the head entity "Cole" and the entity relationship "sports played", and the intelligent agent uses the activation function to obtain the probability distribution estimate according to the historical state parameters, observation parameters and initial query relationship of the current node, and the probability distribution estimate includes the probability value p=0.977 of the next action to the "NBA" node, and the probability value p=0.01 of the next action to the "Golden State Warriors" node, so that the classifier is used to determine the target probability value "0.977" based on the probability distribution estimate, and the associated sport is inferred from the "NBA" node to be "basketball", that is, the tail entity corresponding to the current node is "basketball".

[0158] In another illustrative example, Figure 6As shown, the computer device obtains the initial knowledge graph data to be completed, wherein an initial data group includes only the head entity "Cole" and the entity relationship "who to work for", and the intelligent agent uses the activation function to obtain the probability distribution estimate according to the historical state parameters, observation parameters and initial query relationship of the current node, and the probability distribution estimate includes the probability value p=0.012 of the next action to the "NBA" node, and the probability value p=0.893 of the next action to the "Golden State Warriors" node, so that the classifier is used to determine the target probability value "0.893" according to the probability distribution estimate, so as to determine that the tail entity corresponding to the current node is "Golden State Warriors", that is, the target data group after the completion of the initial data group is obtained, including the head entity "Cole", the entity relationship "who to work for" and the tail entity "Golden State Warriors".

[0159] In summary, the knowledge graph reasoning method provided by the embodiments of the present disclosure, on the one hand, constructs a reinforcement learning environment, establishes a model based on a partially observable Markov decision process, and the intelligent agent infers the distribution of states based on the observation results of the entire domain and partial regions of the model; on the other hand, it provides a non-stationary history-dependent strategy to achieve reinforcement learning modeling, and uses historical states and decisions to be encoded through LSTM and provided to the intelligent agent for decision-making; on the other hand, it uses a two-layer feedforward network including an activation function to perform knowledge reasoning, which improves the performance of computer equipment in performing knowledge graph reasoning, further improves the accuracy of the reasoning results, and ensures the effect of completing the knowledge graph.

[0160] The following is an apparatus embodiment of the present disclosure. For parts not described in detail in the apparatus embodiment, reference may be made to the technical details disclosed in the above method embodiment.

[0161] Please refer to Figure 7 , which shows a schematic diagram of the structure of a knowledge graph reasoning device provided by an embodiment of the present disclosure. The knowledge graph reasoning device can be implemented by a dedicated hardware circuit, or a combination of software and hardware. Figure 1 The device comprises: an acquisition module 710 and a calling module 720.

[0162] An acquisition module 710 is used to acquire initial knowledge graph data to be completed, where the initial knowledge graph data includes multiple groups of initial data groups, where the initial data groups only include head entities and entity relationships;

[0163] The calling module 720 is used to call the knowledge graph reasoning model to obtain the completed target knowledge graph data based on the initial knowledge graph data. The knowledge graph reasoning model is a model pre-trained based on reinforcement learning. Each target data group in the target knowledge graph data includes a head entity, an entity relationship and a tail entity.

[0164] In a possible implementation, the calling module 720 is further configured to:

[0165] Obtain the trained knowledge graph reasoning model, which is a model based on a partially observable Markov decision process;

[0166] For each initial data group in the initial knowledge graph data, the initial data group is input into the knowledge graph reasoning model to obtain the completed target data group.

[0167] In another possible implementation, the policy network in the knowledge graph reasoning model includes a long short-term memory network LSTM and a feedforward network, and the calling module 720 is further used to:

[0168] Get the action parameters corresponding to the current node. The action parameters are the initial data group associated with the current node.

[0169] According to the action parameters corresponding to the previous node and the observation parameters corresponding to the current node, the historical state parameters corresponding to the current node are obtained using LSTM. The observation parameters are used to indicate the currently observed environment content.

[0170] According to the historical state parameters, the tail entity corresponding to the current node is obtained by using the feedforward network output;

[0171] According to the tail entity corresponding to the current node, the target data group corresponding to the current node is determined.

[0172] In another possible implementation, the feedforward network is a two-layer feedforward network including an activation function and a classifier; the calling module 720 is further used to:

[0173] According to the historical state parameters, the observed parameters and the initial query relationship, an activation function is used to obtain a probability distribution estimate, which is used to indicate the probability values ​​corresponding to each of the multiple candidate action parameters;

[0174] According to the probability distribution estimation, the classifier is used to obtain the target probability value corresponding to the target action parameter;

[0175] Determine the tail entity corresponding to the current node according to the target probability value.

[0176] In another possible implementation, the device further includes: a training module; the training module is used to:

[0177] Obtain a training data set, the training data set includes multiple groups of sample data groups and their corresponding standard tail entities, each group of sample data groups includes a sample head entity and a sample entity relationship, and the standard tail entity is pre-calibrated;

[0178] According to the training data set, reinforcement learning training is performed to obtain the knowledge graph reasoning model.

[0179] In another possible implementation, the device further includes: an updating module; the updating module is configured to:

[0180] Add the target knowledge graph data to the training dataset to obtain an updated training dataset;

[0181] The knowledge graph reasoning model is trained according to the updated training data set to obtain an updated knowledge graph reasoning model.

[0182] In another possible implementation, the knowledge graph data is object relationship knowledge graph data, the head entity is the first object, the tail entity is the second object, the entity relationship is the relationship between the first object and the second object, and the second object is different from the first object.

[0183] It should be noted that the device provided in the above embodiment only uses the division of the above-mentioned functional modules as an example to implement its functions. In practical applications, the above-mentioned functions can be assigned to different functional modules according to actual needs, that is, the content structure of the device can be divided into different functional modules to complete all or part of the functions described above.

[0184] Regarding the device in the above embodiment, the specific manner in which each module performs operations has been described in detail in the embodiment of the method, and will not be elaborated here.

[0185] The embodiment of the present disclosure further provides a computer device, which includes: a processor; a memory for storing instructions executable by the processor; wherein the processor is configured to: implement the steps executed by the computer device in the above-mentioned various method embodiments.

[0186] Among them, computer equipment includes terminal equipment or server.

[0187] The embodiments of the present disclosure further provide a non-volatile computer-readable storage medium on which computer program instructions are stored. When the computer program instructions are executed by a processor, the methods in the above-mentioned various method embodiments are implemented.

[0188] Figure 8 800 is a block diagram of a device 800 for executing a knowledge graph reasoning method according to an exemplary embodiment. For example, the device 800 may be a terminal device. Figure 8 , the device 800 may include one or more of the following components: a processing component 802 , a memory 804 , a power component 806 , a multimedia component 808 , an audio component 810 , an input / output (I / O) interface 812 , a sensor component 814 , and a communication component 816 .

[0189] The processing component 802 generally controls the overall operation of the device 800, such as operations associated with display, phone calls, data communications, camera operations, and recording operations. The processing component 802 may include one or more processors 820 to execute instructions to complete all or part of the steps of the above-mentioned method. In addition, the processing component 802 may include one or more modules to facilitate the interaction between the processing component 802 and other components. For example, the processing component 802 may include a multimedia module to facilitate the interaction between the multimedia component 808 and the processing component 802.

[0190] The memory 804 is configured to store various types of data to support operations on the device 800. Examples of such data include instructions for any application or method operating on the device 800, contact data, phone book data, messages, pictures, videos, etc. The memory 804 can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic disk or optical disk.

[0191] The power supply component 806 provides power to the various components of the device 800. The power supply component 806 may include a power management system, one or more power supplies, and other components associated with generating, managing, and distributing power for the device 800.

[0192] The multimedia component 808 includes a screen that provides an output interface between the device 800 and the user. In some embodiments, the screen may include a liquid crystal display (LCD) and a touch panel (TP). If the screen includes a touch panel, the screen may be implemented as a touch screen to receive input signals from the user. The touch panel includes one or more touch sensors to sense touch, slide, and gestures on the touch panel. The touch sensor may not only sense the boundaries of the touch or slide action, but also detect the duration and pressure associated with the touch or slide operation. In some embodiments, the multimedia component 808 includes a front camera and / or a rear camera. When the device 800 is in an operating mode, such as a shooting mode or a video mode, the front camera and / or the rear camera may receive external multimedia data. Each front camera and rear camera may be a fixed optical lens system or have a focal length and optical zoom capability.

[0193] The audio component 810 is configured to output and / or input audio signals. For example, the audio component 810 includes a microphone (MIC), and when the device 800 is in an operating mode, such as a call mode, a recording mode, and a speech recognition mode, the microphone is configured to receive an external audio signal. The received audio signal can be further stored in the memory 804 or sent via the communication component 816. In some embodiments, the audio component 810 also includes a speaker for outputting audio signals.

[0194] I / O interface 812 provides an interface between processing component 802 and peripheral interface modules, such as keyboards, click wheels, buttons, etc. These buttons may include but are not limited to: home button, volume button, start button, and lock button.

[0195] The sensor assembly 814 includes one or more sensors for providing various aspects of the status assessment of the device 800. For example, the sensor assembly 814 can detect the open / closed state of the device 800, the relative positioning of components, such as the display and keypad of the device 800, and the sensor assembly 814 can also detect the position change of the device 800 or a component of the device 800, the presence or absence of user contact with the device 800, the orientation or acceleration / deceleration of the device 800, and the temperature change of the device 800. The sensor assembly 814 may include a proximity sensor configured to detect the presence of a nearby object without any physical contact. The sensor assembly 814 may also include an optical sensor, such as a CMOS or CCD image sensor, for use in imaging applications. In some embodiments, the sensor assembly 814 may also include an accelerometer, a gyroscope sensor, a magnetic sensor, a pressure sensor, or a temperature sensor.

[0196] The communication component 816 is configured to facilitate wired or wireless communication between the device 800 and other devices. The device 800 can access a wireless network based on a communication standard, such as WiFi, 2G or 3G, or a combination thereof. In an exemplary embodiment, the communication component 816 receives a broadcast signal or broadcast-related information from an external broadcast management system via a broadcast channel. In an exemplary embodiment, the communication component 816 also includes a near field communication (NFC) module to facilitate short-range communication. For example, the NFC module can be implemented based on radio frequency identification (RFID) technology, infrared data association (IrDA) technology, ultra-wideband (UWB) technology, Bluetooth (BT) technology and other technologies.

[0197] In an exemplary embodiment, the apparatus 800 may be implemented by one or more application specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable gate arrays (FPGAs), controllers, microcontrollers, microprocessors or other electronic components to perform the above method.

[0198] In an exemplary embodiment, a non-volatile computer-readable storage medium is also provided, such as a memory 804 including computer program instructions, which can be executed by the processor 820 of the device 800 to perform the above method.

[0199] Fig. 9 is a block diagram of a device 900 for executing a knowledge graph reasoning method according to another exemplary embodiment. For example, the device 900 may be provided as a server. Fig. 9 , the apparatus 900 includes a processing component 922, which further includes one or more processors, and a memory resource represented by a memory 932 for storing instructions, such as an application, that can be executed by the processing component 922. The application stored in the memory 932 may include one or more modules, each corresponding to a set of instructions. In addition, the processing component 922 is configured to execute the instructions to perform the above method.

[0200] The device 900 may also include a power supply component 926 configured to perform power management of the device 900, a wired or wireless network interface 950 configured to connect the device 900 to a network, and an input / output (I / O) interface 958. The device 900 may operate based on an operating system stored in the memory 932, such as Windows Server™, Mac OS X™, Unix™, Linux™, FreeBSD™, or the like.

[0201] In an exemplary embodiment, a non-volatile computer-readable storage medium is also provided, such as a memory 932 including computer program instructions that can be executed by the processing component 922 of the device 900 to perform the above method.

[0202] The present disclosure may be a system, a method and / or a computer program product. The computer program product may include a computer-readable storage medium carrying computer-readable program instructions for causing a processor to implement various aspects of the present disclosure.

[0203] A computer-readable storage medium may be a tangible device that can hold and store instructions used by an instruction execution device. A computer-readable storage medium may be, for example, but not limited to, an electrical storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, or any suitable combination of the foregoing. More specific examples of computer-readable storage media (a non-exhaustive list) include: a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), a static random access memory (SRAM), a portable compact disk read-only memory (CD-ROM), a digital versatile disk (DVD), a memory stick, a floppy disk, a mechanical encoding device, such as a punch card or a raised structure in a groove on which instructions are stored, and any suitable combination of the foregoing. As used herein, a computer-readable storage medium is not to be interpreted as a transient signal per se, such as a radio wave or other freely propagating electromagnetic wave, an electromagnetic wave propagating through a waveguide or other transmission medium (e.g., a light pulse through a fiber optic cable), or an electrical signal transmitted through a wire.

[0204] The computer-readable program instructions described herein can be downloaded from a computer-readable storage medium to each computing / processing device, or downloaded to an external computer or external storage device via a network, such as the Internet, a local area network, a wide area network, and / or a wireless network. The network can include copper transmission cables, optical fiber transmissions, wireless transmissions, routers, firewalls, switches, gateway computers, and / or edge servers. The network adapter card or network interface in each computing / processing device receives the computer-readable program instructions from the network and forwards the computer-readable program instructions for storage in the computer-readable storage medium in each computing / processing device.

[0205] The computer program instructions for performing the operation of the present disclosure may be assembly instructions, instruction set architecture (ISA) instructions, machine instructions, machine-related instructions, microcode, firmware instructions, state setting data, or source code or object code written in any combination of one or more programming languages, including object-oriented programming languages, such as Smalltalk, C++, etc., and conventional procedural programming languages, such as "C" language or similar programming languages. Computer-readable program instructions may be executed completely on a user's computer, partially on a user's computer, as an independent software package, partially on a user's computer, partially on a remote computer, or completely on a remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer via any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., using an Internet service provider to connect via the Internet). In some embodiments, an electronic circuit, such as a programmable logic circuit, a field programmable gate array (FPGA), or a programmable logic array (PLA), may be customized by utilizing the state information of the computer-readable program instructions, and the electronic circuit may execute the computer-readable program instructions, thereby realizing various aspects of the present disclosure.

[0206] Various aspects of the present disclosure are described herein with reference to the flowcharts and / or block diagrams of the methods, devices (systems) and computer program products according to the embodiments of the present disclosure. It should be understood that each box in the flowchart and / or block diagram and the combination of each box in the flowchart and / or block diagram can be implemented by computer-readable program instructions.

[0207] These computer-readable program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, thereby producing a machine, so that when these instructions are executed by the processor of the computer or other programmable data processing device, a device that implements the functions / actions specified in one or more boxes in the flowchart and / or block diagram is generated. These computer-readable program instructions can also be stored in a computer-readable storage medium, and these instructions cause the computer, programmable data processing device, and / or other equipment to work in a specific manner, so that the computer-readable medium storing the instructions includes a manufactured product, which includes instructions for implementing various aspects of the functions / actions specified in one or more boxes in the flowchart and / or block diagram.

[0208] Computer-readable program instructions may also be loaded onto a computer, other programmable data processing apparatus, or other device so that a series of operating steps are performed on the computer, other programmable data processing apparatus, or other device to produce a computer-implemented process, thereby causing the instructions executed on the computer, other programmable data processing apparatus, or other device to implement the functions / actions specified in one or more boxes in the flowchart and / or block diagram.

[0209] The flow chart and block diagram in the accompanying drawings show the possible architecture, function and operation of the system, method and computer program product according to multiple embodiments of the present disclosure. In this regard, each square box in the flow chart or block diagram can represent a part of a module, program segment or instruction, and a part of the module, program segment or instruction includes one or more executable instructions for realizing the specified logical function. In some alternative implementations, the function marked in the square box can also occur in a sequence different from that marked in the accompanying drawings. For example, two continuous square boxes can actually be executed substantially in parallel, and they can sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each square box in the block diagram and / or flow chart, and the combination of the square boxes in the block diagram and / or flow chart can be implemented with a dedicated hardware-based system that performs the specified function or action, or can be implemented with a combination of special hardware and computer instructions.

[0210] The embodiments of the present disclosure have been described above, and the above description is exemplary, not exhaustive, and is not limited to the disclosed embodiments. Many modifications and variations will be apparent to those of ordinary skill in the art without departing from the scope and spirit of the described embodiments. The selection of terms used herein is intended to best explain the principles of the embodiments, practical applications, or technical improvements to the technology in the market, or to enable other persons of ordinary skill in the art to understand the embodiments disclosed herein.

Claims

1. A knowledge graph reasoning method, characterized in that: For use in a computer device, the method comprises: Acquire initial knowledge graph data to be completed, wherein the initial knowledge graph data includes multiple groups of initial data groups, and the initial data groups only include head entities and entity relationships; According to the initial knowledge graph data, a knowledge graph reasoning model is called to obtain the completed target knowledge graph data, wherein the knowledge graph reasoning model is a model pre-trained based on reinforcement learning, each target data group in the target knowledge graph data includes the head entity, the entity relationship and the tail entity, and the knowledge graph reasoning model is applied to the question-answer search graph, the target knowledge graph data is object relationship knowledge graph data or object attribute knowledge graph data, the head entity is a first object, the tail entity is a second object, the entity relationship is a relationship between the first object and the second object, and the second object is different from the first object; Among them, the policy network in the knowledge graph reasoning model includes a long short-term memory network LSTM and a feedforward network. The input parameters of the LSTM are the action parameters of the previous step and the current observation parameters, and the output parameters are historical state parameters. The observation parameters are used to indicate the currently observed environmental content; the input parameters of the feedforward network are the historical state parameters, and the output parameters are the target probability values ​​corresponding to the target action parameters.

2. The method according to claim 1, characterized in that The step of calling the knowledge graph inference model to obtain the completed target knowledge graph data according to the initial knowledge graph data includes: Obtaining the trained knowledge graph reasoning model, where the knowledge graph reasoning model is a model based on a partially observable Markov decision process; For each of the initial data groups in the initial knowledge graph data, the initial data group is input into the knowledge graph reasoning model to obtain the completed target data group.

3. The method according to claim 2, characterized in that For each of the initial data groups in the initial knowledge graph data, inputting the initial data group into the knowledge graph reasoning model to obtain the completed target data group includes: Obtaining an action parameter corresponding to a current node, wherein the action parameter is the initial data group associated with the current node; According to the action parameters corresponding to the previous node and the observation parameters corresponding to the current node, the historical state parameters corresponding to the current node are obtained using LSTM; According to the historical state parameters, the tail entity corresponding to the current node is obtained by using the feedforward network output; The target data group corresponding to the current node is determined according to the tail entity corresponding to the current node.

4. The method according to claim 3, characterized in that The feedforward network is a two-layer feedforward network including an activation function and a classifier; According to the historical state parameters, the tail entity corresponding to the current node is obtained by using the feedforward network output, including: According to the historical state parameter, the observation parameter and the initial query relationship, the activation function is used to obtain a probability distribution estimate, wherein the probability distribution estimate is used to indicate probability values ​​corresponding to each of the plurality of candidate action parameters; According to the probability distribution estimation, the target probability value corresponding to the target action parameter is obtained using the classifier; The tail entity corresponding to the current node is determined according to the target probability value.

5. The method according to claim 1, characterized in that Before calling the knowledge graph inference model to obtain the completed target knowledge graph data according to the initial knowledge graph data, the method further includes: Acquire a training data set, the training data set comprising a plurality of sample data groups and respective corresponding standard tail entities, each sample data group comprising a sample head entity and a sample entity relationship, the standard tail entity being pre-calibrated; According to the training data set, reinforcement learning training is performed to obtain the knowledge graph reasoning model.

6. The method according to claim 5, characterized in that After the knowledge graph inference model is called according to the initial knowledge graph data to obtain the completed target knowledge graph data, the method further includes: Adding the target knowledge graph data to the training data set to obtain an updated training data set; The knowledge graph reasoning model is trained according to the updated training data set to obtain an updated knowledge graph reasoning model.

7. A knowledge graph reasoning device, characterized in that: Used in a computer device, the device comprises: An acquisition module, used to acquire initial knowledge graph data to be completed, wherein the initial knowledge graph data includes multiple groups of initial data groups, and the initial data groups only include head entities and entity relationships; A calling module, used to call a knowledge graph reasoning model to obtain completed target knowledge graph data according to the initial knowledge graph data, wherein the knowledge graph reasoning model is a model pre-trained based on reinforcement learning, each target data group in the target knowledge graph data includes the head entity, the entity relationship and the tail entity, the knowledge graph reasoning model is applied to the question-and-answer search graph, the target knowledge graph data is object relationship knowledge graph data or object attribute knowledge graph data, the head entity is a first object, the tail entity is a second object, the entity relationship is a relationship between the first object and the second object, and the second object is different from the first object; Among them, the policy network in the knowledge graph reasoning model includes a long short-term memory network LSTM and a feedforward network. The input parameters of the LSTM are the action parameters of the previous step and the current observation parameters, and the output parameters are historical state parameters. The observation parameters are used to indicate the currently observed environmental content; the input parameters of the feedforward network are the historical state parameters, and the output parameters are the target probability values ​​corresponding to the target action parameters.

8. A computer device, characterized in that: The computer device comprises: a processor; a memory for storing instructions executable by the processor; Wherein, the processor is configured to: A knowledge graph reasoning method, characterized in that it is used in a computer device, and the method comprises: Acquire initial knowledge graph data to be completed, wherein the initial knowledge graph data includes multiple groups of initial data groups, and the initial data groups only include head entities and entity relationships; According to the initial knowledge graph data, a knowledge graph reasoning model is called to obtain the completed target knowledge graph data, wherein the knowledge graph reasoning model is a model pre-trained based on reinforcement learning, each target data group in the target knowledge graph data includes the head entity, the entity relationship and the tail entity, and the knowledge graph reasoning model is applied to the question-answer search graph, the target knowledge graph data is object relationship knowledge graph data or object attribute knowledge graph data, the head entity is a first object, the tail entity is a second object, the entity relationship is a relationship between the first object and the second object, and the second object is different from the first object; Among them, the policy network in the knowledge graph reasoning model includes a long short-term memory network LSTM and a feedforward network. The input parameters of the LSTM are the action parameters of the previous step and the current observation parameters, and the output parameters are historical state parameters. The observation parameters are used to indicate the currently observed environmental content; the input parameters of the feedforward network are the historical state parameters, and the output parameters are the target probability values ​​corresponding to the target action parameters.

9. A non-volatile computer-readable storage medium having computer program instructions stored thereon, characterized in that: When the computer program instructions are executed by a processor, the method according to any one of claims 1 to 6 is implemented.

Images