A common sense missing information multi-hop reasoning method based on a knowledge graph

Abstract

The application discloses a common-sense missing information multi-hop reasoning method based on a knowledge graph, constructs an input head entity and a query relationship according to common-sense missing information, and carries out preprocessing, constructs a long short-term memory network to encode path historical information, and obtains a hidden state variable; a high-level strategy network is constructed, and the hidden state variable, the preprocessed input head entity and the query relationship are taken as the high-level strategy network to obtain a high-level decision result; a low-level strategy network is constructed to combine the high-level decision result to carry out prediction, obtain a predicted target entity, score, and carry out reinforcement learning iteration according to a preset step maximum value, and obtain a final prediction result according to the score; the learning framework of the application is layered and reinforced, the prediction task is decomposed into a high-level decision process for relationship detection and a low-level decision process for entity reasoning, the action space is effectively controlled, and a more accurate prediction target entity is obtained.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of knowledge graph, and particularly relates to a common sense missing information multi-hop reasoning method based on a knowledge graph. BACKGROUND

[0002] In the past few years, Knowledge Graph, which represents the structural relationship between entities, has attracted great research attention from both academia and industry.

[0003] Common sense knowledge graph stores rich information, especially in the form of triples (head entity, relation, tail entity), such as (Tom Hanks, born in, California). The reasoning task of common sense knowledge graph is mainly to solve the problem of missing information, such as (Tom Hanks, born in,?), predicting the tail entity, i.e. California

[0004] In the prior art, knowledge graph reasoning can be divided into two methods, one is the embedding-based method, which focuses on how to embed nodes and edges into vector space, although the embedding-based model can predict the correct missing information, but its reasoning process cannot be explained and is not intuitive; the other is the multi-hop reasoning method, which applies multiple steps to achieve the target answer while generating its reasoning path. The method based on reinforcement learning is a classic multi-hop reasoning method, which searches for the path in the continuous space, and by introducing multiple reward functions, the path search is more flexible and controllable.

[0005] However, there is still a lack of an elegant method to deal with the one-to-many dilemma, that is, the same entity has multiple relationships, or the same entity relationship pair has multiple tail entities, especially in some data sets with large average entity degree. Previous methods have not considered the logical relationship between entities and relationships, and use (relation, entity) pairs to form the potential action space, which ignores a large amount of valuable and inherent information, making learning difficult and insufficient. Another problem is that the action space (i.e. the combination of all relations and tail entities) will significantly increase, which also increases the difficulty of learning and slows down the training speed. SUMMARY

[0006] In view of the above problems in the prior art, the present application provides a knowledge graph reasoning method based on hierarchical reinforcement learning, which solves the one-to-many dilemma in the prior art, which leads to a significant increase in the action space, slows down the training speed, increases the difficulty of learning and insufficiency.

[0007] In order to achieve the above application purpose, the technical scheme adopted by the present application is:

[0008] A common sense missing information multi-hop reasoning method based on a knowledge graph, comprising the following steps:

[0009] S1, according to the common sense missing information constructs the input head entity and query relationship, and pre-processes, obtains the pre-processed input head entity and query relationship, and initializes the current time step number;

[0010] S2, a long short-term memory network is constructed, and path history information is encoded to obtain a hidden state variable;

[0011] S3, a high-level strategy network is constructed, and the hidden state variable, the pre-processed input head entity and the query relationship are used as the high-level strategy network to obtain a high-level decision result;

[0012] S4, a low-level strategy network is constructed, and the high-level decision result and the pre-processed input head entity and the query relationship are used to obtain a predicted target entity;

[0013] S5, it is judged whether the current step number reaches a preset maximum step number, if yes, the predicted target entity is output, a target entity candidate set is obtained, and step S6 is entered, otherwise, step S2 is returned;

[0014] S6, each target entity in the target candidate set is scored, and the predicted entity corresponding to the highest score is selected as the final prediction result.

[0015] Preferably, step S1 is specifically:

[0016] The common sense missing information is mapped to a vector space, and the representation of the input head entity and the query relationship is initialized, the pre-processed input head entity and the query relationship are obtained, and the current time step number is initialized.

[0017] Preferably, step S3 specifically includes the following steps:

[0018] S31, a high-level strategy state at the current time is constructed according to the pre-processed input head entity and the query relationship;

[0019] S32, a high-level action set is constructed according to the high-level strategy state at the current time;

[0020] S33, an initial high-level strategy network is constructed by using not less than one full connection layer, and the hidden state variable and the high-level action set are used as high-level strategy network inputs to obtain an initial high-level decision result;

[0021] S34, a relationship dropout network is added after the initial high-level decision network to obtain a high-level decision network, and a high-level decision result is obtained in combination with the initial high-level decision result;

[0022] S35, constructing a policy gradient function by using a REINFORCE algorithm, optimizing a high-level policy network, and a gradient update of the high-level policy network can be expressed as:

[0023]

[0024] wherein θ H* is an updated gradient, is an expected function of the high-level policy network, θ H is a high-level policy function parameter, a is a learning rate, and β is a weight of information entropy, is a gradient of the high-level policy function, and log is a logarithmic function, is an entropy regularization term, is a high-level reward function, r q is a query relationship, e t is a current entity accessed at a current time t, is a gradient of the high-level policy network before being updated.

[0025] Preferably, the step S32 is specifically:

[0026] An initial high-level action set is constructed according to a high-level policy state at a current time, and a reverse triple corresponding to a data set in the high-level policy state at the current time is added to obtain a high-level action set.

[0027] Preferably, the initial high-level policy network model in the step S33 is expressed as:

[0028]

[0029] wherein, is a model function output by the initial high-level policy network, σ(.) is a softmax activation function, O t is a stacked option space-based representation, ReLU(.) is an activation function, W2 is a learnable parameter, W1 is a learnable parameter, h t is a hidden state variable, is a high-level policy state at a current time t, r q is a query relationship, e t is a current entity accessed at the current time t.

[0030] Preferably, the model expression of the high-level decision network in the step S34 is:

[0031]

[0032] m i ~ Bernoulli(1-α), i = 1, …, |O t |

[0033] wherein, is a model function outputted by the high-level decision network, is a proportional symbol, m is a mask vector, m i is the i-th parameter vector in the mask vector, is a constant parameter, is a mapping, Bernoulli(.) is a Bernoulli function, and is a learning rate.

[0034] Preferably, step S4 specifically comprises the following sub-steps:

[0035] S41, constructing a low-level strategy state according to the high-level decision result and the preprocessed source entity and query relationship, the low-level strategy state being represented as:

[0036]

[0037] wherein, is the low-level state at the current time t, e s is the source entity, r q is the query relationship, e t is the current entity visited at the current time t, o t is the high-level decision result.

[0038] S42, constructing a low-level action set according to the low-level strategy state at the current time, and then scoring the low-level action space to obtain an optimized low-level action space;

[0039] S43, using a long short-term memory network to encode and update a hidden state variable to obtain an updated hidden state variable, which is represented as:

[0040] h0= LSTM(0, [r0, e s ])

[0041] h t* = LSTM(h t-1 , (o t-1 , a t-1 )), t > 0

[0042] wherein, h t* is the updated hidden state variable at the current time t, h t-1 is the hidden state variable at the previous time, o t-1 , a t-1 are the high-level action vector and the low-level action vector at the previous time, respectively, r0 is an initial relationship, e s is the source entity, and LSTM(.) is a long short-term memory network model function.

[0043] S44, constructing an initial low-level policy network by using not less than one full connection layer, and obtaining a predicted initial target entity probability according to the updated hidden state variable, the low-level policy state and a low-level action set, the initial low-level policy network being expressed as:

[0044]

[0045] wherein, is an initial low-level policy network output function value, σ(.) is a softmax activation function, ReLU(.) is an activation function, W3 and W4 are learnable parameters respectively, A t is a low-level action set, o t is a high-level decision result, i.e., a high-level action vector at a current time t, e t is a current entity visited at the current time t, r q is a query relationship;

[0046] S45, constructing a reward function according to the current entity and the high-level decision result, the reward function being expressed as:

[0047]

[0048] wherein, R l (s T ) is a reward function, e T is an entity at T time, e target is a target entity, and f(.) is an embedding model function based;

[0049] S46, adding a low-level relationship dropout network after the initial low-level policy network to obtain a low-level policy network, and obtaining a predicted target entity probability according to the initial target entity;

[0050] S47, constructing a policy gradient by using a REINFORCE algorithm to optimize the low-level policy network, the gradient update being expressible as:

[0051]

[0052] wherein, θ L is a low-level policy function parameter, β is a weight of information entropy, and α is a learning rate, is a gradient of the low-level policy function, is an expected function of the low-level policy network, log is a logarithmic function, and H(μ) is an entropy regularization term; θ L * is an updated gradient.

[0053] Preferably, the step S42 is specifically:

[0054] According to the current entity and the corresponding query relationship in the low-level strategy state at the current moment, an initial low-level action set is obtained, and a self-loop entity action on the current node is fused to obtain a low-level action set, then according to the current entity and the selected relationship, the probability of all entities as tail entities is calculated, and the top K scored entities are taken as an action space to obtain an optimized low-level action space.

[0055] Preferably, the low-level strategy network of step S46 is represented as:

[0056]

[0057] Wherein, is a low-level strategy network output function value, is a proportional relationship symbol, is a constant parameter, and n is a mask vector of the low-level strategy network.

[0058] Preferably, step S6 is specifically:

[0059] According to the low-level strategy network output function value corresponding to each target entity in the target candidate set, the candidate probability of each target entity is obtained, and the target entity corresponding to the highest score is selected as the final prediction result as the scoring standard.

[0060] The present application has the following beneficial effects:

[0061] By constructing a multi-hop reasoning method as a semi-Markov decision process, input head entities and query relationships are constructed and preprocessed according to common sense missing information, and a long short-term memory network is constructed to encode path history information to obtain a hidden state variable; a high-level strategy network is constructed, and the hidden state variable, the preprocessed input head entity and the query relationship are taken as the high-level strategy network to obtain a high-level decision result; and a low-level strategy network is constructed to combine the high-level decision result for prediction to obtain a predicted target entity, scoring is performed, reinforcement learning iteration is performed according to a preset step maximum value, and a final prediction result is obtained according to the score; the present application adopts a hierarchical reinforcement learning framework, divides the prediction task into a high-level decision process for relationship detection and a low-level decision process for entity reasoning, effectively controls the action space, and obtains a more accurate prediction target entity. BRIEF DESCRIPTION OF DRAWINGS

[0062] Figure 1 A step flowchart of a common sense missing information multi-hop reasoning method based on a knowledge graph provided by the present application;

[0063] Figure 2 A high-level decision process in an embodiment of the present application;

[0064] Figure 3The sub-step flow chart of step S3;

[0065] Figure 4 The low-level decision process in the embodiment of the present application;

[0066] Figure 5 The sub-step flow chart of step S4;

[0067] Figure 6 The history information coding process of the low-level decision process in the embodiment of the present application;

[0068] Figure 7 The experimental comparison graph of embedding different dimensions in the UMLS dataset in the embodiment of the present application. DETAILED DESCRIPTION

[0069] The specific embodiments of the present application are described below to facilitate the understanding of the present application by those skilled in the art, but it should be clear that the present application is not limited to the scope of the specific embodiments, and for those skilled in the art, it is obvious that various changes are within the spirit and scope of the present application defined and determined by the appended claims, and all the inventions utilizing the concept of the present application are within the scope of protection.

[0070] As shown in Figure 1 , the embodiment of the present application provides a common sense missing information multi-hop reasoning method based on a knowledge graph, comprising the following steps:

[0071] S1, constructing input head entity and query relationship according to common sense missing information, and preprocessing to obtain preprocessed input head entity and query relationship, and initializing the current time step;

[0072] Preferably, step S1 is specifically:

[0073] Mapping the common sense missing information to the vector space, and initializing the representation of the input head entity and the query relationship to obtain the preprocessed input head entity and the query relationship, and initializing the current time step.

[0074] Optionally, the common sense missing information of the embodiment of the present application, for example: Tom Hanks, born in x, wherein the birthplace is the missing content in the common sense information, maps the common sense information with information missing to the vector space to obtain the input head entity and the query relationship;

[0075] Optionally, in the embodiment of the present application, the reasoning process of multi-hop reasoning using semi-Markov process is used to calculate the formula representation: p((r, e) | s t ) = p(r | s t )p(e | s t , r), wherein e is an entity, r is a relationship, and s tS1, encode the current time t reinforcement learning agent state; the target of the strategy can be converted by decomposing the learning process, that is, by step-by-step learning, the predicted tail entity is obtained; the reinforcement learning agent can learn the current high-level state from the environment, then combine the high-level policy network to select the optimal scheme, get the current selected relationship as the output of the high-level policy network, enter the low-level policy network, iterate, and finally get the tail entity that meets the target entity or exceeds the maximum preset step number as the predicted tail entity.

[0076] S2, construct a long short-term memory network to encode the path history information to obtain a hidden state variable; optionally, in the present example, the graph and search path information are considered comprehensively, in order to better meet the sorting, the record information that the agent needs to meet not only includes what is happening now, but also what happened in the past, the search history information is encoded by a long short-term memory network (Long short term memory, LSTM), that is, (e s ,r1,e1,…,r t ,e t ) is denoted as h t , in the embodiment of the present application, the dimension of the hidden unit of the long short-term memory network is set to 200, and the number of layers is 3, and the calculation process can be represented as:

[0077] h0=LSTM(0,r0),

[0078] h t =LSTM(h t-1 ,α t-1 ),t>0

[0079] Wherein, r0 is the initial relationship, generally using random or directly using 0 assignment, h0 is the initial hidden state variable, h t-1 is the hidden state variable at the previous moment, h t is the hidden state variable at the current moment, a t-1 is the action vector at the previous moment.

[0080] S3, construct a high-level policy network, and use the hidden state variable, the preprocessed input head entity and the query relationship as the high-level policy network to obtain a high-level decision result.

[0081] Optionally, the goal of the high-level policy network is to select the most likely relationship connected to the target entity, based on the head entity, the query relationship and the entity at the current time t to form the state of the high-level policy, and the high-level policy network makes a decision on the corresponding relationship.

[0082] As shown in Figure 2 , 3 , step S3 specifically includes the following steps:

[0083] S31, constructing a high-level policy state at the current time according to the pre-processed input head entity and the query relationship;

[0084] Optionally, the strategy of the relationship selection in the embodiment of the application depends not only on the current node information in the knowledge graph, but also on the query relationship, and the high-level policy state at the current time can be expressed as: Wherein, e s is the source entity, r q is the query relationship, e t is the current entity accessed at the current time t.

[0085] S32, constructing a high-level action set according to the high-level policy state at the current time;

[0086] Preferably, step S32 is specifically:

[0087] According to the high-level policy state at the current time, an initial high-level action set is constructed, and a reverse triple corresponding to the data set in the high-level policy state at the current time is added, so as to obtain the high-level action set.

[0088] Optionally, the high-level action set O from the state t covers all outgoing relationships of the current entity e t , and can be expressed as: O t ={r|e t ,(e t ,r,e′)∈G}, wherein e′ is the tail entity, so that from O t , a high-level policy μ t is selected, and in order to explore widely, the expansion search process is defined as a fixed time step, and a reverse triple is also added, specifically, taking (e1, r, e2) as an example, a reverse triple (e2, r -1 , e1) is added to the data set, wherein r -1 is the reverse relationship of the triple relationship r.

[0089] S33, constructing an initial high-level policy network by using not less than one full connection layer, and taking the hidden state variable and the high-level action set as the high-level policy network input, so as to obtain an initial high-level decision result;

[0090] Optionally, the initial high-level policy network adopts two full connection layers, the hidden unit dimension of which is 200, and the activation function is ReLU function.

[0091] Preferably, the initial high-level policy network model in step S33 is expressed as:

[0092]

[0093] wherein, is the model function of the initial high-level policy network output, σ(.) is the softmax activation function, O t is the stacked option space-based representation, ReLU(.) is the activation function, W2 is the learnable parameter, W1 is the learnable parameter, h t is the hidden state variable, is the high-level policy state at the current time t, r q is the query relation, e t is the current entity accessed at the current time t.

[0094] S34, adding a relation dropout network after the initial high-level decision network to obtain a high-level decision network, and combining the initial high-level decision result to obtain a high-level decision result;

[0095] Optionally, in order to realize the randomness of the search process, a relation dropout network based on the high-level action is added to make the search more diversified.

[0096] Preferably, the model expression of the high-level decision network in step S34 is:

[0097]

[0098] m i ~ Bernoulli(1-α), i = 1, …, |O t |

[0099] wherein, is the model function of the high-level decision network output, ∝ is the proportional relationship symbol, m is the mask vector, m i is the i-th parameter vector in the mask vector, ∈ is a constant parameter; when m i = 0, the constant parameter ∈ takes a value close to 0, Bernoulli(.) is the Bernoulli function, and α is the learning rate.

[0100] S35, constructing a policy gradient function using the REINFORCE algorithm to optimize the high-level policy network, and the gradient update can be expressed as:

[0101]

[0102] wherein, θ H* is the updated gradient, is the expected function of the high-level policy network, θ H is the high-level policy function parameter, α is the learning rate, and β is the weight of information entropy, is the gradient of the high-level policy function, log is the logarithm function, is the entropy regularization term, is the high-level reward function, r q is the query relation, e t is the current entity visited at the current time t, is the gradient of the high-level policy network before the update.

[0103] Optionally, the high-level reward can be fed back from the environment to obtain an intermediate reward for subsequent reward estimation, if the reinforcement learning agent reaches the target entity, it will get 1 as encouragement, otherwise it will get a number less than 1 as punishment; in the embodiment of the application, a pre-trained embedding-based model can be used to assist in assigning a score to evaluate the possibility of the triple.

[0104] Optionally, the goal of the high-level policy network is to maximize the expected cumulative return, which can be expressed as:

[0105] where J(θ H ) is the objective function of the high-level policy, is its expected function, G is a knowledge graph, is its expected function, R h (s T ) is the high-level reward function, e s is the source entity, r q is the query relation; in order to optimize the above objective, the policy gradient method and the REINFORCE algorithm are used in the embodiment of the application, and in order to explore more paths, the entropy regularization terms H(μ) and H(π) are added in the embodiment of the application.

[0106] S4, constructing a low-level policy network, and obtaining a predicted target entity according to the high-level decision result and the preprocessed head entity and query relation;

[0107] Optionally, similar to the high-level policy network, the task of low-level reinforcement learning is to determine the tail entity according to the head entity and the selected relation; in order to make the selected relation accessible in the low-level policy process, the embodiment of the application selects the option o t as an additional input of the low-level tail entity decision process. In this way, the result of the high-level decision will affect the learning of the low-level policy.

[0108] As shown in Figure 4 , 5 , step S4 specifically includes the following steps:

[0109] S41. Construct a low-level strategy state based on the high-level decision results and the preprocessed source entities and query relationships. The low-level strategy state is represented as follows:

[0110]

[0111] in, e represents the low-level state at the current time t. s For the source entity, r q To query the relationship, e t For the current entity visited at time t, o t This is a result of high-level decision-making.

[0112] S42. Construct a low-level action set based on the low-level policy state at the current moment, and then score the low-level action space to obtain the optimized low-level action space.

[0113] Preferably, step S42 specifically includes:

[0114] Based on the current entity and its corresponding query relationship edge set in the current low-level strategy state, an initial low-level action set is obtained. The self-loop entity actions on the current node are then merged to obtain a low-level action set. Then, based on the relationship between the current entity and the selection, the probability of all entities being tail entities is calculated. The entities with the top K scores are used as the action space to obtain the optimized low-level action space.

[0115] Optionally, a variant, a dynamic prospect mechanism, is included in the constructed high-level and low-level frameworks to improve the action space. In the low-level reinforcement learning process, the action space originally consists of all entities connected to the corresponding head entities and relations. Embedded models are robust due to their mapping to a low-dimensional vector space. Therefore, in this embodiment, embedding information is injected into the action space during decision-making. Specifically, for the current entity and the selected relation, the probability of all entities being tail entities is calculated, and then the action space is based on the top K rated entities, where K is a preset value in each dataset.

[0116] Optionally, the action at time t is to select the most promising tail entity for the previous (selected) entity-relationship pair and the head entity-relationship (query) pair. (Low-level state) The action space At is the current entity e t and the selected relation r t The set of outgoing edges, where A t ={(e′|e t ,r t ,(e t ,r t ,e t ′)∈G)},et ′ represents the corresponding tail entity; to enable the reinforcement learning agent to make more intelligent choices, this embodiment of the invention also adds a special action to each action space, namely, a self-looping entity action that stays on the current node, and according to the decision relationship of the higher-level policy, the action space At satisfies This indicates that all outgoing edges (tail entities) of a specific head entity and relationship are superimposed in the action space At, where d2 is the entity embedding dimension.

[0117] S43. Using a Long Short-Term Memory (LSTM) network, the hidden state variables are encoded and updated to obtain the updated hidden state variables, represented as follows:

[0118] h0 = LSTM(0, [r0, e) s ])

[0119] h t* =LSTM(h t-1 ,(o t-1 ,α t-1 ),t>0

[0120] Among them, h t* h is the updated hidden state variable at the current time t. t-1 o is the hidden state variable from the previous time step. t-1 a t-1 These are the high-level action vector and low-level action vector from the previous time step, respectively. r0 is the initial relation, and e s For the source entity;

[0121] Optional, such as Figure 6 As shown, to simplify calculation and representation, a Long Short-Term Memory (LSTM) network with the same structure as the LSM network is adopted, that is, the hidden unit dimension of its LSM network is set to 200 and the number of layers is 3; the entire path is recorded by high-level and low-level decision strategies.

[0122] S44. Construct an initial low-level policy network using at least one fully connected layer, and obtain the predicted initial target entity based on the updated hidden state variables, low-level policy state, and low-level action set. The initial low-level policy network is expressed as:

[0123]

[0124] in, Let σ(.) be the initial low-level policy network output function value, σ(.) be the softmax activation function, ReLU(.) be the activation function, W3 and W4 be the learnable parameters, and A be the output function of the network. t For a set of low-level actions, o t For the high-level decision result, i.e., the high-level action vector at the current time t, et r is the current entity accessed at the current time t q is the query relation

[0125] Optionally, the initial low-level policy network in the embodiment comprises two fully connected layers, the hidden unit dimension of which is 200, and the activation function adopts the ReLU function.

[0126] S45, according to the current entity and the high-level decision result, a reward function is constructed for scoring, and the reward function is represented as:

[0127]

[0128] wherein, R l (s T ) is the reward function, e T is the entity at T time, e target is the target entity, and f(.) is a function based on an embedded model;

[0129] S46, a low-level relation dropout network is added after the initial low-level policy network to obtain a low-level policy network, and a predicted target entity is obtained according to the initial target entity;

[0130] Preferably, the low-level policy network of step S46 is represented as:

[0131]

[0132] wherein, is the output function value of the low-level policy network, ∝ is a proportional relationship symbol, ∈ is a constant parameter, and n is a mask vector of the low-level policy network.

[0133] S47, a policy gradient is constructed by using the REINFORCE algorithm to optimize the low-level policy network, and the gradient update can be represented as:

[0134]

[0135] wherein, θ L is a low-level policy function parameter, β is the weight of information entropy, α is the learning rate, is the gradient of the low-level policy function, is the expected function of the low-level policy network, log is a logarithmic function, and H(μ) is an entropy regularization term; θ L * is the updated gradient.

[0136] Optionally, the target of the low-level policy is to maximize the expected cumulative return, which can be represented as: wherein, J(θ L ) is the target function of the low-level policy, G is a knowledge graph for its corresponding expected function, R is its expected function, l (s T ) is a low-level reward function; and based on the goal of maximizing the expected cumulative return, the embodiment of the application adopts a policy gradient method and a REINFORCE algorithm, and in order to explore more paths, entropy regularization terms H(μ) and H(π) are added to optimize the low-level policy network.

[0137] S5, judging whether the current step number reaches a preset maximum step number, if yes, outputting a predicted target entity, obtaining a target entity candidate set, and entering step S6, otherwise returning to step S2;

[0138] S6, scoring each target entity predicted in the target candidate set, and selecting a predicted entity corresponding to the highest score as a final prediction result.

[0139] Preferably, step S6 specifically comprises:

[0140] According to the low-level policy network output function value corresponding to each target entity predicted in the target candidate set, a candidate probability of each target entity is obtained, and the candidate probability is taken as a scoring standard, and a target entity corresponding to the highest score is selected as the final prediction result.

[0141] Optionally, the embodiment of the application evaluates the effectiveness of the above algorithm based on four benchmark data sets; that is:

[0142] 1) FB15K-237 from Freebase, following DacKGR, we constructed FB15K-237-10% and FB15K-237-50%, which reserved 10% and 50% of the original data set facts;

[0143] 2) NELL-995;

[0144] 3) Kinship;

[0145] 4) Unified Medical Language System (UMLS);

[0146] The entities and relations between the data sets are shown in Table 1. It can be seen that small-scale knowledge graphs often face the dilemma of large average degree and one-to-many. At the same time, large-scale knowledge graphs are often affected by huge entity spaces.

[0147] Table 1

[0148]

[0149] Evaluation index:

[0150] On the test set, based on (e eadr,?) (tail entity unknown), ranking the true tail entity e tail in the set of candidate tail entities, assuming rank i is the i-th query of the correct tail entity. The effectiveness is verified by using the following two evaluation indexes:

[0151] 1) Hits@K = å i 1(rank i <K) / N

[0152] 2)

[0153] Wherein, MRR (Mean Reciprocal Rank) is the average reciprocal rank, and Hits@K is the probability that the correct answer is predicted to be ranked no more than K.

[0154] Parameter setting:

[0155] In practice, in order to make a fair comparison, the dimensions of entities and relations can be set to 200, in the historical information coding part, a three-layer LSTM is used and the dimension of the hidden state is set to 200; in addition, the weight of the added entropy regularization term is set to be between 0 and 0.1; and Adam optimization is used, and the learning rate is between 0.001 and 0.003; specifically, different learning rates are used for different data sets; in the dynamic prospect of the low-level action space, the K of UMLS is set to 16, the Kinship is set to 96, the NELL-995 is set to 32, and the FB15K-237 is set to 64; in the embodiment of the application, the PyTorch framework is used to realize and train the model on a GPU machine.

[0156] Experimental results:

[0157] 1) Model comparison

[0158] As shown in Table 2, the results of UMLS show that the model of the embodiments of the present application has a larger average degree. On the same small dataset Kinship, the embodiments of the present application improve the Hits@1 and MRR performance. Interestingly, on NELL-995, the model of the embodiments of the present application is significantly better than previous methods. On FB15K-237-10% and FB15K-237-50%, the embodiments of the present application show advantages in Hits@3 and Hits@10, and competitive results in MRR; the embodiments of the present application slightly improve the FB15K-237 dataset Guess that the entities and relations of FB15K-237 cover many fields compared with other datasets, which may confuse the agent learning. Not only for entities with large degrees, but also for entities with small degrees, the embodiments of the present application can solve the dilemma and achieve excellent results. This means that the model of the embodiments of the present application is robust and effective for multi-hop reasoning;

[0159] 2) Ablation experiments

[0160] For the variant of the hierarchical framework, the dynamic prospect is evaluated, as shown in Table 3, when the dynamic prospect module is removed, the MRR is slightly decreased compared with the model of the embodiments of the present application on NELL-995, UMLS and Kinship datasets. In addition, the embodiments of the present application calculate the MRR results of 16, 32, 96g in K, where K is the number of candidates. The embodiments of the present application can see that different K values need to be set for different sizes of datasets. In general, datasets with larger average degrees of entities need larger K, except for UMLS, where K=16 is the best. In summary, dynamic prospect is an indispensable component in the framework of the embodiments of the present application to improve the action space, so that the hierarchical framework of the embodiments of the present application achieves better performance in common sense knowledge graph reasoning.

[0161] 3) Analysis experiments

[0162] a. Generalization ability

[0163] The generalization ability to unseen data is crucial for further applications. In the experiments of the embodiments of the present application, some triplets in the test set are seen in the training set. According to the first row in Table 4 and Table 5, the embodiments of the present application can see about 96.8% of the validation set on Kinship have seen the query data before, while only 15.3% have seen the query on NELL-995. To investigate the generalization of the model of the embodiments of the present application to unseen queries, the embodiments of the present application conduct experiments on seen and unseen queries in the datasets, including a small number of unseen queries (Kinship and UMLS) and a large number of unseen queries (NELL-995), compared with Multihop-KG. The results of the above two tables show that the performance of the model of the embodiments of the present application is better than the Multihop-KG method in terms of seen and unseen queries. The hierarchical framework of the embodiments of the present application shows strong generalization in different datasets of different sizes;

[0164] b. The impact of embedding dimension

[0165] Entity and relation embeddings carry extremely diverse and rich information. Different dimensions of embeddings can pass different amounts of information to agents for reasoning. In this way, the embodiments of the present application conduct an experiment to investigate how the dimensions of relation and entity embeddings affect learning. The embedding dimensions are tested at 100, 200, 300, as shown in Figure 7

[0166] The embodiments of the present application can see that the performance of the indicators (e.g., MRR, Hits@1, Hits@3 and Hits@10) is better as the dimension increases; it can be known that this is because the relation and entity embeddings with larger dimensions can carry enough information for reasoning.

[0167] Table 2

[0168]

[0169] Table 3

[0170] Model NELL-995 UMLS Kinship Remove dynamic look-ahead mechanism 81.2 89.2 81.1 K＝16 81.0 94.5 83.8 K＝32 82.1 90.0 82.0 K＝96 80 89.9 85.8

[0171] Table 4

[0172]

[0173] Table 5

[0174]

[0175] ​The present application is described in reference to the flowchart and / or block diagram of the method, apparatus (system) and computer program product according to the embodiments of the present application. It should be understood that each flow and / or block in the flowchart and / or block diagram, and the combination of flows and / or blocks in the flowchart and / or block diagram can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing apparatus to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing apparatus generate a means for implementing the functions specified in the flowchart and / or block diagram. Figure 1 one or more flows and / or blocks. Figure 1 one or more flows and / or blocks.

[0176] These computer program instructions can also be stored in a computer readable memory that can direct the computer or other programmable data processing apparatus to work in a specific manner, so that the instructions stored in the computer readable memory produce a manufacture product including instruction apparatus, which implements the functions specified in the flowchart and / or block diagram. Figure 1 one or more flows and / or blocks. Figure 1 one or more flows and / or blocks.

[0177] These computer program instructions can also be loaded into a computer or other programmable data processing apparatus, so that a series of operation steps are performed on the computer or other programmable data processing apparatus to produce a computer implemented process, so that the instructions executed on the computer or other programmable data processing apparatus provide a means for implementing the functions specified in the flowchart and / or block diagram. Figure 1 one or more flows and / or blocks. Figure 1 one or more flows and / or blocks.

[0178] The principles and implementation manners of the present application are described in the specific embodiments, and the above embodiment description is only used to help understand the method of the present application and its core idea; at the same time, for those skilled in the art, according to the idea of the present application, the specific implementation manners and application scope will be changed, and the above description should not be understood as limiting the present application.

[0179] Those skilled in the art will realize that the embodiments described herein are for the purpose of helping the reader understand the principles of the present application, and should be understood as not limited to such specific statements and embodiments. Those skilled in the art can make various other specific modifications and combinations according to the technical inspiration disclosed in the present application without departing from the essence of the present application, and these modifications and combinations are still within the protection scope of the present application.

Claims

1. A knowledge graph-based common-sense missing information multi-hop reasoning method, characterized in that, The method comprises the following steps: S1, constructing a common sense knowledge graph according to the common sense missing text information to obtain an input head entity and a query relationship, and performing preprocessing to obtain a preprocessed input head entity and a query relationship, and initializing a current time step; S2, constructing a long short-term memory network to encode search history information to obtain a hidden state variable of search path information; S3, constructing a high-level strategy network, and taking the hidden state variable of the search path information, the preprocessed input head entity and the query relationship as the high-level strategy network to obtain a relationship decision result connected to a target entity, specifically comprising the following steps: S31, constructing a high-level strategy state at the current time according to the preprocessed input head entity and the query relationship; S32, constructing a high-level action set according to the high-level strategy state at the current time; S33, constructing an initial high-level strategy network by using a full connection layer of no less than one layer, and taking the hidden state variable and the high-level action set as high-level strategy network inputs to obtain an initial high-level decision result; S34, adding a relationship dropout network after the initial high-level decision network to obtain a high-level decision network, and combining the initial high-level decision result to obtain a high-level decision result; S35, constructing a strategy gradient function by using a REINFORCE algorithm to optimize the high-level strategy network, and the gradient update can be expressed as: wherein, is the updated gradient, is the expected function of the high-level policy network, is the high-level policy function parameter, is the learning rate, is the weight of the information entropy, is the gradient of the high-level policy function, is the logarithmic function, is the entropy regularization term, is the high-level reward function, is the query relation, is the current entity accessed at the current time t, is the gradient of the high-level policy network before updating; S4, constructing a low-level strategy network, and taking the relationship decision result connected to the target entity and the preprocessed input head entity and the query relationship to obtain a predicted target entity; S5, judging whether the current step number reaches a preset maximum step number, if yes, outputting the predicted target entity to obtain a target entity candidate set, and entering step S6, otherwise returning to step S2; S6, scoring each predicted target entity in the target candidate set, and selecting a predicted entity corresponding to the highest score as a final prediction result.

2. The knowledge graph-based common sense missing information multi-hop reasoning method according to claim 1, characterized in that, Step S1 is specifically: mapping the common sense missing information to a vector space, initializing the representation of the input head entity and the query relationship, obtaining the preprocessed input head entity and the query relationship, and initializing the current time step.

3. The knowledge graph-based common sense missing information multi-hop reasoning method according to claim 1, characterized in that, Step S32 is specifically: constructing an initial high-level action set according to the high-level strategy state at the current time, and adding inverse triplets corresponding to the data set in the high-level strategy state at the current time to obtain the high-level action set.

4. The knowledge graph-based common sense missing information multi-hop reasoning method according to claim 1, characterized in that, The initial high-level policy network model is represented in step S33 as: wherein, is a model function for the initial high-level policy network output, is a softmax activation function, is a stacked option space based representation, is an activation function, is a learnable parameter, is a learnable parameter, is a hidden state variable, is a high-level policy state at the current time t under the current time is a query relation, is a current entity accessed at the current time t.

5. The knowledge graph-based common sense missing information multi-hop reasoning method according to claim 4, characterized in that, The model expression of the high-level decision network in step S34 is: in, The model function output by the high-level decision network. The sign for a direct proportional relationship is... For the mask vector, For the first digit in the mask vector i A parameter vector It is a constant parameter. For mapping, It is a Bernoulli function. This is the learning rate.

6. The knowledge graph-based common sense missing information multi-hop reasoning method according to claim 1, characterized in that, Step S4 specifically comprises the following steps: S41, constructing a low-level strategy state according to the high-level decision result and the preprocessed source entity and the query relationship, and the low-level strategy state is expressed as: wherein, is a low-level state at a current time t, is a source entity, is a query relationship, is a current entity accessed at the current time t, is a high-level decision result; S42, constructing a low-level action set according to the low-level strategy state at the current time, and then scoring the low-level action space to obtain an optimized low-level action space; S43, encoding and updating the hidden state variable by using a long short-term memory network to obtain an updated hidden state variable, which is expressed as: wherein, is an updated hidden state variable at a current time t, is a hidden state variable at a previous time, are a high-level action vector and a low-level action vector, respectively, of a previous time, is an initial relationship, is a source entity, is a long short-term memory network model function; S44, an initial low-level policy network is constructed by using not less than one full connection layer, and an initial target entity probability is obtained according to the updated hidden state variable, the low-level policy state and the low-level action set, and the initial low-level policy network is expressed as: wherein, is an initial low-level policy network output function value, is a softmax activation function, is an activation function, are learnable parameters, respectively, is a low-level action set, is a high-level decision result, i.e., a high-level action vector at the current time t, is a current entity accessed at the current time t, is a query relationship; S45, a reward function is constructed according to the current entity and the high-level decision result, and the reward function is expressed as: wherein, is a reward function, is an entity at time T, is a target entity, is an embedded model function based; S46, a low-level relationship dropout network is added after the initial low-level policy network to obtain a low-level policy network, and a predicted target entity probability is obtained according to the initial target entity; S47, a policy gradient is constructed by using a REINFORCE algorithm to optimize the low-level policy network, and the gradient update can be expressed as: wherein, is a low-level policy function parameter, is a weight of information entropy, is a learning rate, is a gradient of a low-level policy function, is an expected function of a low-level policy network, is a logarithmic function, is an entropy regularization term; is an updated gradient.

7. The knowledge graph-based common sense missing information multi-hop reasoning method according to claim 6, characterized in that, Step S42 is specifically: According to the current entity and the corresponding query relationship out-line edge set in the low-level policy state at the current time, an initial low-level action set is obtained, and a self-loop entity action on the current node is fused to obtain a low-level action set, then the probability of all entities as tail entities is calculated according to the current entity and the selected relationship, and the top K scored entities are taken as the action space to obtain an optimized low-level action space.

8. The knowledge graph-based common sense missing information multi-hop reasoning method according to claim 6, characterized in that, The low-level policy network of step S46 is expressed as: wherein, is a low-level policy network output function value, is a proportional relationship symbol, is a constant parameter, is a mask vector of the low-level policy network. 9.The knowledge graph based common sense missing information multi-hop reasoning method according to claim 1, characterized in that, Step S6 is specifically: According to the low-level policy network output function value corresponding to each target entity in the target candidate set, the candidate probability of each target entity is obtained, and the candidate probability is taken as the scoring standard to select the target entity corresponding to the highest score as the final prediction result.

Images