Method and device for predicting links in knowledge graph

Abstract

The invention relates to the field of link prediction, in particular to a method and a device for predicting links in a knowledge graph. Comprising the steps of initializing a rule generator and a reasoning trainer; training a rule generator by using a training sample, and allocating weights to the relationship based on the time sequence information to obtain at least one link prediction rule for optimization; using a knowledge representation model to represent semantic information related to data, and optimizing a rule generator and a reasoning trainer based on different link prediction rules; and obtaining at least one link prediction rule for prediction by using the optimized rule generator, obtaining at least one prediction link in the knowledge graph according to each link prediction rule for prediction, scoring each prediction link by using the trained reasoning trainer, and taking the prediction link with the highest score as a link prediction result. According to the method, the modeling multilateral relationship in link prediction is solved, and the accuracy of model training is improved.

Description

Technical Field

[0001] The present invention relates to the field of link prediction, and in particular to a method and device for link prediction in a knowledge graph. Background Art

[0002] With the rapid development of network technology and big data, most existing knowledge graphs are constructed by manual or semi-automatic methods, so they often face the challenges of incomplete entity information and poor interpretability. For incompleteness, knowledge graphs often show sparsity, and many potential entity relationships have not yet been discovered. These relationships contain richer and deeper semantic information, which is crucial to improving the effectiveness of knowledge-driven tasks. For poor interpretability, some knowledge graphs constructed by semi-automatic methods do not understand the method of human semantic reasoning, so the constructed knowledge graphs have the problem of being incomprehensible to humans.

[0003] In the application of knowledge graph, the knowledge graph can be queried by specifying query conditions to obtain the link between the query starting point and the query result, thereby obtaining the indirect relationship between two entities that do not have a direct relationship in the knowledge graph, so as to predict new knowledge from the existing knowledge graph or identify logical inconsistencies and complete them. Each link contains a head entity and a tail entity. The head entity is the query starting point, and the tail entity is the query result obtained according to the query conditions. The head entity and the tail entity can contain multiple intermediate entities connected in sequence by different relationships. A path obtained by connecting these entities and relationships in sequence is a link. In actual applications, there can be multiple paths based on different relationships between the head entity and the tail entity. Therefore, the same query condition can also obtain multiple different links.

[0004] Most link prediction methods are complemented by path reasoning and knowledge representation models. Most existing traditional methods, such as path ranking and Markov logic networks, treat the relationship paths on the knowledge graph as candidate logic rules and learn the weight of each rule as an evaluation of the quality of the logic rule. There are also some methods based on neural logic programming and neural theorem provers that can simultaneously learn logic rules and their weights in a differentiable way. Although these methods are effective for prediction in experiments, their exponential search complexity leads to excessive overhead, so identifying high-quality logic rules requires a lot of training. In addition, there are methods that map entities and relationships to vector spaces for reasoning through knowledge representation. For example, TransE attempts to represent relationship vectors as translations in space, but cannot represent the symmetry of the relationship. RotatE represents relationships as rotations in the complex plane to model four relationship patterns, but due to the rotation distance limit, it cannot handle one-to-many and many-to-one relationships, so this method does not perform well in reasoning about complex relationships.

[0005] In view of this, how to overcome the defects of the existing technology and solve the problem that the existing knowledge graph link prediction method cannot handle complex relationship reasoning is a problem to be solved in this technical field. Summary of the invention

[0006] In response to the above defects or improvement needs of the prior art, the present invention solves the problem that the existing knowledge graph link prediction method cannot handle complex relationship reasoning.

[0007] The embodiment of the present invention adopts the following technical solution:

[0008] In a first aspect, the present invention provides a method for link prediction in a knowledge graph, specifically: initializing a rule generator and an inference trainer, extracting structured data from historical links as training samples; using the training samples to train the rule generator, and assigning weights to relationships based on time series information to obtain at least one link prediction rule for optimization; sending the link prediction rules for optimization to a training inferencer, using a knowledge representation model to represent data-related semantic information, and optimizing the rule generator and the inference trainer based on different link prediction rules; using the optimized rule generator to obtain at least one link prediction rule for prediction, obtaining at least one predicted link in the knowledge graph according to each link prediction rule for prediction, using the trained inference trainer to score each predicted link, and taking the predicted link with the highest score as the result of link prediction.

[0009] Preferably, the method of assigning weights to relationships based on time series information to obtain at least one link prediction rule for optimization specifically includes: calculating the prior probability of each relationship in the training sample, and connecting the relationships into corresponding relationship rules according to the prior probability; assigning a time series vector to each relationship in the relationship rule in sequence based on the time series information, and obtaining the weight of each relationship, and representing each relationship as a time series vector through a nonlinear transformation method based on the weight, so as to convert the relationship rules embedded with the time series vector into link prediction rules.

[0010] Preferably, the step of calculating the prior probability of each relationship in the training sample and connecting the relationships into corresponding relationship rules according to the prior probability specifically includes: taking each relationship as the first relationship in the relationship rule, using a recursive neural network to obtain the prior probabilities of all possible next relationships, and adding the next relationship with the highest prior probability to the relationship rule until the number of relationships in the relationship rule reaches a specified number.

[0011] Preferably, the method of assigning a time series vector to each relationship in the relationship rule in order specifically includes: obtaining the order of each relationship in the relationship rule, using the order as the time series vector of the relationship, and defining a corresponding time series weight vector for each time series vector to represent the weight of each time series vector.

[0012] Preferably, the rule generator and the inference trainer are optimized based on different link prediction rules, and the optimization of the inference trainer specifically includes: the inference trainer obtains at least one candidate answer according to the link prediction rule, and calculates the probability distribution of each candidate answer; calculates the weight of each link prediction rule according to the knowledge representation model, maps the semantic information of the entities and relationships in the candidate answers to the high-dimensional vector space according to the weight, calculates the score of each candidate answer after mapping, optimizes the probability distribution of the candidate answers according to the score, and optimizes the inference trainer according to the optimized probability distribution.

[0013] Preferably, the step of calculating the score of each candidate answer after mapping specifically includes:

[0014] The score of each candidate answer can be calculated according to the following formula:

[0015]

[0016] Among them, score w (e) is the score of candidate answer e under prediction parameter w, score w (e|rule) is the score of candidate answer e under link prediction rule rule, ψ w (rule) is the weight of the link prediction rule. is the weight of the link path corresponding to the candidate answer, z is the set of link prediction rules, and P(h, rule, e) is the path set from the head entity h to the tail entity e in the candidate answer under the link prediction rule rule.

[0017] Preferably, the rule generator and the inference trainer are optimized based on different link prediction rules, and the optimization of the rule generator specifically includes: calculating the score of each link prediction rule according to the score of the candidate answer corresponding to each link prediction rule; obtaining a specified number of link prediction rules with the highest scores, and using the obtained link prediction rules to optimize the rule generator.

[0018] Preferably, calculating the score of each link prediction rule specifically includes:

[0019] The score of a link prediction rule is calculated using the following formula:

[0020]

[0021] Among them, G(rule) is the score of the link prediction rule, eq is the hyperparameter that balances the posterior probability and the prior probability, and score w (t|rule) is the score of the specified candidate answer t under the link prediction rule rule, A is the set of all candidate answers, scorew (e|rule) is the score of rule for each candidate answer e under the link prediction rule, RNN θ (rule|r) is the prior score function known by the rule generator to the link prediction rule rule.

[0022] In a second aspect, the present invention provides a device for link prediction in a knowledge graph, specifically comprising: comprising at least one processor and a memory, wherein the at least one processor and the memory are connected via a data bus, and the memory stores instructions that can be executed by the at least one processor, and after the instructions are executed by the processor, they are used to complete the method for link prediction in the knowledge graph in the first aspect.

[0023] In a third aspect, the present invention further provides a non-volatile computer storage medium, wherein the computer storage medium stores computer executable instructions, and the computer executable instructions are executed by one or more processors to complete the method provided in the first aspect.

[0024] In a fourth aspect, a chip is provided, comprising: a processor and an interface, for calling and running a computer program stored in the memory from a memory, and executing the method provided in the first aspect.

[0025] According to a fifth aspect, a computer program product comprising instructions is provided. When the instructions are executed on a computer or a processor, the computer or the processor executes the method provided in the first aspect.

[0026] Compared with the prior art, the beneficial effects of the present invention are: by fusing temporal information, the order of relationships is further distinguished, thereby improving the accuracy of model training; at the same time, a new method is provided in the fusion learning of knowledge reasoning, and a new knowledge representation model is introduced to solve the problems of modeling multilateral relationships in link prediction, the allocation of rules and path weights, and interpretability. BRIEF DESCRIPTION OF THE DRAWINGS

[0027] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the following briefly introduces the drawings required for use in the embodiments of the present invention. Obviously, the drawings described below are only some embodiments of the present invention, and for ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0028] Figure 1 A flow chart of a method for link prediction in a knowledge graph provided by an embodiment of the present invention;

[0029] Figure 2 A flowchart of another method for link prediction in a knowledge graph provided by an embodiment of the present invention;

[0030] Figure 3 A flowchart of another method for link prediction in a knowledge graph provided by an embodiment of the present invention;

[0031] Figure 4 A flowchart of another method for link prediction in a knowledge graph provided by an embodiment of the present invention;

[0032] Figure 5 A schematic diagram of the structure of a device for link prediction in a knowledge graph provided by an embodiment of the present invention;

[0033] The reference numerals are as follows:

[0034] 11: processor; 12: memory. DETAILED DESCRIPTION

[0035] In order to make the purpose, technical solution and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not used to limit the present invention.

[0036] The present invention is an architecture of a specific functional system, so the specific embodiments mainly illustrate the functional logical relationship between the various structural modules, and do not limit the specific software and hardware implementation methods.

[0037] In addition, the technical features involved in the various embodiments of the present invention described below can be combined with each other as long as they do not conflict with each other. The present invention will be described in detail below with reference to the accompanying drawings and embodiments.

[0038] Embodiment 1:

[0039] In actual scenarios, each entity in the knowledge graph corresponds to an actual or abstract entity in the display, and each relationship represents an actual or abstract connection between two entities. The predicted link result can represent the actual or abstract relationship that may exist between two entities that are not directly related. For example: in the knowledge graph used in the logistics scheduling scenario, each entity represents an area, and the relationships between entities include "deliverable" and "included", which respectively indicate that the two areas can be directly delivered, and that one area is part of another area. In this scenario, in order to reduce the amount of data in the knowledge graph and adapt to the logistics constraints in reality, the one-to-one connection relationship between all areas is not saved, but only the directly deliverable connection relationship is retained. When planning the logistics path, it is necessary to plan the path between the starting area and the end area through the inclusion relationship to obtain the best item concentration and distribution plan. However, due to the existence of abstract relationships such as "inclusion" in the knowledge graph, this planning is different from conventional path planning based only on geographic coordinates, and requires planning using link prediction. The shipping location of the item is used as the head entity of the link, and the indirect relationship of "delivery" is used as the rule for reasoning. The delivery location is inferred in the knowledge graph and used as the tail entity of the link.

[0040] In order to solve the problems in the existing link prediction methods in the knowledge graph, this embodiment proposes a link prediction method that integrates time series information and knowledge representation models. By integrating time series information, the order of relationships is further distinguished, and the accuracy of model training is improved. At the same time, a new method is provided in the fusion learning of knowledge reasoning, and a new knowledge representation model is introduced to solve the problems of modeling multilateral relationships, the allocation of rules and path weights, and interpretability in link prediction.

[0041] like Figure 1 As shown, the specific steps of the method for link prediction in the knowledge graph provided by the embodiment of the present invention are as follows:

[0042] Step 101: Initialize the rule generator and the inference trainer, and extract structured data from the historical links as training samples.

[0043] In this embodiment, a link prediction model is constructed using a neural network to predict links in the knowledge graph. The link prediction model includes a rule generator for generating logical rules for link prediction (hereinafter referred to as link prediction rules), an inference trainer for judging link prediction results, and a rule predictor for logical rule prediction. The link prediction model can use the head entity and the link prediction rule as query conditions, perform inference in the knowledge graph, use the answer obtained by inference as the tail entity, and obtain the link between the head entity and the tail entity.

[0044] In order to build a link prediction model, we first need to build a rule generator and an inference trainer, and input the initialization hyperparameters of the training phase. The hyperparameters of the rule generator include: the maximum length of the link prediction rules it generates, the size of the input layer and hidden layer in the recurrent neural network (RNN) used to generate link prediction rules, the number of hidden layers, and the learning rate of the RNN. The hyperparameters of the inference trainer include: the embedding dimension of the training sample initialization, the balance hyperparameters of the prior probability and the posterior probability (hereinafter referred to as the balance hyperparameters), the hyperparameters of the iterative optimization algorithm, and the number of cycles.

[0045] In order to train the link prediction model, it is also necessary to obtain training samples, each of which is a link. In this implementation, the training samples can be represented by triples (G, q, a), where G is the head entity and a is the tail entity; q is the relationship between the head entity and the tail entity, which may be directly included in the knowledge graph or may be composed of multiple continuous relationships directly included in the knowledge graph. In the specific implementation, the training samples can use the real links actually obtained, as well as the incomplete links obtained manually or semi-automatically in the knowledge graph. In order to facilitate processing, the extracted links can be converted into structured data.

[0046] Step 102: Use the training samples to train a rule generator, and assign weights to relationships based on time series information to obtain at least one link prediction rule for optimization.

[0047] When performing link prediction, the timing of the relationship between each entity in the link will affect the associated path in the link. Therefore, in this embodiment, when inputting training samples, it is also necessary to input the timing vector of each relationship between the head entity and the tail entity. The training sample containing the timing vector is used to train the rule generator, and the timing information is incorporated to assign weights to the relationship to generate link prediction rules, so as to improve the accuracy of connection prediction through timing information.

[0048] Step 103: Send the link prediction rules for optimization to the training reasoner, use the knowledge representation model to represent the semantic information related to the data, and optimize the rule generator and the reasoning trainer based on different link prediction rules.

[0049] In this embodiment, in order to solve the problem that the knowledge graph does not understand human semantics during link prediction, a knowledge representation model is introduced in this embodiment, and the semantic information of the data is represented by the knowledge representation model. The link prediction rules generated by the rule generator are sent to the reasoning trainer, and the knowledge representation score function in the knowledge representation model is used to score the reasoning results of the logical rules. The reasoning trainer is optimized according to different knowledge representation score functions, so that the link prediction model can obtain link prediction results that are more in line with human semantics.

[0050] Step 104: Use the optimized rule generator to obtain at least one link prediction rule for prediction, obtain at least one predicted link in the knowledge graph according to each link prediction rule for prediction, use the trained inference trainer to score each predicted link, and use the predicted link with the highest score as the result of link prediction.

[0051] After completing the training of the link prediction model, the trained link prediction model is applied to the test set. The link prediction model generates link prediction rules using the rule generator according to the query conditions, and inputs them into the rule predictor to obtain the link prediction rules for prediction. Then, according to the link prediction rules for prediction, the inference starts from the head entity, and at least one tail entity that meets the connection prediction rules is obtained as a candidate answer. A link between the head entity and each candidate answer is a predicted link. In order to obtain the most accurate prediction results, it is also necessary to score each candidate answer according to the contribution of different link prediction rules, select the candidate answer with the highest score as the inference answer obtained according to the query results, and use the link between the head entity and the inference answer as the result of the link prediction.

[0052] After steps 101 to 104 provided in this embodiment, more accurate link prediction can be achieved by integrating time series information and the knowledge representation model.

[0053] like Figure 2 As shown, the following method can be used to assign weights to relationships based on timing information, and use a rule generator to obtain at least one link prediction rule for optimization.

[0054] Step 201: Calculate the prior probability of each relationship in the training sample, and connect the relationships into corresponding relationship rules according to the prior probability.

[0055] For each training sample triple in the training sample set, after the relationship of the triple is given, the logical rule set corresponding to all relationships (hereinafter referred to as the relationship rule set) is generated based on the relationship through RNN. Based on the prior probability of each relationship, the RNN model is used to generate the logical rules corresponding to each relationship (hereinafter referred to as the relationship rule) in turn according to the relationship between each entity in each training sample, and finally the relationship rule set with prior distribution is obtained. Specifically: for each training sample, each relationship is taken as the first relationship in the relationship rule, and the recursive neural network is used to obtain the prior probability of all possible next relationships, and the next relationship with the highest prior probability is added to the relationship rule until the number of relationships in the relationship rule reaches the specified number.

[0056] Step 202: Based on the time series information, a time series vector is sequentially assigned to each relationship in the relationship rule, and the weight of each relationship is obtained. Based on the weight, each relationship is represented by a time series vector through a nonlinear transformation method to convert the relationship rule embedded with the time series vector into a link prediction rule.

[0057] After obtaining the relationship rule set, you can use the method of adding time series vectors to optimize the relationship rules generated by the rule generator. Assign a time series vector to each relationship in the relationship rule in order. Specifically: obtain the order of each relationship in the relationship rule, that is, obtain the serial number of the position of the relationship in the relationship rule, use the order as the time series vector of the relationship, and define a corresponding time series weight vector for each time series vector to represent the weight of each time series vector. The time series weight vector can be used to correct and adjust the time and relationship vectors in different situations, strengthen the constraints of time information on relationships, and enable the link prediction model to better adapt to the continuous updating of knowledge.

[0058] The relations in the relational rules are represented by vectors through a nonlinear transformation method and then output, so as to obtain the transformed link prediction rules. The prior probability of each link prediction rule is consistent with its corresponding relational rule.

[0059] After step 201 to step 202 provided in this embodiment, the link prediction rule can be obtained through the rule generator.

[0060] like Figure 3 As shown, the following method can be used to optimize the inference trainer according to different link prediction rules.

[0061] Step 301: The inference trainer obtains at least one candidate answer according to the link prediction rule and calculates the probability distribution of each candidate answer.

[0062] When querying according to the link prediction rule, each head entity may obtain multiple tail entities as candidate answers along different link paths. The probability of occurrence of different candidate answers is different. Therefore, the probability distribution of each candidate answer needs to be calculated. In this embodiment, the weights of different logical rules and paths in the link corresponding to each candidate answer can be obtained, and the generated rules are sent to the inference trainer. The probability distribution of the candidate answers is calculated by defining the distribution of the candidate answers and weighted summation.

[0063] Step 302: Calculate the weight of each link prediction rule according to the knowledge representation model, map the semantic information of entities and relationships in the candidate answers to the high-dimensional vector space according to the weights, calculate the score of each candidate answer after mapping, optimize the probability distribution of the candidate answers according to the scores, and optimize the inference trainer according to the optimized probability distribution.

[0064] After integrating the knowledge representation model into the inference trainer, the semantic information of entities and relationships can be mapped to a high-dimensional vector space. After the mapping is completed, it is necessary to calculate the score of each candidate answer after the mapping, and optimize the answer probability distribution according to the different scores of the candidate answers, so as to optimize the inference trainer.

[0065] In specific implementation, the score of each candidate answer can be calculated using the following formula:

[0066]

[0067] Among them, score w (e) is the score of candidate answer e under prediction parameter w, score w (e|rule) is the score of candidate answer e under link prediction rule rule, ψ w (rule) is the weight of the link prediction rule. is the weight of the link path corresponding to the candidate answer, z is the set of link prediction rules, and P(h, rule, e) is the path set from the head entity h to the tail entity e in the candidate answer under the link prediction rule rule.

[0068] After step 301 and step 302 provided in this embodiment, the optimization of the inference trainer can be completed.

[0069] like Figure 4 As shown, the following method can be used to optimize the inference trainer based on different link prediction rules.

[0070] Step 401: Calculate the score of each link prediction rule according to the score of the candidate answer corresponding to each link prediction rule.

[0071] When training the link prediction model, it is necessary to enable the trained rule generator to generate high-quality link prediction rules. In this embodiment, the score of each link prediction rule can be calculated according to the score of the candidate answer obtained by each link prediction rule, so as to evaluate and screen the link prediction rules.

[0072] In specific implementation, the score of the link prediction rule can be calculated using the following formula:

[0073]

[0074] Among them, G(rule) is the score of the link prediction rule, eq is the hyperparameter that balances the posterior probability and the prior probability, and score w (t|rule) is the score of the specified candidate answer t under the link prediction rule rule, A is the set of all candidate answers, score w(e|rule) is the score of rule for each candidate answer e under the link prediction rule, RNN θ (rule|r) is the prior score function known by the rule generator to the link prediction rule rule.

[0075] Step 402: Obtain a specified number of link prediction rules with the highest scores, and use the obtained link prediction rules to optimize the rule generator.

[0076] For each query condition, the rule generator generates multiple link prediction rules. The link prediction rules can be evaluated according to their scores, and the high-quality link prediction rules with the highest scores can be screened out. The rule generator can then be optimized based on the screening results.

[0077] After step 401 and step 402 provided in this embodiment, the optimization of the rule generator can be completed.

[0078] The method for link prediction in the knowledge graph provided in this embodiment further distinguishes the order of relations by integrating time series information, thereby improving the accuracy of model training. At the same time, a new method is provided in the fusion learning of knowledge reasoning, and a knowledge representation model is introduced to solve the problems of modeling multilateral relations, the allocation of rules and path weights, and interpretability in link prediction.

[0079] Embodiment 2:

[0080] Based on the method for link prediction in the knowledge graph provided in Example 1, in different specific application scenarios, it can also be supplemented and adjusted according to different usage requirements or actual scenarios. In the following technical solutions, if there is no conflict, one or more technical solutions can be selected and used in combination with the technical solution in Example 1.

[0081] According to step 101, the rule generator and the inference trainer are initialized, and the initialization hyperparameters of the training phase are input to build a corresponding link prediction model.

[0082] The hyperparameters of the rule generator include: the maximum length of the generated rules, the size of the input layer and hidden layer in the RNN, the number of hidden layers, the learning rate, and the attention hyperparameters W,b,u m For the rule generator, the maximum length of the generated link prediction rules is set to 4, the size of the input layer and hidden layer in the RNN is set to 512 and 256, the number of hidden layers is 1, and the learning rate is set to 1X10 -2 , the time series vector is set to a unit vector of the form [1,0,0..0], [0,1,0,..0]…[0,0…1], and the dimension is the length of the relationship in a single rule.

[0083] The hyperparameters of the inference trainer include: the initial embedding dimension of the training sample triples, the balancing hyperparameter δ, the hyperparameter eq of the Expectation Maximization Algorithm (EM algorithm), and the number of EM algorithm cycles. For the inference trainer, the embedding dimension of the training sample triples is set to 500 and the hyperparameter δ is set to 9.

[0084] In this embodiment, the EM algorithm is used to iteratively optimize the link prediction model. The EM algorithm is an iterative algorithm for maximum likelihood estimation or maximum a posteriori probability estimation of a probability parameter model containing hidden variables. For the EM optimization process, the equilibrium hyperparameter eq is set to 0.9. At the same time, the number of EM cycles is set to 10 times. In each cycle, 1000 link prediction rules are generated from the rule generator for each query. In the E (Expectation) step, 300 rules are identified as high-quality logical rules according to the method provided in steps 401-402, and are returned to the rule generator in the M (Maximization) step.

[0085] When training the link prediction model, input training samples D = (G, q, a) and test set D' = (G', q', a'). In this embodiment, both the training sample set and the test set are structured data extracted from incomplete link prediction events.

[0086] For each training sample triple in the training set, after the relationship of the triple is given, a relationship rule set is generated based on the relationship through a recursive neural network RNN ​​according to step 201. First, each relationship rule is generated in sequence according to the input relationship by using the RNN, and its prior probability is calculated, and finally a relationship rule set with a prior distribution is obtained.

[0087] Formally, relational rules are generated by the relation r in the training sample. Given a relation r, RNN sequentially generates each relation in the relational rule until the maximum length of the relational rule is 4.

[0088] In specific implementation, the prior function of each relationship can be calculated according to the following formula:

[0089] p θ (S i |r)=RNN θ (r 1 ∣r)·RNN θ (r 2 ∣r,r 1 )…RNN θ (r N ∣r,r 1 ,…,rN-1 );

[0090] Among them, p θ (S i |r) is the relation rule S i The prior probability of relation r in S i is the relational rule output by RNN in the rule generator, θ (r 1 |r) is the next possible relation r obtained by inference based on relation r 1 Prior score function, RNN θ (r 2 ∣r,r 1 ) is based on the relationship r and the relationship r 1 Infer the next possible relationship r 2 Prior score function, RNN θ (r N ∣r,r 1 ,…,r N-1 ) is from relation r to relation r N-1 The next possible relation r obtained by inference N The prior score function of .

[0091] The prior function of the relational rule set is:

[0092]

[0093] Among them, p θ (z*|r) is the prior probability of relation r in the relation rule set z*, θ is the parameter that the rule generator needs to train, N is the number of relation rules in the relation rule set, z* is the relation rule set, and the relation rule S generated by RNN i With relation r as the rule head, i∈N, S i ∈z*.

[0094] In the specific implementation, first, at each time step, the RNN in the rule generator will output a probability distribution based on the characteristics of the input relationship, indicating the probability p of the next possible relationship. θ (z*|r), these probabilities are obtained from the output of the RNN through the normalization (softmax) function. Then, the relationship with the highest probability is selected to join the rule. For each rule S generated i , calculate its prior probability. If the relationship rule contains multiple relationships, its probability will be the product of the probabilities of these relationships being generated sequentially. Then repeat the above steps to generate N relationship rules, which constitute the relationship rule set z*. In this embodiment, the prior probability o θ (S i |r) indicates that a specific relation rule S is generated after a given relation r is giveni The conditional probability of , which is the multinomial probability distribution corresponding to the relational rule set z*.

[0095] According to step 202, the relationship rules generated by the rule generator are optimized by adding a time series vector, a time series vector is sequentially assigned to the relationship in each relationship rule, and the relationships in all relationship rules are vectorized by a nonlinear transformation method and then output, so as to obtain a set of link prediction rules.

[0096] In actual implementation, the i-th relationship rule S obtained in step 201 is i As the input of step 202. Then the relationship rule S i The tth relation in is represented by the vector S it , introduce the time vector T t , and define the time series weight vector τ t Since the time vector T t represents the order of the relation in the entire rule, so the relation r embedded in the time series vector it It can be expressed as:

[0097]

[0098] Among them, the operator Represents the Hadmard product between vectors in complex space, that is, the multiplication of the corresponding elements of the time series weight vector and the time series vector.

[0099] Get the relationship r embedded with the time series vector it Later, the rule vector S i 'As a relationship it The set of relational rules S i The combination of relation vectors in :

[0100]

[0101] Then use multiple sampling (mutilnomial) operation to convert the regular vector S i 'Remap to a combination of relations to obtain the required link prediction rules. Specifically: for the rule vector S i Each row of 'takes n random values, n is the hyperparameter of the number of samplings, and each time the value is taken, S i 'The row label corresponding to the vector is used as the output value of a random sampling, and then the row label that appears most times in the output value is used as the subscript of the final relationship. i 'After all the mutilnomial operations are performed, the text representation of a link prediction rule in the link prediction rule set z can be obtained.

[0102] According to step 301, the probability distribution that the answer a of the query q is the tail entity e is calculated by using the softmax function:

[0103]

[0104] First, the inference trainer defines the distribution of answer a as P w (a|G, q, z), for query q, the link predictor uses the link prediction rule z generated by the inference generator to reason on the knowledge graph G and predict the answer a, w is the parameter of the link predictor that needs to be trained.

[0105] Assume that, for each query q = (h, r, ?), the same link prediction rule can find different paths leading to different candidate answers on the knowledge graph G.

[0106] Let set A be the set of candidate answers derived from the link prediction rules in the link prediction rule set z. For each candidate answer e∈A, the scores of different candidate answers e are calculated as score w (e).

[0107]

[0108] Among them, P(h, rule, e) is the set of different paths with h as the head entity and e as the tail entity, ψ w (rule) is the weight of each link prediction rule, φ w (path) is the weight of each path.

[0109] The score of each candidate answer e is the sum of the scores contributed by different link prediction rules, that is, score w (e|rule). In order to obtain the score w (e|rule), it is necessary to perform a weighted summation on each candidate path found in the knowledge graph G. The score is obtained w (e), the probability distribution P can be calculated w (a=e|G,q,z).

[0110] In this embodiment, the knowledge representation model used is the underwater image open source dataset (Real-world Underwater Image Enhancement, abbreviated as RuIE) and the knowledge graph embedding with Householder parameterization (Knowledge Graph Embedding with Householder Parameterization, abbreviated as HousE). In actual implementation, the knowledge representation model used can be selected according to the actual needs of the scene.

[0111] In the inference trainer optimization process of step 301-step 302, RuIE is introduced to calculate the weight ψ of each link prediction rule w (rule), use position-aware sum to model the relationship between logical rules and relations. Assume g(r) is the corner vector of the returned relation r. For each link prediction rule: r1∧r2…r i , and its corresponding embedding representation R = [R1, R2, ..., Ri] is calculated, where i is the length of the link prediction rule and [·,,] is the cascade operation. Based on the above definition, the weight ψ of the link prediction rule can be w (rule) is transformed into:

[0112]

[0113] Among them, g(r k ) is the relationship r k The angular vector, g(R k ) is the relationship r k The corresponding embedding representation R k The angular vector of .

[0114] Then use the knowledge representation model HousE to calculate φ w (path), first introduce the corresponding embedding representation for each entity, and model each relationship that makes up the link prediction rule as a rotation operator on the embedding of different entities. For example: for different paths derived from the head entity h to different tail entities e according to the same link prediction rule, if the relationship in the rule is defined as a rotation operator and used to rotate the embedding representation of entity h, and then calculate the distance between the embedding representations of different entities e, the distance obtained is the weight of the path path φ w (path), the weight of a path is regarded as a measure of the reliability and consistency of the path.

[0115] In the specific implementation, the weight of the path path φ w The embedding representation used by (path) is initialized based on the embedding of each entity and relation in the complex space. Formally, the embedding of entity e is represented by x e , the embedding of relation r is represented as x r , for an existing path Definition φ w (path) is as follows:

[0116]

[0117] Among them, σ(x) is the activation function, d(x) is the distance function between two complex vectors, δ is a hyperparameter, For entity e 0 The embedding representation of For the relationship i The embedding representation of Represents the Hadmard product between vectors in complex space.

[0118] When the EM algorithm iteration number is less than the set number, the EM algorithm is used to iteratively train the inference trainer and the rule generator.

[0119] According to step 201 and step 202, a link prediction rule set is obtained by sampling from the rule generator. and will Then, the parameter w of the inference trainer is updated according to step S301 to step S302, and the optimization target of the parameter w is to maximize the log-likelihood function of the answer a.

[0120] After completing the optimization of the parameter w of the inference trainer, the parameter θ needs to be optimized using the EM algorithm. According to step 401 and step 402, in step E, from the generated link prediction rule set In , K high-quality link sampling rules are identified and incorporated into a set z', namely The size of the set z' is K. In a certain scenario, the query to be performed is q = (h, r, ?), and the answer a = t is inferred by the link prediction model. For the link prediction rule set generated by the rule generator Calculate the following score G for each link prediction rule:

[0121]

[0122] Among them, A is composed of all The set of candidate answers found by the link prediction rules in w (e|rule) is each The score function for different rules in finding the same candidate answer, score w The calculation method of (e|rule) is given by formula S3-2, RNN θ (rule|r) is the known prior score function of the prediction generator for the current rule, and eq is a balancing hyperparameter.

[0123] In the M step, since a set of high-quality link prediction rules z' is obtained in the E step, these link prediction rules are further used to update the parameters θ of the rule generator in the M step. Specifically, the obtained set z' is regarded as part of the current training sample (G, q, a), and the rule generator parameters θ are updated by maximizing the log-likelihood of the set z':

[0124]

[0125] Among them, p θ (S i |r) is the prior probability of the link prediction rule q in the set z', RNN θ (rule|r) is the a priori score function known by the rule generator to the link prediction rule rule, and const is a specified static parameter.

[0126] After completing iterative training and parameter optimization, the trained link prediction model is applied to the test set. The rule generator is used to generate N rules for the test query and input them into the rule predictor. After reasoning based on the rules, the candidate answer with the highest score is selected as the reasoning answer, thus completing the link prediction.

[0127] Embodiment 3:

[0128] Based on the methods provided in Example 1 and Example 2, in some specific implementations, the implementation can be achieved through the specific implementation in this embodiment. It is understood that the specific implementation provided in this embodiment is only used to illustrate the specific implementation process of the method in Example 1 in certain specific scenarios, and is not intended to limit the scope of protection.

[0129] In a specific scenario, the training sample q for the query is q = (location 1, delivery,?), and the answer to the query result is: location 2. In this scenario, the goal of connection prediction is to generate a set of high-quality link prediction rules z* to assist in answering the question of "which destination does location 1 deliver to". In this scenario, the relationship r between the query starting point and the query result is "delivery", which is the link prediction rule in this scenario, which can contain one or more existing relationships contained in the knowledge graph. Use the rule generator to sequentially generate each relationship in the link prediction rule until the maximum length of the link prediction rule is reached. For example, a link prediction rule Rule1 is generated as: (x, delivery, y) = (x, belongs to, z) ← (z, can be delivered, a) ← (a, contains, y). In this link prediction rule, the "delivery" relationship is derived from the three existing relationships of "belongs to", "can be delivered", and "contains". x, y, z, and a are all replaceable entities. Similarly, another link prediction rule Rule2 is derived as: (x, delivery, y) = (x, transit, z) ← (z, deliverable, a) ← (a, contain, y), which means that the relationship "delivery" is derived from the three existing relationships "transfer", "deliverable", and "contain". After embedding each link prediction rule into the time series information and filtering it using the knowledge representation model, the required set of high-quality link prediction rules z* can be obtained.

[0130] After obtaining the link prediction rule, for a given query q = (location 1, delivery, ?), the relationship rules "delivery ←", "belong to ∧", "deliverable □", and "contain" contained in the link prediction rule can be used for reasoning. In actual implementation, the same link prediction rule may derive two different answers from the knowledge graph: (1) delivery (location 1, location 2) ←

[0131] Belongs to (place 1, country a) ∧ can be delivered to (country a, country b) ∧ includes (country b, place 2); (2) delivered to (place 1, place 3)

[0132] ← belongs to (location 1, country a) ∧ can be delivered to (country a, country c) ∧ includes (country c, location 3). Therefore, two candidate answers are generated, location 2 and location 3. After calculation, score w (Location 2) has a better score than w (Place 3), so choose delivery place 2.

[0133] On the other hand, given a path (place 3 →┴ belongs to country a □ (→┴ can be delivered to country b □ (→┴ contains place 2))), the embedding representation of place 3 is rotated using the rotation operator represented by "belongs to", "can be delivered to", and "contains". Then the distance d(x) between the rotation transformation embedding of place 3 and the entity embedding of place 2 is calculated as φw(path). In this way, a specific φw(path) can also be calculated for each path to obtain more accurate scores for different candidate answers.

[0134] Embodiment 4:

[0135] Based on the method for link prediction in the knowledge graph provided in the above embodiments 1 to 3, the present invention also provides a device for link prediction in the knowledge graph that can be used to implement the above method, such as Figure 5 , which is a schematic diagram of the device architecture of an embodiment of the present invention. The device for link prediction in the knowledge graph of this embodiment includes one or more processors 11 and a memory 12. Figure 5 A processor 11 is taken as an example.

[0136] The processor 11 and the memory 12 may be connected via a bus or other means. Figure 5 The example of connecting through bus is taken in the following.

[0137] The memory 12 is a non-volatile computer-readable storage medium as a method for link prediction in a knowledge graph, and can be used to store non-volatile software programs, non-volatile computer executable programs and modules, such as the method for link prediction in a knowledge graph in Examples 1 to 3. The processor 11 executes various functional applications and data processing of the device for link prediction in a knowledge graph by running the non-volatile software programs, instructions and modules stored in the memory 12, that is, implementing the method for link prediction in a knowledge graph in Examples 1 to 3.

[0138] The memory 12 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, or other non-volatile solid-state storage devices. In some embodiments, the memory 12 may optionally include a memory remotely arranged relative to the processor 11, and these remote memories may be connected to the processor 11 via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0139] The program instructions / modules are stored in the memory 12. When executed by one or more processors 11, the method for link prediction in the knowledge graph in the above-mentioned embodiments 1 to 3 is executed. For example, the method described above is executed. Figures 1 to 4 The steps shown.

[0140] An embodiment of the present invention further provides a non-volatile computer storage medium, wherein the computer storage medium stores computer executable instructions, and the computer executable instructions are executed by one or more processors, such as Figure 5 A processor 11 in the embodiment 1 to the embodiment 3 may enable the one or more processors to execute the method for link prediction in the knowledge graph in the embodiment 1 to the embodiment 3, for example, to execute the method described above. Figures 1 to 4 The steps shown.

[0141] It is worth noting that the information interaction, execution process, etc. between the modules and units within the above-mentioned devices and systems are based on the same concept as the processing method embodiment of the present invention. The specific contents can be found in the description of the method embodiment of the present invention and will not be repeated here.

[0142] A person skilled in the art may understand that all or part of the steps in the various methods of the embodiments may be completed by instructing related hardware through a program, and the program may be stored in a computer-readable storage medium, and the storage medium may include: a read-only memory (ROM), a random access memory (RAM), a disk or an optical disk, etc.

[0143] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present invention should be included in the protection scope of the present invention.

Claims

1. A method for link prediction in a knowledge graph, characterized in that: include: Initialize the rule generator and inference trainer, and extract structured data from historical links as training samples; Using the training samples to train a rule generator, and assigning weights to the relationships based on the time series information, to obtain at least one link prediction rule for optimization; The link prediction rules used for optimization are fed into the training reasoner, the semantic information related to the data is represented by the knowledge representation model, and the rule generator and the reasoning trainer are optimized based on different link prediction rules; Use the optimized rule generator to obtain at least one link prediction rule for prediction, obtain at least one predicted link in the knowledge graph according to each link prediction rule for prediction, use the trained inference trainer to score each predicted link, and use the predicted link with the highest score as the result of link prediction.

2. The method for link prediction in a knowledge graph according to claim 1, characterized in that: The step of assigning weights to the relationships based on the time series information to obtain at least one link prediction rule for optimization specifically includes: Calculate the prior probability of each relationship in the training sample, and connect the relationships into corresponding relationship rules according to the prior probability; Based on the time series information, a time series vector is assigned to each relationship in the relationship rule in sequence, and the weight of each relationship is obtained. Based on the weight, each relationship is represented by a time series vector through a nonlinear transformation method to convert the relationship rules embedded with the time series vector into link prediction rules.

3. The method for link prediction in a knowledge graph according to claim 2, characterized in that: The calculating the prior probability of each relationship in the training sample and connecting the relationships into corresponding relationship rules according to the prior probability specifically includes: Take each relationship as the first relationship in the relationship rule, use a recursive neural network to obtain the prior probabilities of all possible next relationships, and add the next relationship with the highest prior probability to the relationship rule until the number of relationships in the relationship rule reaches the specified number.

4. The method for link prediction in a knowledge graph according to claim 2, characterized in that: The step of sequentially allocating a time series vector to each relationship in the relationship rule specifically includes: The order of each relationship in the relationship rule is obtained, the order is used as the time series vector of the relationship, and a corresponding time series weight vector is defined for each time series vector to represent the weight of each time series vector.

5. The method for link prediction in a knowledge graph according to claim 1, characterized in that: The rule generator and the inference trainer are optimized based on different link prediction rules, and the optimization of the inference trainer specifically includes: The inference trainer obtains at least one candidate answer according to the link prediction rule and calculates the probability distribution of each candidate answer; The weight of each link prediction rule is calculated according to the knowledge representation model, the semantic information of entities and relationships in the candidate answers is mapped to a high-dimensional vector space according to the weights, the score of each candidate answer after mapping is calculated, the probability distribution of the candidate answers is optimized according to the score, and the inference trainer is optimized according to the optimized probability distribution.

6. The method for link prediction in a knowledge graph according to claim 5, characterized in that: The calculation of the score of each candidate answer after mapping specifically includes: The score of each candidate answer can be calculated according to the following formula: Among them, score w (e) is the score of candidate answer e under prediction parameter w, score w (e|rule) is the score of candidate answer e under link prediction rule rule, ψ w (rule) is the weight of the link prediction rule. is the weight of the link path corresponding to the candidate answer, z is the set of link prediction rules, and P(h, rule, e) is the path set from the head entity h to the tail entity e in the candidate answer under the link prediction rule rule.

7. The method for link prediction in a knowledge graph according to claim 6, characterized in that: The rule generator and the inference trainer are optimized based on different link prediction rules, and the optimization of the rule generator specifically includes: Calculate the score of each link prediction rule according to the score of the candidate answer corresponding to each link prediction rule; A specified number of link prediction rules with the highest scores are obtained, and the rule generator is optimized using the obtained link prediction rules.

8. The method for link prediction in a knowledge graph according to claim 7, characterized in that: The step of calculating the score of each link prediction rule specifically includes: The score of a link prediction rule is calculated using the following formula: Among them, G(rule) is the score of the link prediction rule, eq is the hyperparameter that balances the posterior probability and the prior probability, and score w (t|rule) is the score of the specified candidate answer t under the link prediction rule rule, A is the set of all candidate answers, score w (e|rule) is the score of rule for each candidate answer e under the link prediction rule, RNN θ (rule|r) is the prior score function known by the rule generator to the link prediction rule rule.

9. A device for link prediction in a knowledge graph, characterized in that: It includes at least one processor and a memory, wherein the at least one processor and the memory are connected via a data bus, and the memory stores instructions that can be executed by the at least one processor, and after the instructions are executed by the processor, they are used to complete the method for link prediction in the knowledge graph described in any one of claims 1-8.

10. A non-volatile computer storage medium, characterized in that: The computer storage medium stores computer executable instructions, which are executed by one or more processors to complete the method for link prediction in the knowledge graph as described in any one of claims 1-8.

Images