Path reasoning method, device, electronic device and medium based on knowledge graph

Abstract

The present application provides a path reasoning method, device, electronic device and medium based on knowledge graph, the path reasoning method includes: obtaining the search entity and query relationship in the path reasoning task; inputting the search entity and query relationship in the path reasoning task into the path reasoning model trained in the knowledge graph to perform path reasoning, and obtaining the path reasoning result corresponding to the path reasoning task in the knowledge graph; wherein the path reasoning model includes a knowledge embedding module, an environment module and a strategy module; the strategy module is trained by meta-learning. The technical solution provided by the present application can be used to train the model by combining the meta-learning method, input the search entity and query relationship in the path reasoning task into the model to infer and find the effective path between the search entity and the query relationship, thereby improving the efficiency of knowledge graph reasoning and ensuring the accuracy of path reasoning.

Description

Technical Field

[0001] The present application relates to the field of artificial intelligence technology, and in particular to a path reasoning method, device, electronic device and medium based on a knowledge graph. Background Art

[0002] As the scale of information data in the network becomes increasingly large, the data content and storage format are diverse, making it difficult to perform subsequent analysis and processing. To this end, the use of knowledge graphs to store data in the form of graph structures solves the problem of representing heterogeneous information in industry data applications. However, in certain specific scenarios, the data in knowledge graphs has the characteristics of strong privacy and sparse relationships, making it impossible to restore data in specific scenarios. For downstream application tasks, using knowledge graphs with noise and insufficient data as the basis for application model training will cause deviations in the model training results, resulting in the failure of application tasks. The task of performing knowledge graph reasoning under the condition of small sample data needs to be solved urgently.

[0003] At present, in the reasoning process of traditional knowledge graphs, manual mining of reasoning rules is required. However, under normal circumstances, it is difficult to exhaust all the knowledge data in the knowledge graph, and this method consumes a lot of manpower costs while still failing to guarantee the reliability of the rules and the quality of reasoning. Therefore, how to reason about knowledge graph paths has become an urgent problem to be solved. Summary of the invention

[0004] In view of this, the purpose of this application is to provide a path reasoning method, device, electronic device and medium based on a knowledge graph, which can train the model in combination with meta-learning, input the search entities and query relationships in the path reasoning task into the model to infer and find the effective path between the search entities and query relationships, thereby improving the efficiency of knowledge graph reasoning and ensuring the accuracy of path reasoning.

[0005] This application mainly includes the following aspects:

[0006] In a first aspect, an embodiment of the present application provides a path reasoning method based on a knowledge graph, and the path reasoning method includes:

[0007] Obtain search entities and query relations in path reasoning tasks;

[0008] Input the search entity and query relationship in the path reasoning task into the path reasoning model trained in the knowledge graph to perform path reasoning, and obtain the path reasoning result corresponding to the path reasoning task in the knowledge graph;

[0009] The path reasoning model includes a knowledge embedding module, an environment module and a strategy module; the strategy module is trained by meta-learning.

[0010] Furthermore, the step of inputting the search entity and query relationship in the path reasoning task into the path reasoning model trained in the knowledge graph to perform path reasoning, and obtaining the path reasoning result corresponding to the path reasoning task in the knowledge graph, includes:

[0011] Inputting the search entity and the query relationship in the path reasoning task into the knowledge embedding module, determining the search entity vector corresponding to the search entity and the query relationship vector corresponding to the query relationship;

[0012] Inputting the search entity vector and the query relationship vector into the action space of the environment module to determine the environment information at the current moment;

[0013] The environmental information at the current moment is input into the strategy module to obtain the path reasoning result corresponding to the path reasoning task in the knowledge graph.

[0014] Furthermore, the step of inputting the search entity vector and the query relationship vector into the action space of the environment module to determine the environment information at the current moment includes:

[0015] Input the search entity vector and the query relationship vector into the action space of the environment module, use the search entity vector as the entity vector at the current moment, and determine the tail entity vector corresponding to the entity vector at the current moment and the entity transition probability corresponding to each tail entity vector in each triple vector information pre-stored in the action space; wherein each triple vector information is composed of a head entity vector, a corresponding relationship vector and a tail entity vector;

[0016] Determine a set of tail entity vectors corresponding to entity transition probabilities greater than a preset probability threshold as a tail entity probability vector;

[0017] According to the sampling strategy, a tail entity vector is selected from the tail entity probability vector as a predicted tail entity vector;

[0018] Obtaining a historical reasoning information vector at a current moment from a long short-term memory network of the path reasoning model;

[0019] The predicted tail entity vector, the query relationship vector, the entity vector at the current moment, and the historical reasoning information vector at the current moment are concatenated to obtain the environmental information at the current moment.

[0020] Furthermore, the strategy module in the path reasoning model is trained through the following steps:

[0021] Acquire a sample reasoning task, and randomly sample the triplet vector information stored in the action space of the environment module according to a preset ratio based on the sample reasoning task to obtain a support set and a verification set;

[0022] Inputting the head entity vector and the corresponding relationship vector of each triplet vector information in the support set into the strategy module for path search to obtain a first loss function;

[0023] Based on the first loss function, the module parameters in the strategy module are updated, and the path search for the next sample reasoning task is continued to update the module parameters in the strategy module until the first loss function converges to obtain an updated strategy module;

[0024] Inputting the head entity vector and the corresponding relationship vector of each triplet vector information in the verification set into the strategy module for path search to obtain a second loss function;

[0025] Based on the second loss function, the path selection parameters in the strategy module are updated, and the path search for the next sample reasoning task is continued to update the path selection parameters in the strategy module until the second loss function converges to obtain a trained strategy module.

[0026] Furthermore, the triple vector information is stored in the action space of the environment module through the following steps:

[0027] Obtain a sample data set consisting of multiple triples, wherein each triple includes a head entity, a corresponding relationship, and a tail entity of a fact;

[0028] Through the knowledge embedding module in the path reasoning model, each triplet is embedded into the complex space in the form of a rotation vector, and self-adversarial negative sampling training is performed to obtain triplet vector information corresponding to each triplet after embedding;

[0029] Each triplet vector information is input into the action space of the environment module, an entity transition probability corresponding to each triplet vector information is generated, and each triplet vector information and the entity transition probability corresponding to each triplet vector information are stored in the action space of the environment module.

[0030] Furthermore, the step of inputting the current environmental information into the strategy module to obtain the path reasoning result corresponding to the path reasoning task in the knowledge graph includes:

[0031] Inputting the current environmental information into the strategy module, and determining a strategy function of the strategy module based on the environmental information;

[0032] Acquire multiple search paths corresponding to the entity vectors at the current moment in the environment information in the knowledge graph, and determine the relationship probability of each search path at the current moment through the strategy function;

[0033] For each search path, based on the relationship probability of the search path at the current moment, determine the evaluation score of the search path;

[0034] Among the evaluation scores of each search path, the search path corresponding to the evaluation score with the largest value is determined as the reasoning path;

[0035] Performing a similarity comparison between the corresponding relationship vector in the reasoning path and the query relationship vector;

[0036] If the similarity is greater than a preset similarity, the tail entity vector corresponding to the corresponding relationship vector in the reasoning path is determined as the path reasoning result corresponding to the path reasoning task in the knowledge graph.

[0037] Furthermore, the path reasoning method further includes:

[0038] If the similarity is not greater than the preset similarity, the tail entity vector corresponding to the next corresponding relationship vector of the entity vector at the current moment is updated to the entity vector at the current moment in the action space for the next path reasoning until all corresponding relationship vectors of the entity vector at the current moment are traversed.

[0039] In a second aspect, the embodiment of the present application further provides a path reasoning device based on a knowledge graph, the path reasoning device comprising:

[0040] The acquisition module is used to obtain the search entities and query relations in the path reasoning task;

[0041] A reasoning module, used to input the search entity and query relationship in the path reasoning task into the path reasoning model trained in the knowledge graph to perform path reasoning, and obtain the path reasoning result corresponding to the path reasoning task in the knowledge graph;

[0042] The path reasoning model includes a knowledge embedding module, an environment module and a strategy module; the strategy module is trained by meta-learning.

[0043] In the third aspect, an embodiment of the present application also provides an electronic device, comprising: a processor, a memory and a bus, wherein the memory stores machine-readable instructions executable by the processor, and when the electronic device is running, the processor and the memory communicate through the bus, and when the machine-readable instructions are executed by the processor, the steps of the path reasoning method based on the knowledge graph are performed as described above.

[0044] In a fourth aspect, an embodiment of the present application further provides a computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, the steps of the path reasoning method based on the knowledge graph as described above are executed.

[0045] The embodiments of the present application provide a path reasoning method, device, electronic device and medium based on a knowledge graph. The path reasoning method includes: obtaining search entities and query relationships in a path reasoning task; inputting the search entities and query relationships in the path reasoning task into a path reasoning model trained in the knowledge graph to perform path reasoning, and obtaining the path reasoning result corresponding to the path reasoning task in the knowledge graph; wherein the path reasoning model includes a knowledge embedding module, an environment module and a strategy module; the strategy module is trained by meta-learning.

[0046] In this way, the technical solution provided by this application can be used to train the model by combining meta-learning, input the search entities and query relationships in the path reasoning task into the model to infer and find the effective path between the search entities and query relationships, thereby improving the efficiency of knowledge graph reasoning and ensuring the accuracy of path reasoning.

[0047] In order to make the above-mentioned objects, features and advantages of the present application more obvious and easy to understand, preferred embodiments are specifically cited below and described in detail with reference to the attached drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0048] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required for use in the embodiments will be briefly introduced below. It should be understood that the following drawings only show certain embodiments of the present application and therefore should not be regarded as limiting the scope. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without paying creative work.

[0049] Figure 1 A flow chart of a path reasoning method based on a knowledge graph provided in an embodiment of the present application is shown;

[0050] Figure 2 A flowchart of another path reasoning method based on a knowledge graph provided in an embodiment of the present application is shown;

[0051] Figure 3 A schematic diagram of a path reasoning process provided by an embodiment of the present application is shown;

[0052] Figure 4 One of the structural diagrams of a path reasoning device based on a knowledge graph provided in an embodiment of the present application is shown;

[0053] Figure 5 The second structural diagram of a path reasoning device based on a knowledge graph provided in an embodiment of the present application is shown;

[0054] Figure 6 A schematic structural diagram of an electronic device provided in an embodiment of the present application is shown. DETAILED DESCRIPTION

[0055] To make the purpose, technical scheme and advantages of the embodiments of the present application clearer, the technical scheme in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. It should be understood that the drawings in the present application only serve the purpose of explanation and description and are not used to limit the scope of protection of the present application. In addition, it should be understood that the schematic drawings are not drawn in real proportion. The flowchart used in this application shows the operations implemented according to some embodiments of the present application. It should be understood that the operations of the flowchart can be implemented out of sequence, and the steps without logical context can be reversed in order or implemented simultaneously. In addition, those skilled in the art, under the guidance of the content of the present application, can add one or more other operations to the flowchart, or remove one or more operations from the flowchart.

[0056] In addition, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments. The components of the embodiments of the present application described and shown in the drawings here can be arranged and designed in various configurations. Therefore, the following detailed description of the embodiments of the present application provided in the drawings is not intended to limit the scope of the application claimed for protection, but merely represents the selected embodiments of the present application. Based on the embodiments of the present application, all other embodiments obtained by those skilled in the art without making creative work belong to the scope of protection of the present application.

[0057] In order to enable those skilled in the art to use the contents of this application, the following implementation method is given in combination with the specific application scenario "path reasoning based on knowledge graph". For those skilled in the art, the general principles defined here can be applied to other embodiments and application scenarios without departing from the spirit and scope of this application.

[0058] The following methods, devices, electronic devices or computer-readable storage media of the embodiments of the present application can be applied to any scenario that requires path reasoning based on a knowledge graph. The embodiments of the present application are not limited to specific application scenarios. Any solution using a path reasoning method, device, electronic device and storage medium based on a knowledge graph provided by the embodiments of the present application is within the protection scope of the present application.

[0059] It is worth noting that the scale of information data in the current network is huge, and the data content and storage format vary, which makes it difficult to perform subsequent analysis and processing. To this end, the knowledge graph stores data in the form of a graph structure, which solves the problem of representing heterogeneous information in industry data applications. However, in some specific scenarios, the data in the knowledge graph has the characteristics of strong privacy and sparse relationships, making it impossible to restore the data in specific scenarios. For downstream application tasks, using a knowledge graph with noise and insufficient data as the basis for application model training will cause deviations in the model training results, resulting in the failure of the application task. The task of performing knowledge graph reasoning under the condition of small sample data needs to be solved urgently.

[0060] At present, in the reasoning process of traditional knowledge graphs, manual mining of reasoning rules is required. However, under normal circumstances, it is difficult to exhaust all the knowledge data in the knowledge graph, and this method consumes a lot of manpower costs while still failing to guarantee the reliability of the rules and the quality of reasoning. Therefore, how to reason about knowledge graph paths has become an urgent problem to be solved.

[0061] Based on this, the present application proposes a path reasoning method, device, electronic device and medium based on a knowledge graph, wherein the path reasoning method includes: obtaining the search entities and query relationships in the path reasoning task; inputting the search entities and query relationships in the path reasoning task into a path reasoning model trained in the knowledge graph to perform path reasoning, and obtaining the path reasoning result corresponding to the path reasoning task in the knowledge graph; wherein the path reasoning model includes a knowledge embedding module, an environment module and a strategy module; the strategy module is trained by meta-learning.

[0062] In this way, the technical solution provided by this application can be used to train the model by combining meta-learning, input the search entities and query relationships in the path reasoning task into the model to infer and find the effective path between the search entities and query relationships, thereby improving the efficiency of knowledge graph reasoning and ensuring the accuracy of path reasoning.

[0063] To facilitate the understanding of the present application, the technical solution provided by the present application will be described in detail below in conjunction with specific embodiments.

[0064] See also Figure 1 , Figure 1 A flow chart of a path reasoning method based on a knowledge graph provided in an embodiment of the present application, such as Figure 1 As shown in , the path reasoning method includes:

[0065] S101, obtaining search entities and query relations in the path reasoning task;

[0066] As an example, this embodiment can realize knowledge graph-related applications such as knowledge question and answer and knowledge completion. For example, a path reasoning task may be: "Where is Zhang San's birthplace", where the search entity is: "Zhang San" and the query relationship is: "birthplace", and Zhang San's birthplace needs to be inferred based on the search entity and the query relationship.

[0067] S102. Input the search entities and query relations in the path reasoning task into the path reasoning model trained in the knowledge graph to perform path reasoning, and obtain the path reasoning result corresponding to the path reasoning task in the knowledge graph.

[0068] In this step, the path reasoning model includes a knowledge embedding module, an environment module and a strategy module; the strategy module is trained through meta-learning.

[0069] Here, a trained path reasoning model is obtained through reinforcement learning. The process of reinforcement learning is a learning process of modeling dynamic systems. Through interaction with the environment, the neural network model is continuously learned and updated to realize the autonomous reasoning function of the model and complete the automated reasoning task of the knowledge graph. In the process of knowledge graph reasoning and application, it is necessary to use knowledge representation methods to represent knowledge information as symbols and languages ​​that computers can understand, so as to support machines to simulate human ideas to interpret and reason about facts. Knowledge graph embedding technology can be used to convert knowledge graph entity and relationship information into vector representations in vector space, so as to realize knowledge reasoning by representing relationship link information in low-dimensional form. When performing path reasoning in the knowledge graph, the knowledge graph embedding technology is used to represent elements such as entities and relationships in the knowledge graph in the form of vectors in a low-dimensional vector space. According to the characteristics of vector space spatial translation, the representation of a relationship between different entities is described, so as to maximize the retention of element feature information in the knowledge graph. Traditional machine learning often relies on large-scale data set training for specific tasks to adjust parameters, but the training effect is not good when processing tasks with fewer samples. The learning task can be completed at a high speed by extracting the features of small sample data for training through meta-learning technology. Meta-learning is a machine learning paradigm with generalization and adaptability. In the task decision-making process of knowledge graph reasoning, meta-learning can be used to train on related tasks in multiple data sets to obtain an optimized parameter, provide a better initial parameter for the strategy module of the path reasoning model, and accelerate the model training process.

[0070] It should be noted that before the application process, that is, before executing step S102, a trained path reasoning model needs to be obtained, wherein the strategy module in the path reasoning model is mainly trained. Here, the strategy module in the path reasoning model is trained through the following steps:

[0071] 1. Obtain a sample reasoning task, and randomly sample the triple vector information stored in the action space of the environment module according to a preset ratio based on the sample reasoning task to obtain a support set and a verification set;

[0072] It should be noted that the triplet vector information is stored in the action space of the environment module through the following steps:

[0073] 1) Obtain a sample data set consisting of multiple triples;

[0074] In this step, each triple includes a head entity, a corresponding relationship, and a tail entity of a fact; a certain number of triples can be extracted from the general knowledge base to form a small sample data set.

[0075] 2) Through the knowledge embedding module in the path reasoning model, each triplet is embedded into the complex space in the form of a rotation vector, and self-adversarial negative sampling training is performed to obtain the triplet vector information corresponding to each triplet after embedding;

[0076] In this step, firstly, the knowledge embedding module can be used to introduce external knowledge information to enrich and strengthen the knowledge semantic representation of the path reasoning model; secondly, the triple information containing entities and relations is defined in the form of rotation vectors and embedded into the complex vector space. head ,r,e tail ), where the tail entity vector e tail Represented as the head entity vector e head A vector transferred along the rotation direction of the relationship vector r; Here, the knowledge embedding module evaluates the score of the relationship representation by the distance between the entity and the relationship. The calculation formula between the entity vector and the relationship vector can be expressed as Among them, r′ is the bias of the relationship vector, Represents the Hadamard product. The knowledge embedding module performs self-adversarial negative sampling training on negative triple samples according to the current embedding method, takes the sample distribution probability as the negative sample, adopts a similar self-adversarial negative sampling loss function for optimization, trains the knowledge embedding module, and outputs triple vector information.

[0077] 3) Input each triple vector information into the action space of the environment module, generate the entity transfer probability corresponding to each triple vector information, and store each triple vector information and the entity transfer probability corresponding to each triple vector information in the action space of the environment module.

[0078] In this step, the embedded triple vector information is input into the action space of the path reasoning model, and the entity transition probability is generated and stored in the action space, which is applied to the subsequent path search prediction. When making each prediction step, the relationship and the corresponding tail entity are found in the action space, as well as the probability of selecting the relationship (entity transition probability).

[0079] 2. Inputting the head entity vector and the corresponding relationship vector of each triplet vector information in the support set into the strategy module for path search to obtain a first loss function;

[0080] In this step, the model-independent meta-learning model provides the reinforcement learning model (path reasoning model) with the initial parameters and update strategy of the policy update function. r Randomly sample the triple data set and divide it into the support set D S and validation set D Q Two parts. First, based on the support set D S Train the strategy module and get the first loss function

[0081] 3. Based on the first loss function, update the module parameters in the strategy module, continue to perform path search for the next sample reasoning task to update the module parameters in the strategy module, until the first loss function converges, and obtain an updated strategy module;

[0082] In this step, the sample reasoning task T r Perform meta-learning, the strategy module is based on the support set D S Search the path and update the parameter θ r ′, the update function is The learning rate α represents the size of a single gradient step, θ r ′ represents the module parameter in the strategy module, θ represents the path selection parameter in the strategy module, Represents the gradient operator.

[0083] 4. Inputting the head entity vector and the corresponding relationship vector of each triplet vector information in the verification set into the strategy module for path search to obtain a second loss function;

[0084] 5. Based on the second loss function, the path selection parameters in the strategy module are updated, and the path search for the next sample reasoning task is continued to update the path selection parameters in the strategy module until the second loss function converges to obtain a trained strategy module.

[0085] In steps 4 and 5, S After learning the specific parameters in DQ The gradient calculated by evaluation is used to update the path selection parameter θ of the policy network, and the update function is The meta-learning rate β represents the step size of the meta-strategy update, is the second loss function; the path search parameters of the strategy module are updated by batch executing multiple sample reasoning tasks until the loss function converges to obtain a trained strategy module.

[0086] In step S102, the search entity and query relationship in the path reasoning task are input into the path reasoning model to execute the path reasoning task. Path information prediction is performed based on the path reasoning task information obtained in step S101 to guide the path decision of the path reasoning model. Figure 2 , Figure 2 A flowchart of another path reasoning method based on knowledge graph provided in an embodiment of the present application is shown in FIG. Figure 2 As shown in , the search entity and query relationship in the path reasoning task are input into the path reasoning model trained in the knowledge graph to perform path reasoning, and the steps of obtaining the path reasoning result corresponding to the path reasoning task in the knowledge graph include:

[0087] S201, inputting the search entity and query relationship in the path reasoning task into a knowledge embedding module, and determining a search entity vector corresponding to the search entity and a query relationship vector corresponding to the query relationship;

[0088] In this step, the search entities and query relations of the path reasoning task are input into the knowledge embedding module to obtain the corresponding vector representation.

[0089] S202, inputting the search entity vector and the query relationship vector into the action space of the environment module to determine the environment information at the current moment;

[0090] It should be noted that the step of inputting the search entity vector and the query relationship vector into the action space of the environment module to determine the environment information at the current moment includes:

[0091] S2021, inputting the search entity vector and the query relationship vector into the action space of the environment module, taking the search entity vector as the entity vector at the current moment, and determining the tail entity vector corresponding to the entity vector at the current moment and the entity transition probability corresponding to each tail entity vector from each triple vector information pre-stored in the action space;

[0092] In this step, each triple vector information is composed of a head entity vector, a corresponding relationship vector, and a tail entity vector; the path reasoning task is based on the query triple (e start , rquery , e end ) in the query relation r query and search entity e start , a heuristic beam search is performed on the knowledge graph represented by the knowledge embedding module. When expanding the path search information at each step, nodes (relationships and tail entities) with relatively low entity transfer probabilities are removed, and only nodes with higher probabilities are retained.

[0093] S2022, determining a set of tail entity vectors corresponding to entity transition probabilities greater than a preset probability threshold as a tail entity probability vector;

[0094] In this step, the tail entity probability vector obtained by path search is expressed as p∈[p1, p2, …, p i ], the i-th dimension value of p represents the corresponding entity e i is the probability of the correct tail entity.

[0095] S2023, selecting a tail entity vector from the tail entity probability vector according to a sampling strategy as a predicted tail entity vector;

[0096] In this step, the sampling strategy can be a beam search. Sample the entities according to the probability part p to generate prediction information e p , e p The predicted tail entity for the query path (predicted tail entity vector).

[0097] S2024, obtaining the historical reasoning information vector at the current moment from the long short-term memory network of the path reasoning model;

[0098] In this step, the long short-term memory network LSTM stores the historical reasoning information vector, and the historical reasoning information vector in the initial state is defined as h0 = LSTM (0, a1); the LSTM network is used to store the relationship r in the current action t and the corresponding entity e t Embedded information a t =[r t+1 ;e t+1 ] and historical reasoning information vector h t Linked together, the historical reasoning information vector h at the next moment can be output t+1 , the linking process formula is h t+1 =LSTM(h t , a t ).

[0099] S2025. Concatenate the predicted tail entity vector, the query relationship vector, the entity vector at the current moment, and the historical reasoning information vector at the current moment to obtain the environmental information at the current moment.

[0100] In this step, after the query path prediction information is completed, the path information obtained by beam search is fused with the information in the knowledge graph environment, that is, the prediction information e p (prediction tail entity vector) is attached to the state of the path reasoning model environment module, and together constitutes the current environment information s during the path reasoning task training process t =[e p ; r p ;e t ;h t ],e p is the prediction information (prediction tail entity vector) for guiding path reasoning, r q is the query relation r of the knowledge reasoning task query (query relation vector), e t is the entity information selected at the current moment (the entity vector at the current moment), h t is the historical reasoning information of the path under the current state (the historical reasoning information vector at the current moment).

[0101] S203: Input the current environmental information into the strategy module to obtain the path reasoning result corresponding to the path reasoning task in the knowledge graph.

[0102] It should be noted that the step of inputting the current environmental information into the strategy module to obtain the path reasoning result corresponding to the path reasoning task in the knowledge graph includes:

[0103] S2031, inputting the current environmental information into the policy module, and determining a policy function of the policy module based on the environmental information;

[0104] In this step, the current environmental information is input into the strategy module, and the final score of the given relationship and path is output by combining multiple paths. The strategy function is π θ (a t |s t )=σ(A t (W1ReLU(W2s t ))), π θ (a t |s t ) is the probability distribution of all actions at time t, σ represents the normalized exponential function softmax, W1 and W2 are two linear neural networks, A t is the environmental state at time t.

[0105] S2032, obtaining multiple search paths corresponding to the entity vectors at the current moment in the environment information in the knowledge graph, and determining the relationship probability of each search path at the current moment through the strategy function;

[0106] In this step, when the relationship probability of each search path at the current moment is determined by the strategy function, when a state s is observed t When the probability value a of each action is calculated through the strategy function t , and then do random sampling to select actions.

[0107] S2033, for each search path, based on the relationship probability of the search path at the current moment, determining an evaluation score of the search path;

[0108] In this step, the path search parameters of the policy network can be updated through the Monte Carlo policy gradient REINFORCE. The update formula of the path search parameters is: The relationship r corresponds to the generated path, J(θ) is the total score of a reinforcement learning level, and a reward is given for each completed path selection. For the current entity e s Through the relationship r to search entity e o expectations, are all paths in the knowledge graph, is the action expectation, R(s T ∣e s , r) is the reward at the current moment.

[0109] Here, the expected value in the search path can be calculated by random sampling to obtain an empirical average cumulative reward as the evaluation score of the search path.

[0110] S2034. Among the evaluation scores of each search path, determine the search path corresponding to the largest evaluation score as the inference path;

[0111] S2035, performing a similarity comparison between the corresponding relationship vector in the reasoning path and the query relationship vector;

[0112] S2036. If the similarity is greater than a preset similarity, the tail entity vector corresponding to the corresponding relationship vector in the reasoning path is determined as the path reasoning result corresponding to the path reasoning task in the knowledge graph.

[0113] In this step, the reasoning path and the query path information are compared. If the corresponding relationship vector is similar to the query relationship vector, the reasoning result is output as the tail entity vector corresponding to the corresponding relationship vector. Otherwise, the following step (1) is executed to search for the next entity.

[0114] It should be noted that the path reasoning method also includes:

[0115] (1) If the similarity is not greater than the preset similarity, the tail entity vector corresponding to the next corresponding relationship vector of the entity vector at the current moment is updated to the entity vector at the current moment in the action space to perform the next path reasoning until all corresponding relationship vectors of the entity vector at the current moment are traversed.

[0116] For example, see Figure 3 , Figure 3 A schematic diagram of a path reasoning process provided in an embodiment of the present application is shown in FIG. Figure 3 As shown in the figure, the triple information required for the path reasoning task is extracted from a subset (small sample data set) of general knowledge graph data sets such as Freebase and NELL; the triple information of the knowledge graph is embedded into the complex space in the form of a rotation vector, and self-adversarial negative sampling training is performed to optimize the knowledge embedding module. The calculation formula between the entity vector and the relationship vector is: The embedded triple information is stored in the action space of the environment module, sampled, and an independent meta-learning model (environment module and policy module) is constructed. Multiple reasoning tasks are executed in batches for training to obtain a well-fitting path search parameter θ, which supports the path reasoning model to quickly adapt to each relational reasoning task during training; the head entity and query relationship information of the path reasoning task are input into the knowledge embedding module to obtain the head entity vector and the query relationship vector, and a beam search is performed in the action space to obtain prediction information. The prediction information of the path is dynamically added to the state in the environment module, and the prediction information is sent to the state to generate environment information. The environment information is used to guide the policy module to select the correct path for search, avoiding In order to avoid errors in reasoning results due to missing paths or unclear relationships, the path reasoning model strategy module receives the environmental information in the state to perform path reasoning, obtains reasoning information, and sends the reasoning information to the long short-term memory network to determine the historical reasoning information. The long short-term memory network sends the historical reasoning information to the state to update the environmental information, and continues the next reasoning until the similarity between the relationship vector in the reasoning information and the query relationship vector is greater than the preset similarity, and the target entity in the reasoning information is output as the path reasoning result. The strategy module in this embodiment performs decision reasoning through continuous interaction with the environmental information, and finally obtains the relationship path and target entity corresponding to the head entity and the query relationship, thereby completing the path reasoning task.

[0117] In summary, the data initialization in this embodiment can adjust the dimension of vector initialization according to the semantic complexity of the head and tail entities and relationships, thereby improving the efficiency and accuracy of knowledge embedding module training; the path reasoning process is represented as a Markov decision process, and the highest-scoring relationship paths and target entities are automatically learned and searched through the reinforcement learning model, thereby improving the computational efficiency and scalability of path reasoning; LSTM is introduced into the reinforcement learning strategy module to combine the potential semantic information in the knowledge graph relationship and enhance the ability of entities and relationships to represent information in the path reasoning process; the strategy module is guided by predictive information to optimize the path search process; the existing strategy module is optimized by obtaining better initialization parameters through meta-learning update strategies, so that the reinforcement learning model (path reasoning model) can quickly adapt to the relationship information in the knowledge sparse environment, and provide good training features for the knowledge graph reasoning task.

[0118] An embodiment of the present application provides a path reasoning method based on a knowledge graph, and the path reasoning method includes: obtaining search entities and query relationships in a path reasoning task; inputting the search entities and query relationships in the path reasoning task into a path reasoning model trained in the knowledge graph to perform path reasoning, and obtaining the path reasoning result corresponding to the path reasoning task in the knowledge graph; wherein the path reasoning model includes a knowledge embedding module, an environment module and a strategy module; the strategy module is trained by meta-learning.

[0119] In this way, the technical solution provided by this application can be used to train the model by combining meta-learning, input the search entities and query relationships in the path reasoning task into the model to infer and find the effective path between the search entities and query relationships, thereby improving the efficiency of knowledge graph reasoning and ensuring the accuracy of path reasoning.

[0120] Based on the same application concept, the embodiment of the present application also provides a path reasoning device based on a knowledge graph corresponding to the path reasoning method based on a knowledge graph provided in the above embodiment. Since the principle of solving the problem by the device in the embodiment of the present application is similar to the path reasoning method based on a knowledge graph in the above embodiment of the present application, the implementation of the device can refer to the implementation of the method, and the repeated parts will not be repeated.

[0121] See also Figure 4 , Figure 5 , Figure 4 This is one of the structural diagrams of a path reasoning device based on a knowledge graph provided in an embodiment of the present application. Figure 5 This is a second structural diagram of a path reasoning device based on a knowledge graph provided in an embodiment of the present application. Figure 4 As shown in , the path reasoning device 410 includes:

[0122] An acquisition module 411 is used to acquire search entities and query relations in a path reasoning task;

[0123] The reasoning module 412 is used to input the search entities and query relations in the path reasoning task into the path reasoning model trained in the knowledge graph to perform path reasoning, and obtain the path reasoning result corresponding to the path reasoning task in the knowledge graph; wherein the path reasoning model includes a knowledge embedding module, an environment module and a strategy module; the strategy module is trained by meta-learning.

[0124] Optionally, the reasoning module 412 is specifically used for:

[0125] Inputting the search entity and the query relationship in the path reasoning task into the knowledge embedding module, determining the search entity vector corresponding to the search entity and the query relationship vector corresponding to the query relationship;

[0126] Inputting the search entity vector and the query relationship vector into the action space of the environment module to determine the environment information at the current moment;

[0127] The environmental information at the current moment is input into the strategy module to obtain the path reasoning result corresponding to the path reasoning task in the knowledge graph.

[0128] Optionally, when the reasoning module 412 is used to input the search entity vector and the query relationship vector into the action space of the environment module to determine the environment information at the current moment, the reasoning module 412 is specifically used to:

[0129] Input the search entity vector and the query relationship vector into the action space of the environment module, use the search entity vector as the entity vector at the current moment, and determine the tail entity vector corresponding to the entity vector at the current moment and the entity transition probability corresponding to each tail entity vector in each triple vector information pre-stored in the action space; wherein each triple vector information is composed of a head entity vector, a corresponding relationship vector and a tail entity vector;

[0130] Determine a set of tail entity vectors corresponding to entity transition probabilities greater than a preset probability threshold as a tail entity probability vector;

[0131] According to the sampling strategy, a tail entity vector is selected from the tail entity probability vector as a predicted tail entity vector;

[0132] Obtaining a historical reasoning information vector at a current moment from a long short-term memory network of the path reasoning model;

[0133] The predicted tail entity vector, the query relationship vector, the entity vector at the current moment, and the historical reasoning information vector at the current moment are concatenated to obtain the environmental information at the current moment.

[0134] Optional, such as Figure 5 As shown, the path reasoning device 410 further includes a training module 413, and the training module 413 is specifically used for:

[0135] Acquire a sample reasoning task, and randomly sample the triplet vector information stored in the action space of the environment module according to a preset ratio based on the sample reasoning task to obtain a support set and a verification set;

[0136] Inputting the head entity vector and the corresponding relationship vector of each triplet vector information in the support set into the strategy module for path search to obtain a first loss function;

[0137] Based on the first loss function, the module parameters in the strategy module are updated, and the path search for the next sample reasoning task is continued to update the module parameters in the strategy module until the first loss function converges to obtain an updated strategy module;

[0138] Inputting the head entity vector and the corresponding relationship vector of each triplet vector information in the verification set into the strategy module for path search to obtain a second loss function;

[0139] Based on the second loss function, the path selection parameters in the strategy module are updated, and the path search for the next sample reasoning task is continued to update the path selection parameters in the strategy module until the second loss function converges to obtain a trained strategy module.

[0140] Optionally, when the training module 413 is used to store the triplet vector information in the action space of the environment module, the training module 413 is specifically used to:

[0141] Obtain a sample data set consisting of multiple triples, wherein each triple includes a head entity, a corresponding relationship, and a tail entity of a fact;

[0142] Through the knowledge embedding module in the path reasoning model, each triplet is embedded into the complex space in the form of a rotation vector, and self-adversarial negative sampling training is performed to obtain triplet vector information corresponding to each triplet after embedding;

[0143] Each triplet vector information is input into the action space of the environment module, an entity transition probability corresponding to each triplet vector information is generated, and each triplet vector information and the entity transition probability corresponding to each triplet vector information are stored in the action space of the environment module.

[0144] Optionally, when the reasoning module 412 is used to input the current environmental information into the strategy module to obtain the path reasoning result corresponding to the path reasoning task in the knowledge graph, the reasoning module 412 is specifically used to:

[0145] Inputting the current environmental information into the strategy module, and determining a strategy function of the strategy module based on the environmental information;

[0146] Acquire multiple search paths corresponding to the entity vectors at the current moment in the environment information in the knowledge graph, and determine the relationship probability of each search path at the current moment through the strategy function;

[0147] For each search path, based on the relationship probability of the search path at the current moment, determine the evaluation score of the search path;

[0148] Among the evaluation scores of each search path, the search path corresponding to the evaluation score with the largest value is determined as the reasoning path;

[0149] Performing a similarity comparison between the corresponding relationship vector in the reasoning path and the query relationship vector;

[0150] If the similarity is greater than a preset similarity, the tail entity vector corresponding to the corresponding relationship vector in the reasoning path is determined as the path reasoning result corresponding to the path reasoning task in the knowledge graph.

[0151] Optionally, the reasoning module 412 is further used for:

[0152] If the similarity is not greater than the preset similarity, the tail entity vector corresponding to the next corresponding relationship vector of the entity vector at the current moment is updated to the entity vector at the current moment in the action space for the next path reasoning until all corresponding relationship vectors of the entity vector at the current moment are traversed.

[0153] An embodiment of the present application provides a path reasoning device based on a knowledge graph, and the path reasoning device includes: an acquisition module, used to obtain search entities and query relationships in a path reasoning task; a reasoning module, used to input the search entities and query relationships in the path reasoning task into a path reasoning model trained in the knowledge graph to perform path reasoning, and obtain the path reasoning result corresponding to the path reasoning task in the knowledge graph; wherein the path reasoning model includes a knowledge embedding module, an environment module and a strategy module; the strategy module is trained by meta-learning.

[0154] In this way, the technical solution provided by this application can be used to train the model by combining meta-learning, input the search entities and query relationships in the path reasoning task into the model to infer and find the effective path between the search entities and query relationships, thereby improving the efficiency of knowledge graph reasoning and ensuring the accuracy of path reasoning.

[0155] See also Figure 6 , Figure 6 This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present application. Figure 6 As shown in , the electronic device 600 includes a processor 610 , a memory 620 and a bus 630 .

[0156] The memory 620 stores machine-readable instructions executable by the processor 610. When the electronic device 600 is running, the processor 610 communicates with the memory 620 via the bus 630. When the machine-readable instructions are executed by the processor 610, the above-mentioned Figure 1 as well as Figure 2 The steps of the knowledge graph-based path reasoning method in the method embodiment shown, the specific implementation method can be found in the method embodiment, and will not be repeated here.

[0157] The present application also provides a computer-readable storage medium on which a computer program is stored. When the computer program is executed by a processor, the computer program can execute the above-mentioned Figure 1 as well as Figure 2 The steps of the knowledge graph-based path reasoning method in the method embodiment shown, the specific implementation method can be found in the method embodiment, and will not be repeated here.

[0158] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0159] In the several embodiments provided in the present application, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. The device embodiments described above are merely schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some communication interfaces, and the indirect coupling or communication connection of devices or units can be electrical, mechanical or other forms.

[0160] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0161] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.

[0162] If the functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a non-volatile computer-readable storage medium that is executable by a processor. Based on this understanding, the technical solution of the present application can essentially be embodied in the form of a software product, or in other words, the part that contributes to the prior art or the part of the technical solution. The computer software product is stored in a storage medium, including several instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.

[0163] Finally, it should be noted that the above-described embodiments are only specific implementation methods of the present application, which are used to illustrate the technical solutions of the present application, rather than to limit them. The protection scope of the present application is not limited thereto. Although the present application is described in detail with reference to the above-mentioned embodiments, ordinary technicians in the field should understand that any technician familiar with the technical field can still modify the technical solutions recorded in the above-mentioned embodiments within the technical scope disclosed in the present application, or can easily think of changes, or make equivalent replacements for some of the technical features therein; and these modifications, changes or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present application, and should be included in the protection scope of the present application. Therefore, the protection scope of the present application shall be based on the protection scope of the claims.

Claims

1. A path reasoning method based on knowledge graph, characterized in that: Knowledge graph-related applications used to implement knowledge question answering and knowledge completion; The path reasoning method comprises: Obtaining a search entity and a query relationship in a path reasoning task; the path reasoning task is to query a birthplace; Input the search entity and query relationship in the path reasoning task into the path reasoning model trained in the knowledge graph to perform path reasoning, and obtain the path reasoning result corresponding to the path reasoning task in the knowledge graph; The path reasoning model includes a knowledge embedding module, an environment module and a strategy module; the strategy module is trained by meta-learning; The step of inputting the search entity and query relationship in the path reasoning task into the path reasoning model trained in the knowledge graph to perform path reasoning, and obtaining the path reasoning result corresponding to the path reasoning task in the knowledge graph, includes: Inputting the search entity and the query relationship in the path reasoning task into the knowledge embedding module, determining the search entity vector corresponding to the search entity and the query relationship vector corresponding to the query relationship; Inputting the search entity vector and the query relationship vector into the action space of the environment module to determine the environment information at the current moment; Inputting the current environmental information into the strategy module to obtain the path reasoning result corresponding to the path reasoning task in the knowledge graph; The step of inputting the current environmental information into the strategy module to obtain the path reasoning result corresponding to the path reasoning task in the knowledge graph includes: Inputting the current environmental information into the strategy module, and determining a strategy function of the strategy module based on the environmental information; Among them, the determined strategy function is , for The probability distribution of all actions at the moment, represents the normalized exponential function softmax, and are two linear neural networks, for The state of the environment at any given moment; express The state of the moment; Acquire multiple search paths corresponding to the entity vectors at the current moment in the environment information in the knowledge graph, and determine the relationship probability of each search path at the current moment through the strategy function; For each search path, based on the relationship probability of the search path at the current moment, determine the evaluation score of the search path; Among the evaluation scores of each search path, the search path corresponding to the evaluation score with the largest value is determined as the reasoning path; The corresponding relationship vector in the reasoning path and the query relationship vector are compared for similarity; wherein the action space of the environment module pre-stores triple vector information; each triple vector information is composed of a head entity vector, a corresponding relationship vector and a tail entity vector; If the similarity is greater than a preset similarity, the tail entity vector corresponding to the corresponding relationship vector in the reasoning path is determined as the path reasoning result corresponding to the path reasoning task in the knowledge graph; The step of inputting the search entity vector and the query relationship vector into the action space of the environment module to determine the environment information at the current moment includes: The search entity vector and the query relationship vector are input into the action space of the environment module, the search entity vector is used as the entity vector at the current moment, and the tail entity vector corresponding to the entity vector at the current moment and the entity transfer probability corresponding to each tail entity vector in the tail entity vector corresponding to the entity vector at the current moment are determined in each triple vector information pre-stored in the action space; wherein the entity transfer probability corresponding to each tail entity vector refers to the probability of selecting the corresponding relationship vector from the entity vector at the current moment and transferring to the tail entity vector; Determine a set of tail entity vectors corresponding to entity transition probabilities greater than a preset probability threshold as a tail entity probability vector; According to the sampling strategy, a tail entity vector is selected from the tail entity probability vector as a predicted tail entity vector; Obtaining a historical reasoning information vector at a current moment from a long short-term memory network of the path reasoning model; The predicted tail entity vector, the query relationship vector, the entity vector at the current moment, and the historical reasoning information vector at the current moment are concatenated to obtain the environmental information at the current moment.

2. The path reasoning method according to claim 1, characterized in that: Train the policy module in the path reasoning model by following these steps: Acquire a sample reasoning task, and randomly sample the triple vector information stored in the action space of the environment module according to a preset ratio based on the sample reasoning task to obtain a support set and a verification set; Inputting the head entity vector and the corresponding relationship vector of each triplet vector information in the support set into the strategy module for path search to obtain a first loss function; Based on the first loss function, the module parameters in the strategy module are updated, and the path search for the next sample reasoning task is continued to update the module parameters in the strategy module until the first loss function converges to obtain an updated strategy module; Inputting the head entity vector and the corresponding relationship vector of each triplet vector information in the verification set into the strategy module for path search to obtain a second loss function; Based on the second loss function, the path selection parameters in the strategy module are updated, and the path search for the next sample reasoning task is continued to update the path selection parameters in the strategy module until the second loss function converges to obtain a trained strategy module.

3. The path reasoning method according to claim 2, characterized in that: The triplet vector information is stored in the action space of the environment module through the following steps: Obtain a sample data set consisting of multiple triples, wherein each triple includes a head entity, a corresponding relationship, and a tail entity of a fact; Through the knowledge embedding module in the path reasoning model, each triplet is embedded into the complex space in the form of a rotation vector, and self-adversarial negative sampling training is performed to obtain triplet vector information corresponding to each triplet after embedding; Each triple vector information is input into the action space of the environment module, an entity transition probability corresponding to each triple vector information is generated, and each triple vector information and the entity transition probability corresponding to each triple vector information are stored in the action space of the environment module.

4. The path reasoning method according to claim 1, characterized in that: The path reasoning method also includes: If the similarity is not greater than the preset similarity, the tail entity vector corresponding to the next corresponding relationship vector of the entity vector at the current moment is updated to the entity vector at the current moment in the action space for the next path reasoning until all corresponding relationship vectors of the entity vector at the current moment are traversed.

5. A path reasoning device based on knowledge graph, characterized in that: Knowledge graph-related applications used to implement knowledge question answering and knowledge completion; The path reasoning device comprises: An acquisition module, used to acquire search entities and query relations in a path reasoning task; the path reasoning task is to query birthplace; A reasoning module, used to input the search entity and query relationship in the path reasoning task into the path reasoning model trained in the knowledge graph to perform path reasoning, and obtain the path reasoning result corresponding to the path reasoning task in the knowledge graph; The path reasoning model includes a knowledge embedding module, an environment module and a strategy module; the strategy module is trained by meta-learning; The reasoning module is specifically used for: Inputting the search entity and the query relationship in the path reasoning task into the knowledge embedding module, determining the search entity vector corresponding to the search entity and the query relationship vector corresponding to the query relationship; Inputting the search entity vector and the query relationship vector into the action space of the environment module to determine the environment information at the current moment; Inputting the current environmental information into the strategy module to obtain the path reasoning result corresponding to the path reasoning task in the knowledge graph; When the reasoning module is used to input the current environmental information into the strategy module to obtain the path reasoning result corresponding to the path reasoning task in the knowledge graph, the reasoning module is specifically used to: Inputting the current environmental information into the strategy module, and determining a strategy function of the strategy module based on the environmental information; Among them, the determined strategy function is , for The probability distribution of all actions at the moment, represents the normalized exponential function softmax, and are two linear neural networks, for The state of the environment at any given moment; express The state of the moment; Acquire multiple search paths corresponding to the entity vectors at the current moment in the environment information in the knowledge graph, and determine the relationship probability of each search path at the current moment through the strategy function; For each search path, based on the relationship probability of the search path at the current moment, determine the evaluation score of the search path; Among the evaluation scores of each search path, the search path corresponding to the evaluation score with the largest value is determined as the reasoning path; The corresponding relationship vector in the reasoning path and the query relationship vector are compared for similarity; wherein the action space of the environment module pre-stores triple vector information; each triple vector information is composed of a head entity vector, a corresponding relationship vector and a tail entity vector; If the similarity is greater than a preset similarity, the tail entity vector corresponding to the corresponding relationship vector in the reasoning path is determined as the path reasoning result corresponding to the path reasoning task in the knowledge graph; When the reasoning module is used to input the search entity vector and the query relationship vector into the action space of the environment module to determine the environment information at the current moment, the reasoning module is specifically used to: The search entity vector and the query relationship vector are input into the action space of the environment module, the search entity vector is used as the entity vector at the current moment, and the tail entity vector corresponding to the entity vector at the current moment and the entity transfer probability corresponding to each tail entity vector in the tail entity vector corresponding to the entity vector at the current moment are determined in each triple vector information pre-stored in the action space; wherein the entity transfer probability corresponding to each tail entity vector refers to the probability of selecting the corresponding relationship vector from the entity vector at the current moment and transferring to the tail entity vector; Determine a set of tail entity vectors corresponding to entity transition probabilities greater than a preset probability threshold as a tail entity probability vector; According to the sampling strategy, a tail entity vector is selected from the tail entity probability vector as a predicted tail entity vector; Obtaining a historical reasoning information vector at a current moment from a long short-term memory network of the path reasoning model; The predicted tail entity vector, the query relationship vector, the entity vector at the current moment, and the historical reasoning information vector at the current moment are concatenated to obtain the environmental information at the current moment.

6. An electronic device, characterized in that: include: A processor, a memory and a bus, wherein the memory stores machine-readable instructions executable by the processor. When the electronic device is running, the processor and the memory communicate through the bus, and the machine-readable instructions are executed by the processor to execute the steps of the path reasoning method based on the knowledge graph as described in any one of claims 1 to 4.

7. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the path reasoning method based on the knowledge graph as described in any one of claims 1 to 4 are executed.

Images