A knowledge reasoning method, device and equipment of a knowledge graph and a storage medium

Abstract

An embodiment of the present invention discloses a knowledge reasoning method for a knowledge graph, comprising: obtaining an initial knowledge graph, and generating alternative rules based on the initial knowledge graph; judging the alternative rules to determine qualified rules with a confidence level greater than a set threshold; reasoning and completing the initial knowledge graph based on the qualified rules, obtaining new nodes and corresponding relationships, and adding them to the graph knowledge set. The knowledge reasoning method for the knowledge graph provided by the embodiment of the present invention combines a rule reasoning method with a graph neural network reasoning method to form a hybrid reasoning framework based on a generative adversarial model, designs a hybrid reasoning framework to implement hybrid reasoning, and uses a link prediction method based on a hierarchical structure to predict nodes and relationships, combining the advantages of different reasoning methods, thereby improving the generalization ability and computational efficiency of knowledge reasoning, and ensuring the accuracy and interpretability of the reasoning results.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of data processing, and in particular to a knowledge reasoning method and device for a knowledge graph, an equipment and a storage medium. BACKGROUND

[0002] Knowledge reasoning refers to a process of inferring unknown facts or knowledge based on known facts or knowledge, and the task of knowledge graph reasoning is to reason new knowledge or identify incorrect knowledge according to existing knowledge in a knowledge graph. As a main method for human problem solving, knowledge reasoning has always been concerned. Generally speaking, knowledge reasoning includes two main methods: one is a traditional knowledge reasoning mode based on rules; and the other is a reasoning method based on a graph neural network.

[0003] The traditional rule-based reasoning has good accuracy, but usually requires a large amount of expert cost to construct business rules, and the constructed business rules are usually only applicable to specific scenarios and have weak generalization ability; and in a large-scale multi-modal knowledge graph scenario, its efficiency is also prone to become a bottleneck. The single-step reasoning based on the neural network attempts to model the knowledge graph fact tuple by using the powerful learning ability of the neural network, to obtain good reasoning ability and generalization ability, but the accuracy and rationality are difficult to guarantee, and the inherent explainability problem of the neural network still exists in the application of the knowledge graph, and how to properly explain the reasoning ability of the neural network is a big problem. SUMMARY

[0004] The embodiments of the present application provide a knowledge reasoning method, device, equipment and storage medium for a knowledge graph, which realizes the knowledge reasoning with good generalization ability, calculation efficiency and accuracy.

[0005] In a first aspect, the embodiments of the present application provide a knowledge reasoning method for a knowledge graph, comprising:

[0006] An initial knowledge graph is acquired, and candidate rules are generated according to the initial knowledge graph;

[0007] The candidate rules are discriminated to determine qualified rules with a confidence greater than a set threshold;

[0008] The initial knowledge graph is reasoned and completed according to the qualified rules, new nodes and corresponding relationships are acquired, and are added to a knowledge graph set.

[0009] Further, the candidate rules are generated according to the initial knowledge graph, comprising:

[0010] Images and texts in the initial knowledge graph are extracted and represented as nodes;

[0011] An entity vector and a relation vector corresponding to the node are initialized respectively to generate an entity embedding vector and a relation embedding vector;

[0012] Each triple sequence in the entity embedding vector and the relation embedding vector is extracted and input into a long short-term memory model to obtain the candidate rule; the triple sequence includes two entity embedding vectors and a corresponding relation embedding vector.

[0013] Further, images and texts in the initial knowledge graph are extracted and represented as nodes, including:

[0014] The images in the initial knowledge graph are extracted, and image descriptions are generated according to the images;

[0015] The texts in the initial knowledge graph are extracted, and the texts and image descriptions are semantically analyzed, and the entities and attributes obtained by the analysis are determined as the nodes.

[0016] Further, the candidate rules are discriminated to determine qualified rules with a confidence greater than a set threshold, including:

[0017] The initial knowledge graph is subjected to rule reasoning to determine new rules;

[0018] The confidence of each candidate rule is determined according to the new rules, and the candidate rules with a confidence greater than the set threshold are determined as the qualified rules.

[0019] Further, the initial knowledge graph is subjected to rule reasoning to determine new rules, including:

[0020] The rule conversion logic corresponding to the initial knowledge graph is determined;

[0021] The new rules are learned from the initial knowledge graph according to the rule conversion logic.

[0022] Further, the confidence of the candidate rule is determined according to the new rule, including:

[0023] A discriminator is generated according to a rule reasoning algorithm and a learning reasoning algorithm, and the new rule is determined as a true value in the discriminator;

[0024] The candidate rules are sequentially input into the discriminator, and the output of the discriminator is the confidence of each candidate rule.

[0025] Further, the initial knowledge graph is reasoned and completed according to the qualified rule to obtain new nodes and corresponding relations, including:

[0026] According to the qualified rule, a subgraph with the same relation in the initial knowledge graph is extracted;

[0027] input the initial knowledge graph and the subgraph into a link prediction model, and output the predicted new node and corresponding relationship.

[0028] In a second aspect, the embodiments of the present application further provide a knowledge reasoning device for a knowledge graph, comprising:

[0029] an alternative rule generation module configured to acquire an initial knowledge graph and generate alternative rules according to the initial knowledge graph;

[0030] a qualified rule determination module configured to determine qualified rules with a confidence greater than a set threshold by discriminating the alternative rules;

[0031] a reasoning module configured to complete the initial knowledge graph according to the qualified rules, acquire new nodes and corresponding relationships, and add them to the knowledge graph set.

[0032] Optionally, the alternative rule generation module is further configured to:

[0033] extract images and texts in the initial knowledge graph and represent them as nodes;

[0034] initialize entity vectors and relationship vectors corresponding to the nodes respectively to generate entity embedding vectors and relationship embedding vectors;

[0035] extract each triple sequence in the entity embedding vectors and the relationship embedding vectors, input a long short-term memory model, and obtain the alternative rules; the triple sequence includes two entity embedding vectors and a corresponding relationship embedding vector.

[0036] Optionally, the alternative rule generation module is further configured to:

[0037] extract images in the initial knowledge graph, generate image descriptions according to the images;

[0038] extract texts in the initial knowledge graph, perform semantic analysis on the texts and the image descriptions, and determine entities and attributes obtained by the analysis as the nodes.

[0039] Optionally, the qualified rule determination module is further configured to:

[0040] perform rule reasoning on the initial knowledge graph to determine new rules;

[0041] determine the confidence of each alternative rule according to the new rules, and determine alternative rules with a confidence greater than the set threshold as the qualified rules.

[0042] Optionally, the qualified rule determination module is further configured to:

[0043] determine rule conversion logic corresponding to the initial knowledge graph;

[0044] learn the new rule from the initial knowledge graph according to the rule conversion logic.

[0045] Optionally, the qualified rule determination module is further configured to:

[0046] generate a discriminator according to the rule reasoning algorithm and the learning reasoning algorithm, and determine the new rule as a true value in the discriminator;

[0047] input the alternative rules into the discriminator in turn, and the output of the discriminator is a confidence degree corresponding to each alternative rule.

[0048] Optionally, the reasoning module is further configured to:

[0049] extract a subgraph having the same relationship in the initial knowledge graph according to the qualified rule;

[0050] input the initial knowledge graph and the subgraph into a link prediction model, and output the predicted new node and corresponding relationship.

[0051] In a third aspect, an embodiment of the present application further provides a knowledge reasoning computer device for a knowledge graph, comprising:

[0052] a memory, a processor, and a computer program stored in the memory and executable on the processor, and the processor implements the knowledge reasoning method for a knowledge graph according to any of the embodiments of the present application when executing the program.

[0053] In a fourth aspect, an embodiment of the present application further provides a knowledge reasoning storage medium for a knowledge graph, and the storage medium has a computer program stored thereon, and the program is executable by a processing device to implement the knowledge reasoning method for a knowledge graph according to any of the embodiments of the present application.

[0054] The embodiment of the present application first acquires an initial knowledge graph, and generates alternative rules according to the initial knowledge graph; then discriminates the alternative rules to determine qualified rules with a confidence degree greater than a set threshold; and finally reasons and completes the initial knowledge graph according to the qualified rules, acquires new nodes and corresponding relationships, and adds them to the knowledge graph set. The knowledge reasoning method for a knowledge graph provided by the embodiment of the present application combines the rule reasoning method with the graph neural network reasoning method, uses the graph neural network as a generator to generate alternative rules, and inputs the alternative rules into a discriminator constructed by the rule reasoning method to discriminate, thereby forming a hybrid reasoning framework based on a generative adversarial model. The hybrid reasoning framework implements hybrid reasoning, and uses a link prediction method based on a hierarchical structure to predict nodes and relationships, which combines the advantages of different reasoning methods, improves the generalization ability and computational efficiency of knowledge reasoning, and guarantees the accuracy and interpretability of the reasoning result. BRIEF DESCRIPTION OF DRAWINGS

[0055] Figure 1 is a knowledge reasoning method flowchart of a knowledge graph in embodiment one of the present application;

[0056] Figure 2 is a knowledge reasoning method flowchart of a knowledge graph in embodiment two of the present application;

[0057] Figure 3 is a graph neural network reasoning process schematic diagram of image data in embodiment two of the present application;

[0058] Figure 4 is a hybrid reasoning process schematic diagram based on a generative adversarial model in embodiment two of the present application;

[0059] Figure 5a is a knowledge graph subgraph schematic diagram in embodiment two of the present application;

[0060] Figure 5b is a subgraph hierarchical structure schematic diagram in embodiment two of the present application;

[0061] Figure 6 is a knowledge reasoning device structure schematic diagram of a knowledge graph in embodiment three of the present application;

[0062] Figure 7 is a structure schematic diagram of a computer device in embodiment four of the present application. DETAILED DESCRIPTION

[0063] The present application will be further described below in conjunction with the drawings and embodiments. It can be understood that the specific embodiments described herein are only used to explain the present application, and not to limit the present application. In addition, it should be noted that, for the convenience of description, only the parts related to the present application are shown in the drawings, not all the structures.

[0064] Embodiment one

[0065] Figure 1 A knowledge reasoning method flowchart of a knowledge graph is provided in embodiment one of the present application, the present embodiment can be applicable to the case of knowledge reasoning according to a knowledge graph, the method can be executed by a knowledge reasoning device of the knowledge graph, the device can be composed of hardware and / or software, and generally can be integrated in a device having the knowledge reasoning function of the knowledge graph, the device can be an electronic device such as a server or a server cluster. As shown in the figure, the specific steps include the following steps: Figure 1

[0066] Step 110, obtaining an initial knowledge graph, and generating candidate rules according to the initial knowledge graph.

[0067] ​The knowledge graph is a series of various graphs showing the development process and structural relationship of knowledge, and uses visualization technology to describe knowledge resources and their carriers, and to mine, analyze, construct, draw, and display knowledge and their mutual relationships. The initial knowledge graph is a raw knowledge graph without reasoning, and new knowledge can be obtained by reasoning the initial knowledge graph through the method in the embodiment. The alternative rule is a new rule learned from the entity relationship information of the old rule according to the initial knowledge graph.

[0068] In the embodiment, the knowledge graph can contain various forms of data such as image data and text data, and such a knowledge graph can be referred to as a multi-modal knowledge graph. The knowledge graph can include multiple nodes and relationships, each node can correspond to an entity, and entities can be connected by one or more paths, and each path corresponds to a relationship. The way to generate alternative rules from the initial knowledge graph can be to obtain multi-hop connection paths between entity pairs from the multi-modal knowledge graph, and to obtain a unified overall representation of the multiple paths between the entity pairs through an aggregation function, thereby generating new rules, i.e., alternative rules.

[0069] Optionally, the influence of different edge relationships in the knowledge graph structure on the nodes can be processed by using a set algorithm and model, the link prediction and entity classification of the knowledge graph are performed, the relationship between nodes is learned from the entity relationship information of the old rule of the initial knowledge graph, and finally the generation of the alternative rule is realized.

[0070] Step 120, discriminating the alternative rule to determine the qualified rule with a confidence greater than a set threshold.

[0071] In the embodiment, after the alternative rule is generated, the confidence of the alternative rule can be calculated to evaluate the quality of the alternative rule, and the rule with good quality is output as a qualified rule.

[0072] Optionally, a discriminator can be established by using a logical rule-based reasoning method, an automatic reasoning rule conversion logic of the rule can be designed according to the data characteristics of the initial knowledge graph, and an automatic rule reasoning technology based on a logic multi-modal graph can be used to learn rules with high confidence from the alternative rule.

[0073] The logical rule-based reasoning process can be as follows:

[0074] For example, assume that the initial knowledge graph contains nodes Alice and Bob, and the relationship between the nodes is has-son(Alice, Bob). For the conversion relationship between the relationships "has-son" and "has-child", it can be obtained that "has-son" is a child relationship of "has-child", and therefore it can be inferred that Alice is the child of Bob, that is, the new relationship has-child(Alice, Bob) can be obtained. The category conversion logic can also be designed through the category attribute information of the entity. For example, for the entity Alice, it has the attribute mother, and mother is a child attribute of women, and therefore it can be obtained that the attribute of Alice is women. The above is an example of the rule inference process of the logic.

[0075] Step 130: reasoning and completing the initial knowledge graph according to the qualified rules, obtaining new nodes and corresponding relationships, and adding them to the graph knowledge set.

[0076] In the embodiment, there can be implicit nodes and relationships in the initial knowledge graph that are not shown in the graph. After the qualified rules are determined, the initial knowledge graph can be reasoned and completed, and new nodes and their relationships can be inferred. These new nodes and relationships are new knowledge inferred from the existing knowledge in the knowledge graph. Further, the new knowledge inferred can be added to the graph knowledge set together with the original old knowledge in the knowledge graph.

[0077] Optionally, the multi-modal knowledge graph can be linked to predict according to the hierarchical structure, so as to obtain new nodes and relationships. The hierarchical structure in the knowledge graph is similar to the relationship path, and contains rich inference modes, so as to improve the effect of the link prediction task. The hierarchical structure refers to the structure in which the entities are organized into layers by a relationship. Different layers mean different orders. For each entity, other entities are above, below or in the same layer, and the meaning varies with the relationship. The hierarchical structure can be divided into two categories, single-step hierarchical structure and multi-step hierarchical structure. A relationship path is a combination of all relationships between a certain head entity and a certain tail entity. A single-step hierarchical structure is a subgraph of a hierarchical structure. The entities in the single-step hierarchical structure are distributed in two different layers and are linked by a single-step relationship, and the tail entities have a common parent node. The entities in the multi-step hierarchical structure are distributed in different layers and are linked by relationship paths, and these paths have the same head entity and tail entity. The knowledge graph is analyzed according to the hierarchical structure, and the distribution of the entities combined by the hierarchical relationship in the embedding space is relatively obvious. This limitation can be well used in the link prediction task, so as to predict new nodes and relationships.

[0078] The embodiment of the present invention first obtains an initial knowledge graph and generates alternative rules based on the initial knowledge graph; then discriminates the alternative rules to determine qualified rules with a confidence level greater than a set threshold; finally, the initial knowledge graph is reasoned and completed based on the qualified rules to obtain new nodes and corresponding relationships, and added to the graph knowledge set. The knowledge reasoning method of the knowledge graph provided by the embodiment of the present invention combines the rule reasoning method with the graph neural network reasoning method, uses the graph neural network as a generator to generate alternative rules, and inputs them into the discriminator constructed by the rule reasoning method for judgment, thereby forming a hybrid reasoning framework based on the generative adversarial model. The hybrid reasoning framework is designed to implement hybrid reasoning, and the nodes and relationships are predicted using a link prediction method based on a hierarchical structure. It combines the advantages of different reasoning methods, which not only improves the generalization ability and computational efficiency of knowledge reasoning, but also ensures the accuracy and interpretability of the reasoning results.

[0079] Example 2

[0080] Figure 2 This is a flow chart of a knowledge reasoning method for a knowledge graph provided by the second embodiment of the present invention. This embodiment is applicable to the case of performing knowledge reasoning based on a knowledge graph. Figure 2 As shown, the specific steps include:

[0081] Step 210: Obtain an initial knowledge graph, extract images and texts in the initial knowledge graph, and represent them into nodes.

[0082] In this embodiment, for the initial knowledge graph, a graph neural network algorithm can be used to obtain multi-hop connection paths between entity pairs from the multimodal knowledge graph. The path is represented by a sequence of triples, i.e., two entities and their corresponding relationships. Multiple paths between entity pairs can be obtained through aggregation functions to obtain a unified overall representation to obtain alternative rules.

[0083] The initial knowledge graph can be a multimodal knowledge graph, i.e., it contains data in different forms, such as image data and text data. Optionally, images and text from the initial knowledge graph can be extracted and represented as nodes by: extracting images from the initial knowledge graph and generating image descriptions based on the images; extracting text from the initial knowledge graph, performing semantic parsing on the text and image descriptions, and identifying the parsed entities and attributes as nodes.

[0084] Specifically, image data can be processed using a predefined algorithm. Preferably, the Faster-RCNN algorithm can be used to extract visual regions, followed by an image description generator to generate image descriptions. The generated image descriptions and text data are then processed using semantic parsing tools, with the resulting entities and attributes serving as nodes. Edges exist between any two nodes, modifying the spatial relationship between regions. Preferably, edges between nodes can be modified by semantic relationships.

[0085] Step 220, the entity vector and the relationship vector corresponding to the node are initialized respectively to generate the entity embedding vector and the relationship embedding vector.

[0086] Wherein, the entity and the relationship corresponding to the node can obtain the embedded representation of the entity and the relationship by using the set algorithm, that is, to generate the entity embedding vector and the relationship embedding vector. The inference technology based on the graph neural network can use the graph convolutional neural network (GCN) to represent the multi-modal graph atlas to obtain the embedding vector of the entity, so that the entity can obtain the knowledge of the adjacent entity.

[0087] Optionally, the relationship representation model can use the TransE algorithm, the core of which is that the head entity vector plus the relationship vector is equal to the tail entity vector. The obtained entity vector can be initialized using the GCN model, and the relationship vector can be initialized using the one-hot method. Then input it into the TransE algorithm for fine-tuning, and train it to the model convergence to obtain the final embedded representation of the entity and the relationship.

[0088] Step 230, extract each triple sequence in the entity embedding vector and the relationship embedding vector, and input it into the long short-term memory model to obtain the candidate rule.

[0089] Wherein, the triple sequence includes two entity embedding vectors and the corresponding relationship embedding vector. The long short-term memory model (LSTM) is a kind of time recurrent neural network, which is specially designed to solve the long-term dependence problem of general RNN (recurrent neural network). In this embodiment, the representation in different sequences between entities can be obtained by using LSTM to determine the rule in the knowledge graph.

[0090] In this embodiment, the multi-hop connection path between the entity pairs can be obtained from the multi-modal knowledge graph, which is represented by the triple sequence, wherein the entity embedding and the relationship embedding vector are the representation embedding obtained in the above steps.

[0091] Optionally, a plurality of triple sequences between two entity pairs are input into the long short-term memory model (LSTM) to obtain the representation in different sequences of the two entities, and then the multi-hop connection path between the entity pairs identified by the triple sequence is obtained by using the aggregation function to obtain the unified overall representation to determine the candidate rule.

[0092] Figure 3is a schematic diagram of an image data graph neural network inference process in the embodiment, as shown, for image data, first, through the Faster-RCNN algorithm, the visual area is extracted, and then the visual area is used as an object to generate an image description by using an image description generator, and the image description includes a case, a fan, a ladder and the like. Further, the entities in the image are represented as nodes through semantic parsing, and finally the multi-hop connection path between the entity pairs is extracted and represented in the form of a visual graph after generating the entity embedding vector and the relationship embedding vector.

[0093] Step 240, rule inference is performed on the initial knowledge graph to determine new rules.

[0094] In the embodiment, the rule-based inference technology can learn rules with high confidence from a large-scale knowledge graph through an automatic rule learning method, and be applied to an inference task.

[0095] Optionally, the way of performing rule inference on the initial knowledge graph to determine new rules can be: determining rule conversion logic corresponding to the initial knowledge graph; and learning new rules from the initial knowledge graph according to the rule conversion logic.

[0096] Specifically, a rule inference method based on logical rules can be used to learn new rules from the rule conversion logic in the initial knowledge graph, and the new rules can be used as a basis for discriminating the candidate rules and judging the confidence of the candidate rules, so as to determine qualified rules.

[0097] Preferably, the AMIE algorithm can be used to generate new rules. The full name of the AMIE algorithm is Association Rule Mining under Incomplete Evidence, which learns rules to predict each relationship in turn. For each relationship, starting from the rule with an empty body, the body part is expanded through three operations, and the candidate (closed) rules with support greater than the threshold are retained. The three operations are:

[0098] Add a hanging edge: the hanging edge refers to an edge with one end being a variable that has not appeared before and the other end (variable or constant) appearing in the rule;

[0099] Add an instance edge: the instance edge is similar to the hanging edge, and one end of the edge is a variable or constant that has appeared in the rule, but the other end is a constant that has not appeared before, that is, an entity in the knowledge base;

[0100] Add a closed edge: the closed edge is an edge connecting two elements (variables or constants) that already exist in the rule.

[0101] Step 250, determine the confidence of each candidate rule according to the new rule, and determine the candidate rule with the confidence greater than the set threshold as a qualified rule.

[0102] In the embodiment, the discriminator can be generated by using the rule reasoning and learning reasoning algorithm, and the new rule generated by the rule reasoning algorithm is used as the discrimination basis to discriminate the confidence of the candidate rule generated by the graph neural network algorithm in the foregoing step.

[0103] Optionally, the way of determining the confidence of the candidate rule according to the new rule can be that a discriminator is generated by using the rule reasoning algorithm and the learning reasoning algorithm, and the new rule is determined as the true value in the discriminator; and the candidate rules are input into the discriminator in turn, and the output of the discriminator is the confidence of each candidate rule.

[0104] Specifically, the candidate rule is generated by using the graph neural network algorithm, the confidence of the output rule of the graph neural network is identified by using the logic rule-based reasoning algorithm as the discriminator, the hybrid reasoning is realized by using the idea of generative adversarial learning, and the hybrid reasoning framework based on the generative adversarial model is realized. Figure 4 is a hybrid reasoning process based on a generative adversarial model provided by the embodiment, as shown in the figure, the graph neural network algorithm can be used as a rule generator, a plurality of candidate rules are generated based on an initial knowledge graph, and then the candidate rules are input into a discriminator generated by a rule reasoning algorithm and a learning reasoning algorithm. The rule reasoning algorithm and the learning reasoning algorithm are both logic rule-based reasoning algorithms, the new rule generated by the algorithm has high accuracy, can be used as the true value in the discriminator to discriminate the candidate rules, make the discriminator output the confidence of the candidate rules, and determine the candidate rule with the confidence greater than the set confidence threshold as a qualified rule according to the set confidence threshold.

[0105] Further, the candidate rules can also be labeled by using the logic rule-based reasoning algorithm, the data containing the labels are sent to the graph neural network algorithm model for neural network training, and the reasoning accuracy of the graph neural network algorithm is further improved. Specifically, the graph neural network learning RGCN reasoning model is constructed based on the multi-modal knowledge graph, after the entity representation in the knowledge graph is updated according to the labeled data, the long short-term memory network (LSTM) can be used to score the candidate paths, so as to select the best reasoning path, and new candidate rules are reasoned from the entity chain in the path.

[0106] Step 260, according to the qualified rule, a subgraph with the same relationship in the initial knowledge graph is extracted.

[0107] In the embodiment, after obtaining the qualified rules, the hierarchical constraint knowledge can be further used for link prediction of the multi-modal knowledge graph, so as to obtain new knowledge. The hierarchical structure in the knowledge graph is similar to the relationship path, and contains rich reasoning modes, so as to improve the effect of link prediction and other tasks. The hierarchical structure refers to the structure in which entities are organized into layers by a relationship. Different layers mean different orders, and for each entity, other entities are above, below or in the same layer, and the meanings vary with the relationship. The hierarchical relationship is the relationship that generates the hierarchical structure and allocates entities to different layers. The hierarchical structure model can be set, the knowledge graph is input into the model, and the subgraph with the same relationship is output.

[0108] Optionally, the link prediction with hierarchical constraint needs to extract the subgraph with the same relationship in the knowledge graph first, Figure 5a and Figure 5b respectively, a knowledge graph subgraph diagram and a subgraph hierarchical structure diagram provided in the embodiment, as shown in the figure, the solid line in the figure represents the hierarchical relationship, and the dashed line represents the non-hierarchical relationship. Figure 5b is a three-layer hierarchical structure extracted from Figure 5a

[0109] Step 270, input the initial knowledge graph and the subgraph into the link prediction model, output the predicted new node and the corresponding relationship, and add to the knowledge graph set.

[0110] In the embodiment, the hierarchical structure can be divided into two categories, single-step hierarchical structure and multi-step hierarchical structure. A single-step hierarchical structure is a subgraph of a hierarchical structure, the entities in the single-step hierarchical structure are distributed in two different layers and are linked by a single-step relationship, and the tail entities have a common parent node. The entities in the multi-step hierarchical structure are distributed in different layers and are linked by a relationship path, and the paths have the same head entity and tail entity. The distribution of entities combined by the hierarchical relationship in the embedding space will have a relatively obvious distinction, and this limitation can be well used in the link prediction task. By establishing a link prediction model, the knowledge graph, that is, the subgraph with the same relationship, is input into the model, and the output of the model is the predicted node or relationship. Further, the predicted node and relationship are new knowledge obtained by knowledge reasoning from the initial knowledge graph, and can be added to the knowledge graph set.

[0111] ​The embodiment of the application first acquires an initial knowledge graph, extracts images and texts in the initial knowledge graph, and represents the images and texts as nodes, then initializes entity vectors and relation vectors corresponding to the nodes respectively, generates entity embedding vectors and relation embedding vectors, extracts each triple sequence in the entity embedding vectors and the relation embedding vectors, inputs the triple sequence into a long short-term memory model to obtain candidate rules, performs rule reasoning on the initial knowledge graph to determine new rules, determines the confidence of each candidate rule according to the new rules, determines a candidate rule with a confidence greater than a set threshold as a qualified rule, extracts a subgraph with the same relation in the initial knowledge graph according to the qualified rule, and finally inputs the initial knowledge graph and the subgraph into a link prediction model to output predicted new nodes and corresponding relations, and adds the new nodes and the corresponding relations to a knowledge graph set. The knowledge reasoning method of the knowledge graph provided by the embodiment of the application combines the rule reasoning method and the graph neural network reasoning method, uses the graph neural network as a generator to generate candidate rules, and inputs the candidate rules into a discriminator constructed by the rule reasoning method to determine, thereby forming a hybrid reasoning framework based on a generative adversarial model. The hybrid reasoning framework realizes hybrid reasoning, and uses a link prediction method based on a hierarchical structure to predict nodes and relations, which combines the advantages of different reasoning methods, improves the generalization ability and computational efficiency of knowledge reasoning, and guarantees the accuracy and interpretability of the reasoning result.

[0112] Embodiment three

[0113] Figure 6 The device structure schematic diagram of the knowledge graph knowledge reasoning device provided by the embodiment three of the application is shown in FIG. 3. Figure 6 As shown in the figure, the device includes a candidate rule generation module 310, a qualified rule determination module 320, and a reasoning module 330.

[0114] The candidate rule generation module 310 is configured to acquire an initial knowledge graph and generate candidate rules according to the initial knowledge graph.

[0115] Optionally, the candidate rule generation module 310 is further configured to:

[0116] extract images and texts in the initial knowledge graph and represent the images and texts as nodes, initialize entity vectors and relation vectors corresponding to the nodes respectively to generate entity embedding vectors and relation embedding vectors, extract each triple sequence in the entity embedding vectors and the relation embedding vectors, and input the triple sequence into a long short-term memory model to obtain candidate rules; wherein the triple sequence includes two entity embedding vectors and a corresponding relation embedding vector.

[0117] Optionally, the candidate rule generation module 310 is further configured to:

[0118] The image in the initial knowledge graph is extracted, and an image description is generated according to the image; the text in the initial knowledge graph is extracted, and semantic analysis is performed on the text and the image description, and entities and attributes obtained through the analysis are determined as the nodes.

[0119] The qualified rule determination module 320 is configured to determine the qualified rules by judging the candidate rules, and determine the qualified rules with the confidence greater than the set threshold.

[0120] Optionally, the qualified rule determination module 320 is further configured to:

[0121] The initial knowledge graph is subjected to rule reasoning to determine new rules; the confidence of each candidate rule is determined according to the new rules, and the candidate rules with the confidence greater than the set threshold are determined as the qualified rules.

[0122] Optionally, the qualified rule determination module 320 is further configured to:

[0123] The rule conversion logic corresponding to the initial knowledge graph is determined; and the new rules are learned from the initial knowledge graph according to the rule conversion logic.

[0124] Optionally, the qualified rule determination module 320 is further configured to:

[0125] The discriminator is generated according to the rule reasoning algorithm and the learning reasoning algorithm, and the new rules are determined as the true values in the discriminator; the candidate rules are input into the discriminator in sequence, and the output of the discriminator is the confidence of each candidate rule.

[0126] The reasoning module 330 is configured to perform reasoning and completion on the initial knowledge graph according to the qualified rules, obtain new nodes and corresponding relationships, and add them to the knowledge graph set.

[0127] Optionally, the reasoning module 330 is further configured to:

[0128] According to the qualified rules, a subgraph with the same relationship in the initial knowledge graph is extracted; the initial knowledge graph and the subgraph are input into a link prediction model, and the predicted new nodes and corresponding relationships are output.

[0129] The above device can perform the method provided by all the preceding embodiments of the present disclosure, has the corresponding function modules and beneficial effects for performing the above method. Technical details not described in detail in the present embodiment can be referred to the method provided by all the preceding embodiments of the present disclosure.

[0130] Embodiment Four

[0131] Figure 7 A structural schematic diagram of a computer device provided by Embodiment Four of the present application is shown. Figure 7 A block diagram of a computer device 412 suitable for implementing embodiments of the present application is shown. Figure 7The computer device 412 shown is only an example and should not limit the functionality and scope of use of the embodiments of the present invention. The device 412 is a typical knowledge reasoning computing device for a knowledge graph.

[0132] like Figure 7 As shown, computer device 412 is implemented as a general-purpose computing device. Components of computer device 412 may include, but are not limited to, one or more processors 416, storage device 428, and a bus 418 that connects various system components (including storage device 428 and processor 416).

[0133] Bus 418 represents one or more of several types of bus structures, including a memory bus or memory controller, a peripheral bus, an accelerated graphics port, a processor, or a local bus using any of a variety of bus architectures. Examples of these architectures include, but are not limited to, an Industry Standard Architecture (ISA) bus, a Micro Channel Architecture (MCA) bus, an Enhanced ISA bus, a Video Electronics Standards Association (VESA) local bus, and a Peripheral Component Interconnect (PCI) bus.

[0134] The computer device 412 typically includes a variety of computer system readable media. These media can be any available media that can be accessed by the computer device 412, including volatile and non-volatile media, removable and non-removable media.

[0135] The storage device 428 may include computer system readable media in the form of volatile memory, such as random access memory (RAM) 430 and / or cache memory 432. The computer device 412 may further include other removable / non-removable, volatile / non-volatile computer system storage media. By way of example only, the storage system 434 may be used to read and write non-removable, non-volatile magnetic media ( Figure 7 Not shown, often called a "hard drive"). Although Figure 7A disk drive, a floppy disk drive, a CD-ROM drive, a DVD-ROM drive, or other removable media drive, can be provided for reading from and writing to a removable nonvolatile media (such as a floppy disk, a CD-ROM, a DVD-ROM, or other media). In such cases, each drive can be connected to the system bus 418 by one or more data media interfaces. The storage device(s) 428 can include at least one program product having a set (e.g., at least one) of program modules that are configured to carry out the functions of embodiments of the application.

[0136] Program 436 having a set (at least one) of program modules 426 can be stored in, for example, storage device 428, which can include, without limitation, operating system, one or more application programs, other program modules, and program data, and have various sub-modules that can perform one or more particular functions relating to the subject matter described herein. Program modules 426 typically carry out the functions and / or methodologies of embodiments of the application described herein.

[0137] Computer device 412 can also communicate with one or more external devices 414 (such as a keyboard or a pointing device, camera, display 424, etc.) that can enable a user to interact with computer device 412 and / or one or more devices that enable a user to interact with computer device 412 and / or any devices (such as a networking module, a modem, etc.) that enable computer device 412 to communicate with one or more other computing devices. Such communication can occur via Input / Output (I / O) interface(s) 422. Still yet, computer device 412 can communicate with one or more networks (such as a local area network (LAN), a wide area network (WAN), and / or the Internet) through network adapter 420. As an example, network adapter 420 can include a modem, a network card (wireless or wired), or other well-known interface devices. As depicted, network adapter 420 communicates with the other

[0138] Processor 416 can execute instructions loaded into memory 424 to perform various functions and data processing, such as implementing the knowledge reasoning method of the knowledge graph according to the embodiments of the present application.

[0139] Example Five

[0140] The computer readable storage medium of the embodiments of the present application stores a computer program, and the program is executed by a processing device to implement the knowledge reasoning method of the knowledge graph in the embodiments of the present application. The computer readable medium of the present application can be a computer readable signal medium or a computer readable storage medium or any combination of the two. The computer readable storage medium can be, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or apparatus, or any combination of the above. More specific examples of the computer readable storage medium can include, but are not limited to, an electrical connection having one or more wires, a portable computer diskette, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present disclosure, the computer readable storage medium can be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, device or apparatus. In the present disclosure, the computer readable signal medium can include a data signal carried in a baseband or as a part of a carrier wave, which carries computer readable program code. Such a propagated data signal can take various forms, including but not limited to an electromagnetic signal, an optical signal or any suitable combination of the above. The computer readable signal medium can also be any computer readable medium other than the computer readable storage medium, which can send, propagate or transmit a program for use by or in conjunction with an instruction execution system, device or apparatus. The program code contained in the computer readable medium can be transmitted by any suitable medium, including but not limited to a wire, a cable, an RF (radio frequency) or the like, or any suitable combination of the above.

[0141] In some embodiments, the client, server, or both can communicate using any current known or future developed network protocol, such as HTTP (HyperText Transfer Protocol), and can be interconnected with any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include local area networks ("LAN"), wide area networks ("WAN"), the Internet, and peer-to-peer networks (e.g., ad hoc peer-to-peer networks), as well as any current known or future developed networks.

[0142] The computer readable medium described above can be included in the electronic device described above; or can exist separately and not be assembled into the electronic device.

[0143] The computer readable medium described above carries one or more programs, which, when executed by the electronic device, cause the electronic device to: acquire an initial knowledge graph, and generate candidate rules according to the initial knowledge graph; discriminate the candidate rules to determine qualified rules with a confidence greater than a set threshold; and perform reasoning completion on the initial knowledge graph according to the qualified rules, acquire new nodes and corresponding relationships, and add them to the graph knowledge set.

[0144] Computer program code for carrying out operations of the present disclosure can be written in one or more programming languages or combinations of languages including object oriented programming languages, such as Java, Smalltalk, C++, and conventional procedural programming languages, such as the "C" programming language or similar programming languages. The program code can execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer, or entirely on the remote computer or server. In the latter scenario, the remote computer can be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or the connection can be made to an external computer (for example, through the Internet using an Internet Service Provider).

[0145] The flow diagrams and the block diagrams in the drawings are illustrations of possible architectures, functions, and operations for systems, methods, and computer program products according to various embodiments of the present disclosure. In this regard, each block in the flow diagrams and the block diagrams can represent a module, a segment, or a portion of code, which comprises one or more executable instructions for implementing the specified logical function(s). It should also be noted that in some alternative implementations, the functions noted in the blocks can occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently or the blocks may be executed in the reverse order, depending on the functionality involved. It will also be noted that each block of the block diagrams and / or flow diagrams, and combinations thereof, can be implemented by special purpose hardware-based systems that perform the specified functions or operations, or combinations of special purpose hardware and computer instructions.

[0146] The units described in the embodiments of the present disclosure can be implemented by software, or by hardware. In some cases, the names of the units do not constitute a limitation on the units themselves.

[0147] The functionality described herein above can be performed, at least in part, by one or more hardware logic components. For example, and without limitation, illustrative types of hardware logic components that can be used include Field-programmable Gate Arrays (FPGAs), Program- specific Integrated Circuits (ASICs), Application-specific Standard Products (ASSPs), System-on-a-chip systems (SOCs), Complex Programmable Logic Devices (CPLDs), etc.

[0148] In the context of this disclosure, a machine-readable medium can be a tangible medium that contains or stores a program for use by or in connection with an instruction execution system, apparatus, or device. The machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any suitable combination of the foregoing. More specific examples of a machine-readable storage medium will include one or more lines of electrical connections, portable computer disks, hard disk drives, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or Flash memory), optical fiber, portable compact disc read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0149] It is to be understood that the above description is directed to the preferred embodiments and that those skilled in the art will be able to devise various further embodiments without departing from the scope of the present application. The scope of the application is thus indicated by the appended claims.

Claims

1. A knowledge reasoning method for knowledge graph, characterized in that: include: Obtaining an initial knowledge graph, extracting images and text from the initial knowledge graph and representing them into nodes, respectively initializing entity vectors and relationship vectors corresponding to the nodes to generate entity embedding vectors and relationship embedding vectors, extracting each triple sequence from the entity embedding vectors and the relationship embedding vectors, and inputting the triple sequences into a long short-term memory model to obtain candidate rules; the triple sequence includes two entity embedding vectors and the corresponding relationship embedding vectors; Determine the rule conversion logic corresponding to the initial knowledge graph; learn new rules from the initial knowledge graph according to the rule conversion logic; generate a discriminator according to the rule inference algorithm and the learning inference algorithm, and determine the new rule as the true value in the discriminator; wherein the learning inference algorithm is an inference algorithm based on logical rules; input the candidate rules into the discriminator in sequence, the output of the discriminator is the confidence level corresponding to each candidate rule, and determine the candidate rules with a confidence level greater than a set threshold as qualified rules; The initial knowledge graph is inferred and completed according to the qualified rules to obtain new nodes and corresponding relationships, and added to the graph knowledge set.

2. The method according to claim 1, characterized in that Extracting images and text from the initial knowledge graph and representing them into nodes, including: Extracting an image from the initial knowledge graph and generating an image description based on the image; Extract text from the initial knowledge graph, perform semantic parsing on the text and image description, and determine the entities and attributes obtained from the parsing as the nodes.

3. The method according to claim 1, characterized in that The initial knowledge graph is inferred and completed according to the qualified rules to obtain new nodes and corresponding relationships, including: Extracting subgraphs with the same relationship in the initial knowledge graph according to the qualification rule; The initial knowledge graph and subgraph are input into a link prediction model, and the predicted new nodes and corresponding relationships are output.

4. A knowledge reasoning device for a knowledge graph, characterized in that: include: An alternative rule generation module is configured to obtain an initial knowledge graph, extract images and text from the initial knowledge graph, and represent them as nodes. The entity vectors and relationship vectors corresponding to the nodes are initialized to generate entity embedding vectors and relationship embedding vectors, extract triple sequences from the entity embedding vectors and the relationship embedding vectors, and input the triple sequences into a long short-term memory model to obtain alternative rules. The triple sequence includes two entity embedding vectors and the corresponding relationship embedding vectors. A qualified rule determination module is configured to determine the rule conversion logic corresponding to the initial knowledge graph; learn new rules from the initial knowledge graph based on the rule conversion logic; generate a discriminator based on a rule inference algorithm and a learning inference algorithm, and determine the new rule as a true value in the discriminator; wherein the learning inference algorithm is an inference algorithm based on logical rules; sequentially input the candidate rules into the discriminator, the output of the discriminator being the confidence level corresponding to each candidate rule, and determine the candidate rules having a confidence level greater than a set threshold as qualified rules; The reasoning module is used to reason and complete the initial knowledge graph according to the qualified rules, obtain new nodes and corresponding relationships, and add them to the graph knowledge set.

5. A computer device, characterized in that: include: A memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, the knowledge reasoning method of the knowledge graph as described in any one of claims 1 to 3 is implemented.

6. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processing device, it implements the knowledge reasoning method of the knowledge graph as described in any one of claims 1-3.

Images