Neural symbol multi-hop reasoning method and system for domain knowledge graph

Abstract

The invention relates to a neural symbol multi-hop reasoning method and system for a domain knowledge graph. The method comprises the following steps: dividing a domain knowledge graph into multi-order sub-graphs, and performing symbol reasoning on each multi-order sub-graph according to a chained domain ontology to obtain an extended sub-graph; taking each multi-order sub-graph as a training sample, and coding instances in the multi-order sub-graphs to obtain class coding vectors and assertion coding vectors; constructing an assertion set reasoning network; taking assertions in the extended subgraph and the negative example set as a positive example and a negative example to construct a supervision label, and training an assertion set reasoning network by using a training sample and the supervision label; dividing a to-be-inferred domain knowledge graph into to-be-inferred multi-order sub-graphs, performing instance coding, inputting the to-be-inferred domain knowledge graph into the trained assertion set inference network, and obtaining corresponding extension sub-graphs according to the output extension type assertion and the established relation assertion; and combining the extended sub-graphs to output the extended domain knowledge graph. By adopting the method, a high-quality, strong-generalization and high-efficiency chain domain knowledge graph reasoning task can be realized.

Description

Technical Field

[0001] This application relates to the field of neural symbolic reasoning technology, and in particular to a neural symbolic multi-hop reasoning method and system oriented towards domain knowledge graphs. Background Technology

[0002] As artificial intelligence technology leaps from perceptual intelligence to cognitive intelligence, domain knowledge graphs, capable of accurately depicting entities, relationships, and deep semantic connections within a domain in a structured form, bridge the semantic gap in deep learning and become a core technology supporting complex knowledge reasoning tasks. Simultaneously, neurosymbolic learning technology is gradually emerging, organically integrating the efficient numerical computation capabilities of neural networks with the rigorous logical reasoning advantages of symbolic systems. It can generate supervised data containing domain logic through symbolic reasoning and learn logical mapping patterns from subgraphs to extended subgraphs using neural networks, providing a technical path that balances efficiency and accuracy for domain knowledge graph reasoning. It has already demonstrated some applicability in small-scale graph reasoning tasks using lightweight domain ontologies (such as OWL Lite). Currently, mainstream knowledge graph reasoning methods in the field can be mainly divided into three categories: first, neurosymbolic reasoning methods represented by CFR; second, traditional symbolic reasoning machines represented by Pellet and ARBS; and third, multi-hop reasoning methods based on Knowledge Graph Representation Learning (KGRL).

[0003] However, these traditional methods have significant limitations when dealing with chained domain ontology (such as ontology with role transit axioms and SWRL rules) and large-scale domain knowledge graph reasoning, including OWL DL and other ontology with chained reasoning axioms and rules, as well as large-scale domain knowledge graph reasoning. Neural symbolic reasoning methods, represented by CFR, have poor adaptability. Their method of encoding instance subgraphs using multi-level adjacency matrices is extremely inefficient. If the average instance degree is 10 and the subgraph order is 3, the matrix dimension can reach 1111×1111×20. Furthermore, it cannot adapt to the problem of sparse subgraph encoding or information loss caused by the power-law distribution of node degree. Even with dimensionality reduction methods such as HOPE and PCA, a large amount of key semantic information is still lost, ultimately making it unable to handle multi-hop reasoning tasks for chained domain ontology. While traditional symbolic reasoning engines such as Pellet and ARBS can guarantee reasoning completeness, their reasoning time complexity is as high as EXPTIME, increasing exponentially with the size of the graph. This makes them unsuitable for medium-sized domain knowledge graphs. The test set experienced memory overflow, and large-scale graph inference even experienced more than 10 hours of unresponsiveness, making it completely unsuitable for real-world application scenarios. KGRL-like methods rely on vector representations of existing instances in the domain knowledge graph, making it impossible to generalize to unseen new instances and difficult to model complex chained reasoning logic in chained domain ontology. At the same time, most traditional frameworks use the complete domain knowledge graph as input to the neural network, which increases the difficulty of the model learning complex logical mappings and causes a sharp drop in inference efficiency due to the need to enumerate a massive number of triples to determine validity. In addition, the large difference in the distribution of training and testing data further limits the inference generalization ability, making it difficult to meet the needs of high-quality and high-efficiency knowledge reasoning in complex fields such as medicine and industry. Summary of the Invention

[0004] Therefore, it is necessary to provide a neural symbolic multi-hop reasoning method and system oriented towards domain knowledge graphs to address the aforementioned technical problems.

[0005] A neural symbolic multi-hop reasoning method for domain knowledge graphs, the method comprising: Obtain a domain knowledge graph and a chained domain ontology; divide the domain knowledge graph into multi-level subgraphs; and perform symbolic reasoning on each multi-level subgraph based on the chained domain ontology to obtain the corresponding extended subgraph. Each multi-order subgraph is used as a training sample, and instances in the training samples are encoded to obtain a class encoding vector and an assertion encoding vector; the class encoding vector is a one-hot constant vector based on ontology class assertions, and the assertion encoding vector is a low-dimensional dense continuous vector. An assertion set inference network is constructed, comprising a class assertion neural network and a relation assertion neural network corresponding to each relation type of the chained domain ontology. The class assertion neural network is used to process the concatenation result of the class encoding vector and the assertion encoding vector of the instance, and output an extended class assertion encoding vector. The relation assertion neural network is used to process the concatenation result of the assertion encoding vectors of the subject instance and the object instance in the relation assertion, and output the probability of the relation assertion being true. Supervisory labels are constructed using the extended subgraph and the assertions in the pre-constructed negative example set as positive and negative examples, respectively. The class encoding vector of the training samples and the initial assertion encoding vector are input into the assertion set inference network. The loss is calculated based on the difference between the output and the supervisory label. The assertion encoding vector and network parameters are iteratively updated to obtain the trained assertion set inference network. The knowledge graph of the reasoning domain is divided into multi-level subgraphs to be reasoned, and each instance in the multi-level subgraph to be reasoned is encoded and then input into the trained assertion set reasoning network. The extended class encoding vector of the output is decoded to obtain the extended class assertion of the instance. The established relation assertion is determined according to the probability of the output relation assertion and the preset threshold. The corresponding extended subgraph is output according to the extended class assertion and the established relation assertion. Merge the extended subgraphs of each multi-level subgraph to be reasoned, and output the extended domain knowledge graph of the domain knowledge graph to be reasoned.

[0006] A neural symbolic multi-hop reasoning system oriented towards domain knowledge graphs, the system comprising: The subgraph extension module is used to acquire a domain knowledge graph and a chained domain ontology, divide the domain knowledge graph into multi-level subgraphs, and perform symbolic reasoning on each multi-level subgraph based on the chained domain ontology to obtain the corresponding extended subgraph. The instance encoding module is used to encode instances in the training samples using each multi-order subgraph as training samples, to obtain class encoding vectors and assertion encoding vectors; the class encoding vectors are one-hot constant vectors based on ontology class assertions, and the assertion encoding vectors are low-dimensional dense continuous vectors. A network construction module is used to construct an assertion set inference network, which includes a class assertion neural network and a relation assertion neural network corresponding to each relation type of the chained domain ontology. The class assertion neural network is used to process the concatenation result of the class encoding vector and the assertion encoding vector of the instance, and output an extended class assertion encoding vector. The relation assertion neural network is used to process the concatenation result of the assertion encoding vectors of the subject instance and the object instance in the relation assertion, and output the probability of the relation assertion being true. The network training module is used to construct supervision labels using the extended subgraph and the assertions in the pre-constructed negative example set as positive and negative examples, respectively. The class encoding vector of the training samples and the initial assertion encoding vector are input into the assertion set inference network. The loss is calculated based on the difference between the output and the supervision label. The assertion encoding vector and network parameters are iteratively updated to obtain the trained assertion set inference network. The reasoning generation module is used to divide the knowledge graph of the domain to be reasoned into multi-level subgraphs to be reasoned, encode each instance in the multi-level subgraph to be reasoned, and input it into the trained assertion set reasoning network. It decodes the output extended class encoding vector to obtain the extended class assertion of the instance, determines the valid relation assertion based on the probability of the output relation assertion and a preset threshold, and outputs the corresponding extended subgraph based on the extended class assertion and the valid relation assertion. The results output module is used to merge the extended subgraphs of each multi-level subgraph to be reasoned and output the extended domain knowledge graph of the domain knowledge graph to be reasoned.

[0007] The aforementioned neural symbolic multi-hop reasoning method and system for domain knowledge graphs integrates chained domain ontology with multi-level subgraph modeling. It combines symbolic reasoning to generate extended subgraphs containing implicit assertions, providing a logical foundation for neural reasoning and effectively bridging the interpretability of symbolic methods with the generalization ability of neural methods. Through a dual-encoding mechanism of class encoding vectors and assertion encoding vectors, it preserves the determinism of class concepts while endowing assertions with flexible semantic representation capabilities, solving the problems of weak generalization or logical ambiguity with single encoding. Utilizing an assertion set reasoning network—a class assertion neural network and relation assertion neural networks specific to each relation type—it specifically handles class and relation reasoning tasks, adapting to the modeling needs of complex relation chains in multi-hop scenarios. By iteratively updating the assertion encoding vectors and network parameters, it achieves synergistic optimization of vectors and networks, improving reasoning accuracy. Finally, an extended assertion set is generated during the reasoning phase, efficiently completing multi-hop reasoning. This invention enables high-quality, highly generalizable, and efficient chained domain knowledge graph reasoning tasks. Attached Figure Description

[0008] Figure 1 This is a flowchart illustrating a neural symbolic multi-hop reasoning method for domain knowledge graphs in one embodiment. Figure 2 This is a schematic diagram of the technical framework of TGR in one embodiment; Figure 3 Here is an example of an RDF graph in one embodiment; Figure 4 This is a schematic diagram of a subgraph partitioning algorithm in one embodiment; Figure 5 This is a schematic diagram of the ABRN network structure in one embodiment; Figure 6This is a schematic diagram of the neural network structure of RANN in one embodiment; Figure 7 This is a schematic diagram of the neural network structure of CANN in one embodiment; Figure 8 This is a schematic diagram of the training process of the ABRN model in one embodiment; Figure 9 This is a schematic diagram of the TGR training process in one embodiment; Figure 10 This is a schematic diagram of an algorithm in one embodiment for adding negative example assertions and unknown assertions corresponding to subgraphs in the training data; Figure 11 This is a schematic diagram of the inference process algorithm of TGR in one embodiment; Figure 12 This is a schematic diagram of an extended class assertion encoding vector decoding algorithm in one embodiment; Figure 13 This is a schematic diagram of an algorithm that generates all possible relation assertion combinations in one embodiment. Detailed Implementation

[0009] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0010] In TGR (Tim Gang Reasoner), the neural network is named the Assertion Box Reasoning Network (ABRN). Before introducing the neural network system of TGR, several points need to be emphasized again. (1) In TGR, what is learned is the method for generating instance vector representations, not the instance vectors themselves. This is the essential feature that distinguishes it from the KGLR method. The instance vector representation is constant in TGR. It is not updated during gradient descent, but is iteratively updated through numerical calculation by means of model parameters and specific calculation rules.

[0011] The advantage of doing this is that TGR can use the same computational rules to generate vector representations of unknown instances, so as to have the ability to reason about new instances, and make the reasoning process of TGR have the ability to generalize without relying on the instance data of the domain knowledge graph. (2) TGR adopts a classification model based on triples, which is a common modeling method in KGC tasks. The classification model can calculate the probability of triples in low-dimensional continuous vector representations, which has high computational efficiency and adapts to the characteristics of large-scale multi-order subgraphs. (3) In the modeling process, some elements in the domain ontology are modeled as inherent properties of TGR, while the elements in the domain knowledge graph are modeled as constants or variables independent of the model. This plays an important supporting role in the generalization ability of TGR.

[0012] In one embodiment, such as Figure 1 As shown, a neural symbolic multi-hop reasoning method for domain knowledge graphs is provided, including the following steps: Step 102: Obtain the domain knowledge graph and the chained domain ontology. Divide the domain knowledge graph into multi-level subgraphs. Perform symbolic reasoning on each multi-level subgraph based on the chained domain ontology to obtain the corresponding extended subgraph.

[0013] The definitions and parameter representations involved in step 102 are explained as follows: Domain knowledge graphs and chained domain ontology are constructed and represented based on the open world assumption. Chained domain ontology is a formal expression of pattern-level knowledge in a specific domain. For example, an OWL DL ontology containing chained reasoning axioms (such as relation transitivity and reciprocity axioms) and SWRL rules can characterize long-linked reasoning relationships between instances. Its core components include class axioms (such as class equivalence and subclass relations), relation axioms (such as relation transitivity and reciprocity), and attribute axioms (such as attribute domain / range constraints). It can be constructed and parsed through ontology editing tools. Domain knowledge graphs are instantiations of chained domain ontology, using triples as the basic data unit. The head and tail entities in a triple are specific objects within the domain, and the relation is the association between entities. In a triple, the entity is a real-world object, event, situation, or abstract concept. Triple data can be obtained by recognizing or extracting natural language text. Specifically, head and tail entities are text data obtained from natural language text using named entity recognition tools. This text data includes characters and words, and the named entity recognition tool can be Stanford-NLP. Relationships are text data describing the relationships between head and tail entities. During knowledge graph reasoning, any two of the head, tail, and relationships can be identified, and the unknown one can be reasoned about. Natural language text can be obtained from the internet via web crawlers or manually configured; no specific restrictions are imposed here. Under the open-world assumption, the domain knowledge graph does not classify unstated triples as false, but only as unknowns, supporting dynamic updates (such as adding instances or supplementing relationships).

[0014] The subgraph representation of a domain knowledge graph has the following properties: Subgraph Representation Theorem for Domain Knowledge Graphs: Given a large-scale domain knowledge graph, it can be represented losslessly by all its instances. k - A collection of order subgraphs.

[0015] Therefore, it can be formally expressed as:

[0016] in e i express DKG The instance number in i =1, 2, ..., N . N This represents the number of instances in the graph. k This indicates the order of the subgraph. It's important to note that Equation 1 contains redundant information; the same triple may appear in multiple instances of the subgraph.

[0017] Given a domain knowledge graph DKG Its extended domain knowledge graph is All instancesk The set of --order extended subgraphs is used This indicates that, in the interval [1...], , Within ], there exists a positive integer. k , making Valid, that is, for all instances k The set of --order extended subgraphs can approximate the extended domain knowledge graph, represented as: In the theorem, It is a similarity threshold, defined as follows: Set a [0 , Real numbers between 1 and 1 are denoted as ,if Then it is believed It can approximate well Otherwise, it is considered Cannot approximate well At this point, the real number This is called the similarity threshold. In the definition of the similarity threshold and the subgraph representation theorem of domain knowledge graphs, similarity... Similarity Using a generalized recall rate as a metric, i.e.:

[0018] In the formula, trip Represents an RDF triple. len ( The similarity metric is defined this way because both reasoning on a subgraph and reasoning on the overall domain knowledge graph are logical inferences, and their results are truth-preserving (i.e., the accuracy of both reasoning results is 100%). Furthermore, the subgraph only contains local information. The conclusions drawn from reasoning will not exceed Therefore, using In-depth reasoning triples and The ratio of the inference triples can be used to assess the degree of approximation between the two. The approximation theorem based on subgraphs can be formally expressed as:

[0019] Based on the above formula, large-scale domain knowledge graph reasoning is transformed into an instance-centric approach. k The process of reasoning on a subgraph of order - is called Approximate Reasoning Based on Subgraph (ARBS). ARBS refers to transforming reasoning on a large-scale domain knowledge graph into reasoning on all instances. k - Reasoning on subgraphs, and using all instances kThe set of --order extended subgraphs approximates the extended domain knowledge graph.

[0020] It is worth noting that, from the perspective of the expressive power of domain ontology, the unchained domain ontology is a special case of the chained domain ontology, namely, the length of the reasoning chain. l The case of ≤2; from the perspective of reasoning form, single-hop reasoning is a special case of multi-hop reasoning, that is, within the subgraph reasoning framework, the order of the subgraph is chosen. k =1. Therefore, TGR for chained domain knowledge graph reasoning can also be used for chainless domain knowledge graph reasoning. However, chained domain ontology has stronger expressive power than chainless domain ontology, and its chained reasoning process is more complex, so TGR needs stronger expressive power.

[0021] Step 104: Using each multi-order subgraph as training samples, encode the instances in the training samples to obtain class encoding vectors and assertion encoding vectors. The class encoding vector is a one-hot constant vector based on ontology class assertions, and the assertion encoding vector is a low-dimensional dense continuous vector.

[0022] Step 106: Construct the assertion set inference network, which includes a class assertion neural network and a relation assertion neural network corresponding to each relation type in the chained domain ontology. The class assertion neural network processes the concatenation result of the class encoding vector and the assertion encoding vector of the instance, and outputs an extended class assertion encoding vector. The relation assertion neural network processes the concatenation result of the assertion encoding vectors of the subject instance and the object instance in the relation assertion, and outputs the probability of the relation assertion being true.

[0023] Step 108: Construct supervision labels using the extended subgraph and the assertions in the pre-constructed negative example set as positive and negative examples, respectively. Input the class encoding vector of the training samples and the initial assertion encoding vector into the assertion set inference network. Calculate the loss based on the difference between the output and the supervision label. Iteratively update the assertion encoding vector and network parameters to obtain the trained assertion set inference network.

[0024] from Figure 2 As can be seen, the TGR neural network system is divided into two parts: model training and model inference. Model training includes two main steps: instance vector update and model parameter update. Instance vector update is the process of generating vector representations of instances and imbuing them with the structural and semantic information of the domain knowledge graph. In training, all instances in the training subgraph are first acquired, and vector representations of these instances are randomly generated (e.g., [0...]). ,The initial instance vector representation is chaotic and lacks structural features and semantic information (uniform distribution between 1]. To enable different instance vectors to encode their structural and semantic information, the vector representation of instances in the subgraph is iteratively updated based on the triples in the subgraph and the parameters of ABRN, so that different instances have their own unique vector representation. In TGR, the instance vector representation is constant, and its gradient is not calculated during model training. Instead, the updated instance vector is obtained by calculation after each round of ABRN parameter update. In model parameter update, the updated instance vector is first passed in as input for model training. Based on the triples in the extended subgraph and the negative example set, the vector representation and label of the supervised triples are constructed and input into the ABRN network to achieve model parameter learning. As can be seen from the above two steps, in instance vector update, the parameters of ABRN are used to update the vector representation, that is, to adapt the instance vector representation to the parameters and structure of the neural network; in model parameter update, the instance vector representation is used as input, and the gradient is used to update the model parameters, that is, to adapt the parameters and structure of the neural network to the instance vector representation.

[0025] In this scenario, by iterating through the subgraphs of all instances, the vector representations of the instances and the parameters of the ABRN network can continuously adapt and update to each other, ultimately achieving effective training of the neural network.

[0026] Step 110: Divide the knowledge graph of the reasoning domain into multi-level subgraphs to be reasoned, encode each instance in the multi-level subgraphs to be reasoned, and input them into the trained assertion set reasoning network. Decode the output extended class encoding vector to obtain the extended class assertion of the instance. Determine the valid relation assertion based on the probability of the output relation assertion and the preset threshold. Output the corresponding extended subgraph based on the extended class assertion and the valid relation assertion.

[0027] The model's inference process is divided into two steps: instance vector updating and model inference. Instance vector updating uses the same method as inference: first, vector representations of all instances in the subgraph are randomly generated; then, through training, all subgraphs are traversed, and the ABRN network parameters are used to update the vector representation of each instance in the subgraph, ensuring it contains the structural and semantic information of the subgraph. In inference, to ensure the completeness of the inference result, the model first considers the existing instances in the subgraph and the domain of the relations (…). rdfs : domain ) and range ( rdfs : rangeThe definition of ABRN involves enumerating all possible triple combinations to discover as many potentially correct inference triples as possible. Then, using updated instance vector representations, all possible triple combinations are vectorized and fed into the ABRN network. The output of the ABRN network is a label indicating whether a triple is true; the set of correct triples forms the extended subgraph. .

[0028] In TGR, both the training and inference of the neural network system utilize negative examples. Regarding the use of negative examples, it's important to note that in real-world applications of knowledge graph reasoning within chained domains, inferring true negative examples is sometimes valuable. For instance, in disease diagnosis, inferring that a patient will not have a certain disease is crucial. Therefore, when inferring true negative examples, they should be distinguished from randomly generated unknown negative examples by adding classification labels (i.e., a multi-class model). For example, true negative examples are labeled 1, true positive examples are labeled 2, and randomly generated unknown triples are labeled 0. This paper only considers the set of positive examples in the reasoning; therefore, TGR employs a binary classification model, treating true negative examples and randomly generated unknown negative examples as a single class, a generalized set of negative triples.

[0029] Step 112: Merge the extended subgraphs of each multi-level subgraph to be reasoned, and output the extended domain knowledge graph of the domain knowledge graph to be reasoned.

[0030] It is understandable that TGR has the following advantages: (1) TGR can adapt to the characteristics of large scale and many instances in multi-order subgraphs, and represent instances and triples as vectors in low-dimensional real space, thereby improving the scalability and learning efficiency of the model and realizing the learning of the mapping relationship between multi-order subgraphs of instances and their extended subgraphs. (2) TGR's reasoning has the ability to generalize without relying on instance data in the graph. TGR learns the generation method of instance vector representation rather than the instance vector representation itself, and can directly generate vector representations for new instances and infer their extended subgraphs through the ABRN network. (3) The training process of the TGR model is a joint improvement of the continuous mutual adaptation and adjustment between instance vectors and ABRN network model parameters, which can effectively learn the chain reasoning axioms and rules in the OWL DL domain ontology and realize the chain domain knowledge graph reasoning task end-to-end.

[0031] The aforementioned neural symbolic multi-hop reasoning method for domain knowledge graphs integrates chained domain ontology with multi-level subgraph modeling. Symbolic reasoning generates extended subgraphs containing implicit assertions, providing a logical foundation for neural reasoning and effectively bridging the interpretability of symbolic methods with the generalization ability of neural methods. A dual-encoding mechanism using class encoding vectors and assertion encoding vectors preserves the determinism of class concepts while endowing assertions with flexible semantic representation capabilities, addressing the problems of weak generalization or logical ambiguity with single encoding. The assertion set reasoning network—a class assertion neural network and relation assertion neural networks specific to each relation type—targetedly handles class and relation reasoning tasks, adapting to the modeling needs of complex relation chains in multi-hop scenarios. Iterative updates to assertion encoding vectors and network parameters achieve synergistic optimization of vectors and networks, improving reasoning accuracy. Finally, an extended assertion set is generated during the reasoning phase, efficiently completing multi-hop reasoning. This invention enables high-quality, highly generalizable, and efficient chained domain knowledge graph reasoning tasks.

[0032] In one embodiment, acquiring a domain knowledge graph and a chained domain ontology, dividing the domain knowledge graph into multi-order subgraphs, and performing symbolic reasoning on each multi-order subgraph based on the chained domain ontology to obtain the corresponding extended subgraph includes: parsing the corresponding ontology axioms based on the chained domain ontology, dynamically determining the length of the longest relation chain for the reasoning task, calculating the subgraph order based on the longest relation chain length and a preset mapping rule; dividing the domain knowledge graph into k-order subgraphs centered on each instance according to the subgraph order, inputting the k-order subgraphs and ontology axioms into a symbolic inference machine to perform reasoning, and generating an extended subgraph containing implicit assertions.

[0033] In this embodiment, the choice of subgraph order has a significant impact on the neural symbolic system within the subgraph reasoning framework. From the perspective of the symbolic system, the higher the subgraph order, the closer the subgraph is to a complete system. DKG The set of subgraph reasoning results is related to the whole. DKG A higher degree of approximation in reasoning leads to higher complexity in recursive queries when partitioning the subgraph, as well as higher complexity in reasoning to obtain the extended subgraph. From a neural network perspective, a higher order of subgraph results in more instances and triples, and a richer data pattern, requiring a more powerful learning ability from the neural network model, thus increasing the difficulty of learning and reasoning. Therefore, choosing an appropriate subgraph order in the subgraph reasoning framework is crucial for improving the quality and efficiency of neural symbolic reasoning.

[0034] Subgraph-based approximation reasoning (ARBS) approximates the reasoning results on the overall domain knowledge graph by using the set of extended subgraphs of all instances. ARBS can construct large amounts of high-quality and diverse training data for neural symbolic reasoning, comprehensively improving the reasoning quality, efficiency, and generalization ability of neural network models. However, an important aspect remains to be explained within the subgraph reasoning framework: the approximation effect of ARBS, specifically, the degree to which the set of extended subgraphs approximates the reasoning results of the overall graph. It can be observed that the order of the subgraph... k It is the only variable in ARBS, and it determines the degree of approximation of the overall graph inference results by ABRS.

[0035] order of subgraph k The scope of the neighbor nodes of the instance is defined. In order to illustrate the requirement that the subgraph contains instance nodes for reasoning, the subgraph contains instance theorem is given as follows.

[0036] Subgraph Inclusion Theorem: In subgraph reasoning, if you want to discover implicit knowledge... r ( s,o If a subgraph contains instances, then it must be guaranteed that at least one subgraph contains instances. s and examples o .

[0037] Proof (Subgraph Inclusion Theorem for Instances): Proof by contradiction. Assume that no subgraph contains instances simultaneously. s and examples o Subgraph reasoning can still uncover implicit knowledge. r ( s,o Let any containing s The subgraph is g sk ( i ) Any containing o The subgraph is g OK ( j ) Based on the assumption, s ∈g sk ( i ) and o ∈ / g sk ( i ) This holds true because logical reasoning can only infer implicit knowledge between existing entities, but cannot discover implicit new nodes. Therefore, for instances... s The extended subgraph satisfies o ∈ / g sk ( i )_ ext ,so r ( s,o )∈ / g sk ( i )_ ext Similarly, we can obtain r ( s,o )∈ / g OK ( j )_ ext Therefore, no subgraph can discover implicit knowledge during the reasoning process. r ( s,o This contradicts the assumption, therefore the theorem holds.

[0038] Although the implicit relationship between two instances can only be discovered if they are both in the same subgraph, it is not required that all permutations and combinations of instance pairs (any instance pair) s , o All instances must be contained within a subgraph, but only instances that may have potential implicit relationships are required to be contained within at least one subgraph. To illustrate this property, we first introduce the concepts of chained reasoning and subgraph diameter.

[0039] Chain reasoning: in domain knowledge graphs DKG In inference, if an implicit triple requires the premises of multiple (greater than or equal to 2) triples, then the reasoning process to obtain the implicit triple is called chain reasoning. In chain reasoning, the number of distinct instances implied in the premise triples is called the length of the reasoning chain, denoted as . l .

[0040] Example of chain reasoning: Figure 3 The example given DKG Take the RDF graph as an example. Assume the following inference rules exist: IF: ( a 1 ,isA,C 3) and ( a 1 ,R 1 ,a 6) and ( a 6 ,R 2 ,a 5) and ( a 5 ,R 3 ,a 4) THEN:( a 1 ,R 1 ,a 4) Then, to obtain the implicit triplet ( a 1 ,R 1 ,a 4) This process is a chain of reasoning, where the IF part represents the premises and the THEN part represents the conclusion. The IF part implies... a 1. a 6. a 5 and a Given 4 instances, what is the length of this reasoning chain? l =4.

[0041] Subgraph diameter: for any instance e i ,That k - Order diagram g eik This can be seen as an example e i With the center as the center, k The diameter of a subgraph (DS) of a circle with radius 1 and a basic unit of 1 jump is denoted as DS( e i , k )=2 k +1 indicates the center instance.

[0042] The reasoning chain that obtains an implicit triple is analogous to a straight line connecting two instances in a plane. The subgraph is analogous to a circle in the plane. If we want to ensure that the straight line is surrounded by at least one circle in the plane, then we must ensure that the diameter of the circle is greater than or equal to the length of the straight line.

[0043] In summary, let's assume... s and o It is a knowledge graph of a given domain. DKG The two entities in the text, entity s and o The length of the inference chain between them is l If we want to ensure that at least one subgraph contains... s and o And all entities implied in its reasoning chain, then there exists at least one instance. e i Satisfying DS( e i , k )≥ l Based on this, the order of the selection subgraph is given. k The empirical formula is shown below.

[0044]

[0045] Combining the empirical formula for choosing the subgraph order with the definitions of single-hop and multi-hop reasoning, we can draw the following conclusions: (1) For chainless domain knowledge graph reasoning, the order of the subgraph This indicates that the first-order subgraph of an instance can contain all the patterns in the axioms of a chainless domain ontology. This means that for reasoning in a chainless domain knowledge graph, single-hop reasoning can obtain reasoning results that are approximately the same as those on the entire graph.

[0046] (2) For reasoning about knowledge graphs with chains, the order of the subgraph ,in l max The maximum length of chained reasoning implied in the axioms and rules of a chained domain ontology indicates that multi-level subgraphs of instances are required to contain patterns that effectively reflect these axioms or rules. This means that for reasoning in chained domain knowledge graphs, multi-hop reasoning is needed to obtain reasoning results that approximate the overall graph.

[0047] It is worth noting that the empirical formula for choosing the subgraph order is to ensure that the ARBS inference results can approximate the inference results on the overall domain knowledge graph well. However, in practical applications, the choice of subgraph order can also be adjusted according to the actual application requirements. Even in chained domain knowledge graphs, when the application only focuses on the first-order neighbors of an instance, single-hop inference can be used, but in this case, some logical implications will be lost. In addition, in practical applications, the length of chained inference should be appropriately truncated. In many cases, the longest inference chain can obtain very few logical implications. If these implications are not very important to the supporting domain application, they can be discarded, and a shorter inference chain can be chosen to determine the subgraph order. Although a small number of logical implications are lost, the impact on the domain application is small and the inference efficiency can be greatly improved.

[0048] For a domain knowledge graph DKG When the order of the subgraph k If the above formula is satisfied, then it can be guaranteed that there are instances with a potential relationship. s,o If a graph is contained within at least one subgraph simultaneously, the reasoning result of ARBS can approximate the reasoning result on the overall graph quite well. The choice of the subgraph order is determined by the reasoning chain. In this invention, domain knowledge graph reasoning is defined as deductive reasoning from abstract to concrete. Therefore, in a domain knowledge graph, when chained reasoning is needed and how the length of the reasoning chain is determined are determined by the axioms and rules in the domain ontology.

[0049] In this embodiment, the combination of different constructors forms OWL DL languages ​​with different expressive capabilities. These ontology languages ​​with different expressive capabilities give the domain ontology different expressive capabilities and reasoning complexities. Not all constructors involve chained reasoning, which means that not all OWL DL domain ontology requires chained reasoning. For domain ontology that does not involve chained reasoning axioms and rules, this invention refers to it as a chainless domain ontology, and the task of reasoning in the domain knowledge graph based on a chainless domain ontology is called chainless domain knowledge graph reasoning. If the domain ontology involves axioms or rules related to chained reasoning, it is called a chained domain ontology, and the task of reasoning in the domain knowledge graph based on a chained domain ontology is called chained domain knowledge graph reasoning. The specific definitions are as follows: Unlinked and linked domain ontology, unlinked and linked domain knowledge graph reasoning: Given a domain ontology Onto , DKG It is a domain knowledge graph generated based on this ontology. If Onto If it does not contain axioms and rules related to chain reasoning, then it is called Onto For the unlinked domain ontology, from Onto Departure, at DKG The process of reasoning about implicit knowledge is called chainless domain knowledge graph reasoning; if Onto If it contains axioms or rules related to chain reasoning, then it is called... Onto For the ontology of the chain domain, from Onto Departure, at DKG The process of reasoning about implicit knowledge is called chained domain knowledge graph reasoning.

[0050] Using the SPARQL query language, multi-level subgraphs of instances can be obtained step-by-step through recursive queries. The specific process of the subgraph partitioning method is as follows: Figure 4 As shown. In the algorithm, the input is Onto , DKG subgraph order k and symbol ,in symbol This setting determines whether to retrieve the extended subgraph. When training a neural symbolic reasoning model, the extended subgraph is needed as supervision data during training. However, when using the neural symbolic model for reasoning, only the instance subgraph is required. The algorithm first uses a recursive query function... GetMultiSubgraph Get the instance k - Rank diagram .if symbol If true, then in the complete domain ontology Onto Next, using the ontology reasoning engine Reasoner, logical reasoning is performed on the subgraph to obtain its corresponding extended subgraph. Then, k - A subgraph pair consisting of an order subgraph and its extended subgraph ( , Add to sub-atlas SubgraphSet In this way, all instances are traversed sequentially. At this point, SubgraphSet It can be directly used as a training dataset for training neural symbolic models, where For input data, For monitoring data. If symbol If it is false, then only the subgraph will be... Join SubgraphSet It is then input into a trained neural symbolic reasoning model, and an approximate extended subgraph is obtained through neural network calculation.

[0051] GetMultiSubgraph This represents a recursive query function that receives the following parameters: DKG Examples e order k Hezi Diagram graph ( graph This is the name of a variable inside the function, and the value assigned to this variable is... The subgraph expands continuously through recursive queries (initially an empty graph), and the output consists of instances. k - Subgraph of order. As can be seen from the code, during subgraph partitioning, the given instance is first obtained using the DESCRIBE command in SPARQL. e The query iterates through the 1-order subgraph and neighbor nodes of the instance, and then recursively traverses the 1-order subgraph and neighbor nodes of all neighbor nodes, adding them to the subgraph of the instance, until the query terminates when the set order condition is met.

[0052] In one embodiment, the mapping rule includes the subgraph order and the length of the longest relation chain satisfying the following condition: ,in, Let the order be the subgraph. The length of the longest relationship chain. This is for rounding up.

[0053] In one embodiment, instances in the training samples are encoded to obtain class encoding vectors and assertion encoding vectors, including: determining the dimension of the class encoding vector based on the total number of class concepts in the chained domain ontology; if an instance has a class assertion belonging to a class concept in the subgraph, the class concept dimension of the class encoding vector is 1, and the dimension of other class concepts is 0; and randomly initializing the assertion encoding vector, the dimension of the assertion encoding vector being a preset fixed value, and consistent with the dimension of the hidden layer in the assertion set inference network.

[0054] This invention represents triples as vectors using an instance encoding method. Without considering instance attributes, the domain knowledge graph consists of class assertions and relation assertions. Class assertions are used... C ( e ) indicates that it means an instance. eIs class C Examples; relational assertions using r ( s,o ) indicates that it means an instance. s and examples o There is a relationship r .

[0055] In TGR, each instance in the domain knowledge graph is represented by two vectors: a class-encoded vector and an assertion-encoded vector. The former contains all known class assertion information for the instance in the graph, used for class assertion reasoning and auxiliary relation assertion reasoning. The latter aims to represent the structural and semantic information of the domain knowledge graph through a continuous vector in a low-dimensional space. The instance's class assertion-encoded vector is represented by a one-hot encoding.

[0056] Instance class assertion encoding vector: Suppose the ontology has n Each class, for instances in the subgraph e Its class encoding vector is n Dimensional vector C e And for the first in the ontology i Class C i If the subgraph contains class assertions C i ( e ), then C e The i The value of a dimension is 1 if it is not 1, and 0 otherwise. For ease of understanding, a simple example is given below: Suppose there are four classes in the ontology: Person , Woman , Man , Mother The subgraph contains assertions of the following class: Person ( Susie ), Woman ( Susie ), then the example Susie Class encoding vector C susie For (1) , 1 , 0 , 0).

[0057] Inspired by knowledge graph representation learning, TGR aims to generate a vector for each instance by iteratively updating randomly initialized vectors through the parameters of the ABRN network. This vector should contain as much information as possible about the class and relation assertions of the instance in the subgraph. Such a vector is called the instance's assertion encoding vector. The instance's assertion encoding vector is represented by a low-dimensional, continuous real-valued vector.

[0058] Assertion encoding vector for instances: for instances in a subgraph e There exists a vector e such that, ideally, e contains information about instances in the subgraph.e If we consider all the structural and semantic information of a variable, then e is called an instance. e The assertion encoding vector.

[0059] Assume vector C e 'e' and 'e' are instances respectively. e The class encoding vector and assertion encoding vector. From the definition, C... e And e satisfy the following conditions: (1) e All class assertions can be passed through C. e (2) Infer from e in the subgraph and another instance. e ′ The encoding vector e ′ Instances can be inferred e and e ′ Whether all possible assertions between them hold true.

[0060] In TGR, the class encoding vector C e The class assertion is generated based on the known class assertions of the instances in the subgraph. It is an invariant constant that depends only on the known class assertions of the instances and is intended to express the class assertion information of the instances. The assertion encoding vector e of the instances is generated by random initialization and is obtained by continuous updating and iteration through the model parameters. It depends on the known triple information of the instances in the subgraph and the structural parameters of the model and is intended to express all the semantic information of the instances in the subgraph.

[0061] The network structure of ABRN is as follows: Figure 5 As shown. In the set of assertions (i.e., triples), relational assertions are binary predicate relations, and class assertions are unary predicate relations. Therefore, for these two types of assertions, the ABRN structure has two types of neural networks: Relational Assertion Neural Network (RANN) and Class Assertion Neural Network (CANN). RANN is a binary classifier, and its input is... and These correspond to the assertion encoding vectors of the two instances, and the output is either 0 or 1, where 1 indicates the input subject instance. Examples of objects There is a relationship between them r That is, relational assertion r ( s,o A 0 indicates the relation assertion is true, and a 0 indicates the relation assertion is false. CANN is a multi-classifier whose input is instances. e The assertion encoding vector e and the class encoding vector C of the instance e The output is an instance. e Extended class assertion encoding vector , Same as C e Having the same dimension (both are one-hot encodings of class concepts in the domain ontology), they contain the information about instances that ABRN can infer. e All class assertions.

[0062] from Figure 5 As can be seen, ABRN contains multiple RANNs, but only one CANN. The number of RANNs equals the number of relation types in the ontology; if the domain ontology contains... n rel If there are several relation types, then ABRN contains... n rel There is one RANN. This means that for each relation in the ontology, the ABRN has exactly one corresponding RANN, which is used solely to infer whether a relation assertion of that type is true. The ABRN has only one CANN, which is used to infer class assertions for all instances. This is because relation assertions... C ( e ) can also be represented as a binary relation ( e,isA,C If a class assertion is in the form of a relation type, then the class assertion can be considered as a relation of type 1. isA For the relational assertion, CANN is equivalent to asserting the relational assertion against CANN. isA The RANN of relations, whose input class encoding vector can be viewed as the encoding vector of instances with respect to all concepts, is therefore, although RANN and CANN differ in form, they are intrinsically unified.

[0063] The neural network structure of RANN is as follows: Figure 6 As shown in the figure. It can be seen from the figure that, regarding relation assertions... r ( s,o Reasoning, Relationship r The corresponding neural network first takes the subject instance as an example. s assertion encoding vector Examples of objects o assertion encoding vector As input, the relational neural network layer then computes a vector of the same dimension as the instance assertion encoding vector, which is output. and Through training, relational neural network layers can model relationships. r The reasoning information, and and In contrast, with continuous iteration and updates in the training process, and Able to encode instances more accurately s and o The structural and semantic information in the atlas, therefore, is used and Update and replace and and will use and By inputting a fully connected layer, relational assertions can be inferred. r ( s,o Whether it is true or not.

[0064] The neural network structure of CANN is as follows: Figure 7 As shown in the figure, CANN encodes the assertion vector of the instance. and class encoding vector C i As input, the output is a vector of the same dimension as the assertion encoding vector. Through training, class assertion neural networks can model inference information about the class to which an instance belongs, and... compared to, It can encode more accurately, so it uses Update and replace It is then fed into a fully connected layer, and the instance can eventually be inferred. i The new class encoding vector, i.e., the extended class encoding vector. .

[0065] In fact, in RANN and CANN, and as well as This is the updated instance assertion encoding vector, which is calculated using the parameters of the neural network and is not updated during gradient descent. Additionally, it's worth noting that the assertion encoding vector for an instance will be updated regardless of whether it's processed through a RANN or CANN network; that is, relation assertions and class assertions in the subgraph assertion set will update the instance's assertion encoding vector.

[0066] In one embodiment, processing the concatenation result of the assertion encoding vectors of the subject instance and the object instance in the relation assertion to output the probability of the relation assertion includes: concatenating the assertion encoding vectors of the subject instance and the object instance in the relation assertion to obtain a second input vector; updating the assertion encoding vectors of the subject instance and the object instance in the second input vector respectively through the relation neural network layer of the relation assertion neural network to obtain the updated subject assertion encoding vector and object assertion encoding vector; and processing the concatenation result of the updated subject assertion encoding vector and object assertion encoding vector through the fully connected layer of the relation assertion neural network to output the probability of the relation assertion.

[0067] like Figure 6 As shown, a RANN consists of a relational neural network layer and a fully connected layer. The relational neural network layer comprises two parts: one part is used to compute and update the subject instance. The encoded vector, part of which is used to calculate and update object instances. The encoded vector. Given a set of triples, for relation assertions... r ( s,o ),set up and For each of the following examples, there are instances of the subject. Examples of objects The corresponding m 3D encoding vector, then for relations r Relational neural network layer computation and update The formula for the encoded vector is as follows:

[0068] in, It is 2 m A dimensional vector representing the concatenation of two instance assertion vectors. and yes m dimensional bias vector, and yes m ×2 m dimensional weight matrix, It is a 2m-dimensional weight vector. This represents the operation of multiplying the elements at the same positions of two vectors of the same dimension to form a new vector. |||2 is the 2-norm of the vector. σ ( ) and RELU ) is the activation function, and its formula is as follows:

[0069]

[0070] Relational neural network layers compute and update tail instances The formula for the encoding vector is as follows:

[0071] In one embodiment, processing the concatenation result of the instance's class encoding vector and assertion encoding vector to output an extended class assertion encoding vector includes: concatenating the instance's assertion encoding vector and class encoding vector to obtain a first input vector; calculating the gating vector and candidate update vector of the first input vector through the class neural network layer of the class assertion neural network, multiplying the candidate update vector and the gating vector element-wise and adding them to the assertion encoding vector to obtain an intermediate vector, normalizing the intermediate vector using the L2 norm to obtain the updated assertion encoding vector; and processing the updated assertion encoding vector through the fully connected layer of the class assertion neural network to output the extended class assertion encoding vector.

[0072] like Figure 7 As shown, CANN contains a neural network-like layer and a fully connected layer. Given a set of triples, let... For example i of m 3D encoded vector, C i for i of n If the encoding vector is dimensional, then the computation of the neural network layer is as follows:

[0073] in, It is a vector spliced ​​into a vector The subsequent formation ( m + n )-dimensional vector.

[0074] Based on the above calculation formula, the update of instance assertion encoding vectors can be implemented in RANN and CANN networks. By iteratively traversing the class assertions and relation assertions in the multi-level subgraph of instances, the instance assertion encoding vectors can be iteratively updated, ensuring that the randomly initialized assertion encoding vectors contain the structural and semantic information of the graph after the update.

[0075] In one embodiment, the loss is calculated based on the difference between the output and the supervision label, and the assertion encoding vector and network parameters are iteratively updated to obtain a trained assertion set inference network. This includes: supervision labels including class assertion supervision labels and relation assertion supervision labels; fixing the parameters of the current assertion set inference network, calculating the loss by expanding the difference between the class assertion encoding vector and the class assertion supervision label, and the difference between the relation assertion probability and the relation assertion supervision label, iteratively updating the assertion encoding vector of the instance based on the loss until the first iteration stopping condition is met, at which point the iteration stops; re-inputting the updated assertion encoding vector into the network to calculate the loss, iteratively updating the network parameters of the assertion set inference network using gradient descent until the second iteration stopping condition is met, at which point the iteration stops, resulting in a trained assertion set inference network.

[0076] In this embodiment, although ABRN handles relational assertions and class assertions differently, the design and expression methods of the two types of neural networks in ABRN are unified. Therefore, the same loss function can be used to evaluate the training process of ABRN and to calculate gradients to provide feedback for adjusting the parameters of ABRN. TGR uses the cross-entropy function as the loss function for ABRN, and its calculation formula is as follows:

[0077] in, This indicates the number of triples in the training samples. Indicates the first training samplei th The true label of a triplet This indicates that the ABRN model applies the first training sample to the training samples. i th The inference labels for each triplet, the training process is essentially minimizing the loss function. L The optimization process.

[0078] The training process of TGR is illustrated as follows: Figure 8 As shown, the process of continuously adjusting the weight parameters in ABRN during training is the process of ABRN modeling relationships and class reasoning information. When training with the training dataset, the assertion encoding vectors of instances are continuously updated to more accurately represent the structural and semantic information contained in the subgraphs. Based on this, training with the updated assertion encoding vectors allows for further iterative optimization of the ABRN model. During the convergence of the loss function, the instance encoding vectors and the parameters of ABRN are gradually optimized. By iteratively traversing all subgraphs in the training data, TGR can capture the general patterns commonly contained in these data sets, namely the mapping relationships between the axioms and rules modeled in the domain ontology under logical reasoning.

[0079] Figure 8 In this context, the input is RDF graph data generated based on the domain ontology. Then, it is partitioned into multi-level subgraphs, and a training dataset for the neural network is constructed, which is a set of multi-level subgraphs. Each training data point contains three parts: the multi-level subgraph of the instance. , negative example set and extended multi-order subgraphs During the training process, firstly based on Generate each instance Class encoding vector C ei and based on [0 , The assertion encoding vectors of instances are generated randomly from a uniform distribution between 1]. Then, using multi-order subgraphs The set of triples in the ABRN is used to update the assertion encoding vector of the instance. Finally, based on the negative example set and extended multi-order subgraphs The triples in the input and supervision labels of ABRN are constructed using the updated instance assertion encoding vector and the instance class encoding vector. The loss value is calculated by inferring the labels and the true labels, and the parameters of the ABRN model are adjusted based on minimizing the loss function.

[0080] The training process of TGR can be found in [link to TGR training process]. Figure 9The process can be roughly divided into three steps. (1) Constructing the training dataset and generating instance class encoding vectors and assertion encoding vectors (lines 2-5). To achieve end-to-end reasoning, TGR training starts with the domain knowledge graph. First, the training data is processed... Perform subgraph partitioning to obtain instances. k - The subgraphs and extended subgraphs are then constructed, and the sets of true negative assertions and unknown assertions corresponding to the subgraphs are added to the training data (see...). Figure 10 Then, according to Generate a class encoding vector for each instance, which is constant and remains unchanged; generate an assertion encoding vector for each instance randomly using a uniform distribution, and use it as the initial input to the neural network. (2) Update the assertion encoding vector of the instance (lines 8-14). Iterate through the instances respectively. k - Rank diagram In the class assertions and relation assertions, update the assertion encoding vector of the instance, and the parameters of ABRN remain unchanged during this process. (3) Update the weight parameters of ABRN (lines 16-25). Traverse the extended subgraph respectively. and negative example set The class assertion and relation assertion are used to calculate the loss function of ABRN based on the updated instance assertion encoding vector, and the weight parameters of ABRN are updated through gradient descent. During this process, the instance assertion encoding vector remains unchanged. The above processes (2) and (3) are repeated until the set number of training rounds or the loss function value converges. Through such training iteration, the TGR model is trained.

[0081] In one embodiment, the pre-constructed set of negative examples is constructed by: detecting contradictory assertions between the multi-order subgraph and the axioms of the chained domain ontology using a symbolic inference engine, and treating the contradictory assertions as true negative examples; enumerating the triples that instances in the multi-order subgraph may form based on the domain and range of each relation in the chained domain ontology, excluding triples that already exist in the multi-order subgraph and the extended subgraph, and treating the remaining triples as unknown negative examples; and merging the true negative examples and the unknown negative examples to obtain the set of negative examples.

[0082] exist Figure 10 The algorithm shown details the process of constructing the subgraph negative assertion set and the unknown assertion set. From the algorithm, it can be seen that the input of the TGR model includes four types of assertions: (1) Input subgraph The assertions in the text serve as the premises for known assertions and inferences; (2) Extended subgraphs The assertion in the statement is a reasoning triple that is explicitly obtained through logical reasoning; (3) True negative example NegSet The assertion in the middle is an assertion that has a clear contradiction under logical reasoning and has a real semantic meaning; (4) randomly constructed UnknownSetti Assertions in TGR are assertions whose correctness cannot be definitively determined during the current logical reasoning process. Therefore, the TGR model fully adheres to the open-world assumption. However, in the TGR reasoning process, this paper primarily considers the acquisition of extended subgraphs; thus, true negative example assertions and the set of unknown assertions are grouped together as a broad set of negative examples. If obtaining true negative example assertions is required in the reasoning scenario, the two need to be distinguished.

[0083] TGR achieves chained domain knowledge graph reasoning through the computation of the ABRN network, the process of which is as follows: Figure 11 As shown. The input to TGR includes: the domain knowledge graph to be reasoned. Domain Ontology Onto The trained neural network model ABRN, and the order of the subgraph. k Instance assertion of the dimension of the encoded vector d Reconstructing threshold t (The refactoring threshold consists of two parts: one is the threshold used for refactoring class assertions.) t 1. Another is the threshold for reconstructing relation assertions. t 2) Parameters that control the number of iterations for updating the instance encoding vector. h The output of TGR is an extended domain knowledge graph. .

[0084] The reasoning process of TGR can be divided into three steps, as follows: (1) Construct the input sub-graph set and initialize the class encoding vector of the instance (corresponding to Figure 11 (Lines 2-3 of the algorithm). First, based on the order of the input subgraph... k The domain knowledge graph to be reasoned Divided into instances k - Rank diagram , forming the input sub-graph set SubgraphSet Unlike the training process, inference only requires the input subgraph and does not need to obtain the extended subgraph. Then, based on... The known triples in the domain ontology are used to construct the class assertion encoding vector for each instance. This vector is a one-hot vector representing the class concept to which the instance belongs in the domain ontology, with the order of the class concepts remaining the same as during training. The class encoding vector is constant and is only used as known information during inference; it is not updated.

[0085] (2) Instance assertion encoding vector initialization and update iteration process (corresponding to) Figure 11 (Lines 5-13 of the algorithm). Based on [0 , The uniform distribution between 1] is used to randomly initialize one for each instance in the subgraph. d dimensional assertion encoding vector The assertion encoding vectors of randomly generated instances are chaotic and do not contain structural and semantic information about the subgraph. Therefore, utilizing the input subgraph... The assertion encoding vector of an instance is updated using the known triples and parameters of the neural network. First, the class assertions in the subgraph are traversed, and the instance encoding is updated according to Equations 9-12. Then, the relation assertions in the subgraph are traversed, and the assertion encoding vectors corresponding to the head and tail instances in the triples are updated according to Equations 1-8. An instance may appear in multiple assertions; by traversing the class and relation assertions, the assertion encoding vector of each instance can be updated multiple times. This process is iterated repeatedly. h This ensures that the assertion encoding vector for each instance contains as much structural and semantic information as possible from the graph. It is important to note that the same instance is encoded independently in different subgraphs.

[0086] (3) Using ABRN neural network to infer class assertions and relational assertions. Based on updating the instance assertion encoding, implicit assertions are inferred through ABRN neural network. First, the subgraph is traversed. The instances in the array, each instance's assertion encoding vector e inst and class encoding vector C inst Since these are known, they are directly input into the CANN network to obtain the extended class assertion encoding vector, and then decoded using the method (see...). Figure 12 Algorithm) Reconstruction about instances inst The class assertion is added to the extended subgraph. Then, to improve the recall of TGR inference results, it is necessary to construct all possible combinations of relation assertions (see...). Figure 13 The algorithm takes all the triples and feeds them into the RANN network, then determines whether each triple is true. The RANN outputs the probability of each relation assertion being true if it exceeds a set threshold. t If the value is 2, it is considered correct and added to the extended subgraph. It is worth noting that, unless otherwise specified, the default settings apply. t = t 1= t 2. Otherwise, set appropriate thresholds according to the different requirements of class assertions and relational assertions. t 1 and t 2.

[0087] By traversing all subgraphs and repeatedly performing the above processes (2) and (3), the extended subgraphs corresponding to all subgraphs are finally obtained. The collection of extended subgraphs is the extended domain knowledge graph under ontology reasoning. TGR cleverly integrates graph data generation, knowledge graph representation learning, knowledge representation and reasoning, neural networks and other technologies into a unified framework for OWL DL domain ontology reasoning, which is expected to achieve fast, high-quality and highly generalizable multi-hop reasoning in domain knowledge graphs.

[0088] In one embodiment, merging the extended subgraphs of each multi-level subgraph to be reasoned and outputting the extended domain knowledge graph of the domain knowledge graph to be reasoned includes: performing cross-subgraph deduplication on the extended subgraphs of each multi-level subgraph to be reasoned, completing the implicit chain assertions across subgraphs according to the chain-based reasoning axioms in the chained domain ontology, merging the deduplicated assertions with the completed chain assertions, and outputting the extended domain knowledge graph of the domain knowledge graph to be reasoned.

[0089] The effectiveness of the method of this invention is verified through experiments. For three ontologies—FAMILY, SEM, and TIME—Neural network models are trained on training data using TGR and CFR, respectively. For each ontology, TGR and CFR use the same training and test sets. The inference accuracy of the two methods (…) are compared. Precision ), recall rate ( Recall )and F The results for the value of 1 are shown in Table 1.

[0090] Table 1. Inference precision, recall, and... F The result of a value of 1

[0091] The performance of TGR was analyzed from the perspective of different triplet types. Table 1 shows that for class assertions, TGR achieved slightly better performance than CFR. For example, for SEM, the precision of TGR and CFR were 1.0 and 0.9434, respectively, and the recall rates were 0.9960 and 0.9630, respectively. F The accuracy values ​​were 0.9980 and 0.9531, respectively; the precision for TIME, TGR, and CFR were 1.0 and 0.9884, respectively, and the recall was 1.0 and 0.9991, respectively. F The I values ​​are 1.0 and 0.9938, respectively. It can be seen that TGR achieves slightly better performance on both SEM and TIME ontologs. For FAMILY, TGR (0.9812) has higher inference precision than CFR (0.8469), but TGR (0.9183) has lower recall than CFR (0.9527). F The value (0.9487) is higher than that of CFR. FA score of 1 (0.8967) indicates that TGR performs slightly better overall. In the three chained domain knowledge graph reasoning experiments mentioned above, TGR outperforms CFR in class assertion reasoning. Notably, TGR achieved nearly perfect and complete results on the SEM and TIME ontologs.

[0092] Regarding relation assertions, Table 1 shows that TGR's overall performance is superior to or significantly superior to CFR across all ontologies, especially in recall, where TGR demonstrates superior performance. On the three ontologies, TGR's precision is 0.9073, 0.9579, and 0.9150, while CFR's precision is 0.9388, 0.9531, and 0.9545. Except for the SEM ontology, where TGR's inference precision is slightly higher than CFR's, CFR performs slightly better on the other two ontologies. However, on the three ontologies, TGR's recall is 0.9477, 0.9804, and 0.9378, while CFR's is only 0.5890, 0.6377, and 0.4897. This indicates that CFR obtains only a smaller portion of the inference relation assertions, but its inference assertion precision is higher. This situation arises primarily because CFR only learns the logical deduction process of a small portion of the axioms in the domain ontology and only outputs the inference assertions corresponding to those axioms, failing to comprehensively learn all the axioms in the domain ontology, especially chained reasoning axioms and rules. Although TGR's inference precision is slightly lower than CFR's, its inference recall is very high, indicating that TGR can learn the complex axioms in chained domain knowledge graph reasoning more completely. Therefore, TGR's overall performance is far superior to CFR, especially across the three ontologs. F The values ​​for TGR are 0.9271, 0.9690, and 0.9263, respectively, while CFR only has 0.7238, 0.7641, and 0.6473. Considering all assertions, TGR's overall performance for reasoning on chained domain knowledge graphs is far superior to CFR. For example, for three ontologies, TGR's... F The values ​​for TGR were 0.9276, 0.9701, and 0.9268, respectively, while CFR only had 0.77, 0.8278, and 0.6685. In fact, TGR's inference accuracy is slightly lower than CFR's; this can be improved by adjusting the reconstruction threshold of TGR. t This can compensate for the shortcomings. Overall, TGR is superior to CFR in terms of the quality of reasoning for knowledge graphs in chained domains.

[0093] It should be understood that, although Figure 1 The steps in the flowchart are shown sequentially as indicated by the arrows, but these steps are not necessarily executed in the order indicated by the arrows. Unless otherwise specified herein, there is no strict order in which these steps are executed, and they can be performed in other orders. Figure 1At least some of the steps in the process may include multiple sub-steps or multiple stages. These sub-steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these sub-steps or stages is not necessarily sequential, but can be executed in turn or alternately with other steps or at least some of the sub-steps or stages of other steps.

[0094] In one embodiment, a neural symbolic multi-hop reasoning system oriented towards domain knowledge graphs is provided, comprising: The subgraph extension module is used to obtain the domain knowledge graph and the chained domain ontology, divide the domain knowledge graph into multi-level subgraphs, and perform symbolic reasoning on each multi-level subgraph based on the chained domain ontology to obtain the corresponding extended subgraph. The instance encoding module is used to encode instances in the training samples by taking each multi-order subgraph as training samples, and to obtain class encoding vector and assertion encoding vector. The class encoding vector is a one-hot constant vector based on ontology class assertion, and the assertion encoding vector is a low-dimensional dense continuous vector. The network construction module is used to build the assertion set inference network, which includes a class assertion neural network and a relation assertion neural network corresponding to each relation type of the chained domain ontology. The class assertion neural network is used to process the concatenation result of the class encoding vector and the assertion encoding vector of the instance, and output the extended class assertion encoding vector. The relation assertion neural network is used to process the concatenation result of the assertion encoding vectors of the subject instance and the object instance in the relation assertion, and output the probability of the relation assertion being true. The network training module is used to construct supervision labels using the extended subgraph and the assertions in the pre-constructed negative example set as positive and negative examples, respectively. The class encoding vector of the training samples and the initial assertion encoding vector are input into the assertion set inference network. The loss is calculated based on the difference between the output and the supervision label. The assertion encoding vector and network parameters are iteratively updated to obtain the trained assertion set inference network. The reasoning generation module is used to divide the knowledge graph of the domain to be reasoned into multi-level subgraphs to be reasoned, encode each instance in the multi-level subgraph to be reasoned, and then input it into the trained assertion set reasoning network. The module decodes the output extended class encoding vector to obtain the extended class assertion of the instance. Based on the probability of the output relation assertion being true and the preset threshold, the module determines the true relation assertion. Based on the extended class assertion and the true relation assertion, the module outputs the corresponding extended subgraph. The results output module is used to merge the extended subgraphs of each multi-level subgraph to be reasoned and output the extended domain knowledge graph of the domain knowledge graph to be reasoned.

[0095] Specific limitations regarding the neural symbolic multi-hop reasoning system for domain-oriented knowledge graphs can be found in the limitations of the neural symbolic multi-hop reasoning method for domain-oriented knowledge graphs mentioned above, and will not be repeated here. Each module in the aforementioned neural symbolic multi-hop reasoning system for domain-oriented knowledge graphs can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in or independent of the processor in a computer device, or stored in the memory of a computer device as software, so that the processor can call and execute the operations corresponding to each module.

[0096] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of the invention. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this application should be determined by the appended claims.

Claims

1. A neural symbolic multi-hop reasoning method oriented towards domain knowledge graphs, characterized in that, The method includes: Obtain a domain knowledge graph and a chained domain ontology; divide the domain knowledge graph into multi-level subgraphs; and perform symbolic reasoning on each multi-level subgraph based on the chained domain ontology to obtain the corresponding extended subgraph. Each multi-order subgraph is used as a training sample, and instances in the training samples are encoded to obtain a class encoding vector and an assertion encoding vector; the class encoding vector is a one-hot constant vector based on ontology class assertions, and the assertion encoding vector is a low-dimensional dense continuous vector. An assertion set inference network is constructed, comprising a class assertion neural network and a relation assertion neural network corresponding to each relation type of the chained domain ontology. The class assertion neural network is used to process the concatenation result of the class encoding vector and the assertion encoding vector of the instance, and output an extended class assertion encoding vector. The relation assertion neural network is used to process the concatenation result of the assertion encoding vectors of the subject instance and the object instance in the relation assertion, and output the probability of the relation assertion being true. Supervisory labels are constructed using the extended subgraph and the assertions in the pre-constructed negative example set as positive and negative examples, respectively. The class encoding vector of the training samples and the initial assertion encoding vector are input into the assertion set inference network. The loss is calculated based on the difference between the output and the supervisory label. The assertion encoding vector and network parameters are iteratively updated to obtain the trained assertion set inference network. The knowledge graph of the reasoning domain is divided into multi-level subgraphs to be reasoned, and each instance in the multi-level subgraph to be reasoned is encoded and then input into the trained assertion set reasoning network. The extended class encoding vector of the output is decoded to obtain the extended class assertion of the instance. The established relation assertion is determined according to the probability of the output relation assertion and the preset threshold. The corresponding extended subgraph is output according to the extended class assertion and the established relation assertion. Merge the extended subgraphs of each multi-level subgraph to be reasoned, and output the extended domain knowledge graph of the domain knowledge graph to be reasoned.

2. The method according to claim 1, characterized in that, The steps of acquiring a domain knowledge graph and a chained domain ontology, dividing the domain knowledge graph into multi-level subgraphs, and performing symbolic reasoning on each multi-level subgraph based on the chained domain ontology to obtain the corresponding extended subgraph include: Based on the ontology axioms corresponding to the chained domain ontology parsing, the longest relation chain length of the reasoning task is dynamically determined, and the subgraph order is calculated according to the longest relation chain length and the preset mapping rules. Centered on each instance in the domain knowledge graph, k-order subgraphs are divided according to the order of the subgraph. The k-order subgraphs are then used in conjunction with the ontology axioms to perform inference on the symbolic inference engine, generating an extended subgraph containing implicit assertions.

3. The method according to claim 1, characterized in that, The mapping rules include the subgraph order and the length of the longest relation chain satisfying the following conditions. ,in, Let the order be the subgraph. The length of the longest relationship chain. This is for rounding up.

4. The method according to claim 1, characterized in that, The process of encoding instances in the training samples to obtain class encoding vectors and assertion encoding vectors includes: The dimension of the class encoding vector is determined based on the total number of class concepts in the chained domain ontology. If an instance has a class assertion belonging to a class concept in the subgraph, the value of the class concept dimension corresponding to the class encoding vector is 1, and the value of the other class concept dimensions is 0. The assertion encoding vector is randomly initialized, and the dimension of the assertion encoding vector is a preset fixed value, which is consistent with the dimension of the hidden layer in the assertion set inference network.

5. The method according to claim 1, characterized in that, The concatenation result of the instance's class encoding vector and assertion encoding vector is processed to output the extended class assertion encoding vector, which includes: The assertion encoding vector of the instance is concatenated with the class encoding vector to obtain the first input vector; The gating vector and candidate update vector of the first input vector are calculated by the neural network layer of the assertion neural network. The candidate update vector and the gating vector are multiplied element-wise and then added to the assertion encoding vector to obtain the intermediate vector. The intermediate vector is normalized by L2 norm to obtain the updated assertion encoding vector. The updated assertion encoding vector is processed by the fully connected layer of the assertion neural network to output an extended assertion encoding vector.

6. The method according to claim 1, characterized in that, The concatenation result of the assertion encoding vectors of the subject instance and object instance in the relation assertion is processed to output the probability of the relation assertion being true, including: The assertion encoding vectors of the subject instance and the object instance in the relation assertion are concatenated to obtain the second input vector; The relational neural network layer of the relational assertion neural network updates the assertion encoding vectors of the subject instance and the object instance in the second input vector respectively, to obtain the updated subject assertion encoding vector and object assertion encoding vector. The concatenation result of the updated subject assertion encoding vector and object assertion encoding vector is processed by the fully connected layer of the relation assertion neural network to output the probability of the relation assertion being true.

7. The method according to claim 1, characterized in that, The pre-built set of negative examples is constructed by the following process: The contradictory assertions between multi-order subgraphs and chained domain ontology axioms are detected by symbolic inference engine, and the contradictory assertions are treated as true negatives. Based on the domain and range of each relation in the chained domain ontology, enumerate the triples that instances in the multi-order subgraph may form, exclude the triples that already exist in the multi-order subgraph and the extended subgraph, and treat the remaining triples as unknown negative instances. The true negative examples and the unknown negative examples are merged to obtain the negative example set.

8. The method according to claim 1, characterized in that, The step of calculating the loss based on the difference between the output and the supervision label, iteratively updating the assertion encoding vector and network parameters to obtain the trained assertion set inference network includes: The supervision labels include class assertion supervision labels and relation assertion supervision labels; The parameters of the current assertion set inference network are fixed. The loss is calculated by expanding the difference between the class assertion encoding vector and the class assertion supervision label, and the difference between the probability of the relation assertion and the relation assertion supervision label. The assertion encoding vector of the instance is iteratively updated according to the loss until the first iteration stopping condition is met, and then the iteration stops. The loss is recalculated by re-inputting the updated assertion encoding vector into the network. The network parameters of the assertion set inference network are iteratively updated using the gradient descent method until the second iteration stopping condition is met, at which point the iteration stops, and the trained assertion set inference network is obtained.

9. The method according to claim 1, characterized in that, The process of merging the extended subgraphs of each multi-level subgraph to be reasoned, and outputting the extended domain knowledge graph of the domain to be reasoned, includes: For each multi-level subgraph to be reasoned, perform cross-subgraph deduplication on the extended subgraph. Based on the chain reasoning axiom in the chained domain ontology, complete the implicit chain assertions across the subgraph. Merge the deduplicated assertions with the completed chain assertions to output the extended domain knowledge graph of the domain knowledge graph to be reasoned.

10. A neural symbolic multi-hop reasoning system oriented towards domain knowledge graphs, characterized in that, The system includes: The subgraph extension module is used to acquire a domain knowledge graph and a chained domain ontology, divide the domain knowledge graph into multi-level subgraphs, and perform symbolic reasoning on each multi-level subgraph based on the chained domain ontology to obtain the corresponding extended subgraph. The instance encoding module is used to encode instances in the training samples using each multi-order subgraph as training samples, to obtain class encoding vectors and assertion encoding vectors; the class encoding vectors are one-hot constant vectors based on ontology class assertions, and the assertion encoding vectors are low-dimensional dense continuous vectors. A network construction module is used to construct an assertion set inference network, which includes a class assertion neural network and a relation assertion neural network corresponding to each relation type of the chained domain ontology. The class assertion neural network is used to process the concatenation result of the class encoding vector and the assertion encoding vector of the instance, and output an extended class assertion encoding vector. The relation assertion neural network is used to process the concatenation result of the assertion encoding vectors of the subject instance and the object instance in the relation assertion, and output the probability of the relation assertion being true. The network training module is used to construct supervision labels using the extended subgraph and the assertions in the pre-constructed negative example set as positive and negative examples, respectively. The class encoding vector of the training samples and the initial assertion encoding vector are input into the assertion set inference network. The loss is calculated based on the difference between the output and the supervision label. The assertion encoding vector and network parameters are iteratively updated to obtain the trained assertion set inference network. The reasoning generation module is used to divide the knowledge graph of the domain to be reasoned into multi-level subgraphs to be reasoned, encode each instance in the multi-level subgraph to be reasoned, and input it into the trained assertion set reasoning network. It decodes the output extended class encoding vector to obtain the extended class assertion of the instance, determines the valid relation assertion based on the probability of the output relation assertion and a preset threshold, and outputs the corresponding extended subgraph based on the extended class assertion and the valid relation assertion. The results output module is used to merge the extended subgraphs of each multi-level subgraph to be reasoned and output the extended domain knowledge graph of the domain knowledge graph to be reasoned.

Images