A knowledge graph-based large model logic reasoning enhancement method and system

Abstract

This invention relates to a method and system for enhancing logical reasoning in large-scale models based on knowledge graph construction. The method includes: acquiring a set of natural language premises and translating it into a set of symbolic language; acquiring a target query set to be solved and constructing annotation prompts for the reasoning task based on the symbolic language set and the target query set; annotating the natural language premise set with knowledge elements based on the annotation prompts; wherein the knowledge element annotation includes at least entity annotation, relation annotation, and attribute annotation; constructing a typified knowledge graph based on the knowledge element annotation results and generating an enhanced fact set based on the typified knowledge graph; and inputting the enhanced fact set and the symbolic language set into a logic solver to perform logical reasoning on the target query set. This invention belongs to the field of large language model technology, solving the problem of low accuracy in logical reasoning in existing technologies and improving the accuracy and reliability of symbolic reasoning results in large-scale models.

Description

Technical Field

[0001] This invention relates to the field of large language model technology, and in particular to a method and system for enhancing logical reasoning in large models based on knowledge graph construction. Background Technology

[0002] Large language models demonstrate strong capabilities in natural language understanding, question answering, and complex text generation tasks. However, in natural language reasoning tasks requiring strict logical constraints, they are still prone to problems such as unstable reasoning chains, unverifiable conclusions, and fidelity of intermediate steps. To improve the reliability of logical reasoning, a class of neurosymbolic reasoning methods has emerged in recent years. These methods first use large language models to translate natural language premises and queries into first-order logic, conjunctive normal form, Datalog, SAT / SMT constraints, or other symbolic languages ​​(SL), and then use symbolic solvers to perform formal reasoning. This type of method can leverage the determinism and verifiability of solvers to alleviate the problem of "seemingly reasonable but not conforming to formal logic" in pure natural language reasoning.

[0003] Existing methods typically treat natural language to symbolic language translation as a separate preprocessing step. In this step, a large language model or semantic parser generates constants, predicates, and logical rules from the natural language text. For example, the natural language sentence "All novels are books" can be translated into... The sentence "Walden is a novel" can be translated as: Subsequently, the solver determines whether the target conclusion can be derived based on these symbolic representations. Representative works of this type include neural symbolic reasoning frameworks that combine language models and first-order logic provers, and frameworks that combine large language models with symbolic solvers to improve faithful logical reasoning capabilities.

[0004] However, information compression and loss are inevitable in the translation from natural language to symbolic language. Entity aliases, synonyms, hypernyms, attribute modifiers, common-sense relationships, and contextual dependencies in natural language may not all be explicitly translated into the final symbolic language expression. Especially when the translation stage limits the predicate vocabulary, predicate number, and formula templates to reduce solution complexity, the translator often only retains components that can directly participate in the main logic formula, ignoring supplementary information related to entity and relational semantics. As a result, although the symbolic language formula received by the solver is formally valid, its semantic context becomes impoverished. If the target query requires entity relations or attribute information discarded during the translation process, the solver may be unable to complete the proof or may need to perform a large number of invalid searches.

[0005] Another related technology is knowledge graph construction and relation extraction. Relation labeling, named entity recognition (NER), and attribute labeling are usually performed directly on natural language text, thus making good use of semantic cues and contextual information in natural language. For example, the model can extract relation annotations from the statement "Walden is a collection of essays by Thoreau, often classified as a literary work." , , Information such as these. Compared to directly recovering semantics from compressed symbolic language formulas, labeling relations and entity types on the original NL text is generally more accurate. However, traditional knowledge graph construction methods have another problem: their labeling system is not consistent with the symbolic vocabulary of downstream logical reasoning, and the extracted relations may not be directly used by the solver, or may require additional manual mapping. Summary of the Invention

[0006] Based on the above analysis, the embodiments of the present invention aim to provide a method and system for enhancing logical reasoning of large models based on knowledge graph construction, in order to solve the problem of low accuracy of logical reasoning caused by the loss of semantic information in the translation process from natural language to symbolic language in the prior art.

[0007] On one hand, embodiments of the present invention provide a method for enhancing logical reasoning in a large model based on knowledge graph construction, comprising: obtaining a set of natural language premises and translating the set of natural language premises into a set of symbolic language; obtaining a target query set to be solved and constructing annotation prompts for the reasoning task based on the symbolic language set and the target query set; annotating the natural language premise set with knowledge elements based on the annotation prompts; wherein the knowledge element annotation includes at least entity annotation, relation annotation, and attribute annotation; constructing a typified knowledge graph based on the knowledge element annotation results and generating an enhanced fact set based on the typified knowledge graph; and inputting the enhanced fact set and the symbolic language set into a logic solver to perform logical reasoning on the target query set.

[0008] Furthermore, translating the natural language premise set into a symbolic language set includes: for each natural language premise sentence in the natural language premise set, calling a large model to perform structured translation, summarizing the structured translation results of each natural language premise sentence to obtain the symbolic language set; wherein, the symbolic language set includes at least: a set of constant symbols, a set of predicate symbols, and signature information for each predicate.

[0009] Furthermore, based on the symbolic language set and the target query set, labeled prompts for reasoning tasks are constructed, including: for each natural language premise sentence, candidate constants related to the sentence and the target query set are selected from the constant symbol set, and candidate predicates related to the sentence and the target query set are selected from the predicate symbol set; each predicate in the symbolic language set and its corresponding signature information are summarized to construct a predicate signature set; wherein, the signature information includes at least the number of signature elements and parameter types; and labeled prompts for each natural language premise sentence are constructed based on the candidate constants, candidate predicates, and the predicate signature set.

[0010] Furthermore, the annotation prompts for each natural language premise sentence include: the corresponding candidate constant, the corresponding candidate predicate, the predicate signature set, the target query set, and the format constraints of the annotation output.

[0011] Furthermore, knowledge element annotation is performed on the natural language premise set based on annotation prompts, including: inputting annotation prompts into the large model, and performing knowledge element annotation on the natural language premise set according to the following constraints: entity nodes obtained from entity annotation are selected from the constant symbol set or aligned with the constant symbol set; predicates in relation triples obtained from relation annotation are selected from the predicate symbol set or aligned with the predicate symbol set; entity type triples and entity attribute triples obtained from attribute annotation satisfy the constraints of the predicate signature set; and the annotation results are output according to the format constraints.

[0012] Furthermore, a typified knowledge graph is constructed based on the knowledge element annotation results, including: writing entity nodes, relation triples, entity type triples, and entity attribute triples into the typified knowledge graph; creating constant-constant edges based on relation triples, creating constant-predicate edges based on entity type triples and entity attribute triples, and extracting logical rules from the natural language premise set to create predicate-predicate edges.

[0013] Furthermore, the process of generating an augmented fact set based on a typified knowledge graph includes: generating a candidate augmented fact set using a preset formula template based on the type of edges in the knowledge graph; and filtering the candidate augmented fact set to obtain the augmented fact set.

[0014] Further, the candidate augmented fact set is screened to obtain the augmented fact set, including: performing grammatical and predicate checks on each candidate augmented fact set to obtain a preliminary augmented fact set; performing symbolic language set implication tests on the preliminary augmented fact set to obtain a secondary augmented fact set; calculating benefit-risk scores on candidate augmented facts in the secondary augmented fact set, and selecting the augmented fact set from the secondary augmented fact set based on the benefit-risk scores.

[0015] Furthermore, the benefit-risk score is calculated for the candidate augmenting facts in the augmenting fact set after secondary screening, including: calculating the information gain of the candidate augmenting facts; determining the type of the edge corresponding to the candidate augmenting fact, determining the risk coefficient and risk aversion weight of the candidate augmenting fact based on the edge type, calculating the risk of the candidate augmenting fact based on the risk coefficient and risk aversion weight; and calculating the benefit-risk score of the candidate augmenting fact based on the information gain and risk of the candidate augmenting fact.

[0016] On the other hand, embodiments of the present invention provide a large-scale model logic reasoning enhancement system based on knowledge graph construction, comprising: a translation module for acquiring a natural language premise set and translating the natural language premise set into a symbolic language set; a prompting module for acquiring a target query set to be solved and constructing labeled prompt words for reasoning tasks based on the symbolic language set and the target query set; a labeling module for labeling knowledge elements of the natural language premise set based on the labeled prompt words; wherein, the knowledge element labeling includes at least entity labeling, relation labeling, and attribute labeling; a construction module for constructing a typified knowledge graph based on the knowledge element labeling results and generating an enhanced fact set based on the typified knowledge graph; and a reasoning module for inputting the enhanced fact set and the symbolic language set into a logic solver to perform logical reasoning on the target query set.

[0017] This invention proposes a method for enhancing logical reasoning in large-scale models based on typified knowledge graphs. It uses translated symbolic language and the target query as constraints, regressing to natural language text to perform entity, relation, and attribute annotation for reasoning tasks. The annotation results are then organized into a typified knowledge graph, and augmenting facts that reduce reasoning uncertainty and are risk-controlled are selected from the knowledge graph and added to the solver to complete the logical reasoning for the target query. This method does not simply perform unconstrained knowledge completion after symbolic language translation; instead, it uses symbolic language and the target query as rigid constraints to standardize natural language annotation. While maintaining the rigor of symbolic reasoning, it uses natural language annotation to controllably compensate for translation loss, improving the reasoning accuracy and reliability of large-scale model symbolic reasoning tasks. Furthermore, incorporating query-related facts into the solution shortens the proof path and reduces the number of expansion nodes, thereby lowering the search cost of logical reasoning.

[0018] In this invention, the above-described technical solutions can be combined with each other to achieve more preferred combinations. Other features and advantages of this invention will be set forth in the following description, and some advantages may become apparent from the description or be learned by practicing the invention. The objects and other advantages of this invention can be realized and obtained from what is particularly pointed out in the description and drawings. Attached Figure Description

[0019] The accompanying drawings are for illustrative purposes only and are not intended to limit the invention. Throughout the drawings, the same reference numerals denote the same parts. Figure 1 This is an optional flowchart of a large-scale model logical reasoning enhancement method based on knowledge graph construction, as described in an embodiment of the present invention. Detailed Implementation

[0020] Preferred embodiments of the present invention will now be described in detail with reference to the accompanying drawings, which form part of this application and are used together with the embodiments of the present invention to illustrate the principles of the present invention, but are not intended to limit the scope of the present invention.

[0021] A specific embodiment of the present invention discloses a method for enhancing logical reasoning in large models based on knowledge graph construction, such as... Figure 1 As shown, it includes the following steps: S101: Obtain the set of natural language premises and translate it into a set of symbolic language. The set of natural language premises is a collection of facts, conditions, and rules expressed in natural language. It is the input material for logical reasoning, retains the original natural language expression, and is used to translate and generate a set of symbolic language. S102: Obtain the target query set to be solved, and construct annotation prompts for the reasoning task based on the symbolic language set and the target query set. The target query set consists of one or more queries to be reasoned, verified, or solved. Each target query is a question that the logical reasoning task needs to answer or prove, which can be a fact inquiry, a truth or falsehood judgment, a rule verification, etc. The target query set is used to constrain the generation of annotation prompts. S103: Perform knowledge element annotation on a natural language premise set based on annotation prompts; wherein, knowledge element annotation includes at least entity annotation, relation annotation, and attribute annotation; entity annotation refers to identifying entity mentions from the natural language premise text. Relation annotation refers to identifying semantic relationships between entities in the text. Attribute annotation refers to identifying the type features and attribute information of entities, and outputting entity-type and entity-attribute class annotation results; S104: Construct a typified knowledge graph based on the knowledge element annotation results, and generate an enhanced fact set based on the typified knowledge graph; This invention has two levels of logical enhancement: First level: knowledge element annotation constrained by symbols, constructing a typified knowledge graph, and producing a complete set of candidate enhanced facts; Second level: performing multi-level screening on candidate enhanced facts, outputting a controllable set of enhanced facts, and then sending it to the logic solver; S105: The augmented fact set and the symbolic language set are input into the logic solver to perform logical reasoning on the target query set. The logic solver receives two types of input: first, the symbolic language set of the original translation output, ensuring the rigor of the original formal knowledge; second, the filtered augmented fact set, which comes from the typified knowledge graph and is used to compensate for the semantic information lost in the natural language to symbolic language translation stage. Based on the above two parts of knowledge, the solver completes the logical deduction for the target query set and outputs the corresponding reasoning results and proof information.

[0022] In the above implementation, a method for enhancing logical reasoning in large-scale models based on typified knowledge graphs is proposed. This method uses translated symbolic language and the target query as constraints, regressing to natural language text to perform entity, relation, and attribute annotation for reasoning tasks. The annotation results are then organized into a typified knowledge graph, and augmenting facts that reduce reasoning uncertainty and are risk-controlled are selected from the knowledge graph and added to the solver to complete the logical reasoning for the target query. This method does not simply perform unconstrained knowledge completion after symbolic language translation; instead, it uses symbolic language and the target query as rigid constraints to standardize natural language annotation. While maintaining the rigor of symbolic reasoning, it uses natural language annotation to controllably compensate for translation loss, improving the reasoning accuracy and reliability of large-scale model symbolic reasoning tasks. Furthermore, incorporating query-related facts into the solution shortens the proof path and reduces the number of expansion nodes, thereby lowering the search cost of logical reasoning.

[0023] set up Represents the set of premises of natural language. This represents the target query set. The natural language-symbolic language translator will... Translated into a set of symbolic languages , It can be a set of first-order logic formulas, a set of conjunction normal form clauses, or a set of Datalog-style rules. Contains a set of constant symbols Predicate symbol set and predicate signature set .

[0024] The typified knowledge graph constructed in this invention is represented as follows: ; in, It is a set of constant nodes, corresponding to entities, entity clusters, or normalized individuals in natural language; This is the set of predicate nodes, corresponding to predicates obtained from translation, candidate predicates supplemented from natural language annotations, or normalized attribute types. Edge set. Includes three types of edges: ; in This indicates a constant-constant edge, for example (Walden, writtenBy, Thoreau); Indicates a constant-predicate edge, such as (Walden, isA, Book) or Book(Walden); Indicates predicate-predicate border, such as (Novel, subclassOf, Book) or .

[0025] First, the system sets the natural language premises. Performing structured translation yields a set of symbolic languages. The translation process includes entity extraction, identity resolution, constant name normalization, predicate pattern induction, and logical rule generation. To facilitate solver processing, constants and predicates are preferably named using ASCII standard names, and a mapping table from natural language surface forms to symbolic names is maintained. .

[0026] The above translation process is completed sentence by sentence by a large language model (using few-sample prompts) or a finely tuned semantic parser. Each premise sentence The model outputs five structured results, including: (1) entity extraction and constant name normalization (e.g., “Walden” → Walden); (2) homoreference resolution, merging different references to the same object into the same constant; (3) predicate pattern induction, giving the predicate name, arity, and parameter types (e.g., ... (4) Logical rule generation, which is a first-order logical clause or Datalog rule generated by logical operators such as quantifiers, implication, and negation in the sentence; for example, outputting a rule for "all novels are books". To present facts regarding the claim that "Walden is a novel". Providing 2-3 such input-output examples in the prompts is sufficient to reliably reproduce the translation; the logical rules generated in this step serve as a symbolic language set. The components, and at the same time, when constructing the typed knowledge graph, the predicate implication, subclass, equivalence or mutual exclusion relation in the logical rules are resolved into predicate-predicate edge; (5) the mapping of natural language surface form to symbol name is written into the mapping table. For example, "Walden" can be uniformly mapped to the constant Walden, and "novel" can be uniformly mapped to the predicate Novel.

[0027] The translation process not only outputs a set of constant symbols Predicate symbol set It also outputs a set of predicate signatures. : ; Used to record the calling specification for each predicate: for the first individual predicates , The number of its parameters (element number). For its first The types allowed for each parameter. For example... . Its purpose is to serve as a constraint for subsequent natural language annotation; annotations can only use... The predicates in the text must be extracted, and the number and type of parameters extracted must be consistent with the predicate signature to ensure that the annotation results can be directly used by the logic solver. Therefore, in this invention, knowledge element annotation is still performed on natural language text to fully utilize the semantic information of natural language. However, knowledge element annotation is subject to... The constraints enable the annotation results to be adapted to downstream inference.

[0028] In obtaining Then, the system constructs knowledge element annotation prompts for reasoning tasks. These prompts include four types of information: First, the target query set. The constants and predicates appearing in the annotation process are used to guide the annotation process to prioritize entities and relations related to the conclusion to be proven; secondly, There is already a set of constant symbols in the middle. and predicate symbol set The first part is used as entity and relation candidates; the second part is the number of predicates and parameter types, i.e., predicate signatures, which are used to constrain the number and type of parameters in the extracted results; the third part is the local context extracted from the natural language premises, which is used to preserve the original semantics of the natural language.

[0029] The above four types of information will be assembled into a fixed-structure prompt, as shown in the template example below (square brackets are placeholders): "Task: Label entities, relation triples, and entity types in a given sentence; [Target query related symbols]: Target query set" Constants and predicates appearing in the text; [Usable constants]: [Available predicates and signatures]: and its atom / parameter types (taken from) ); [Sentence to be annotated]: and its local context; [Output Format]: Output only JSON, with fields entities, relations, and types. Relationship and type labels can only be taken from the predicates given above. A completed example: query = "Author exists?"; available constants = {Walden}; available predicates = ; Sentence = "Thoreau wrote Walden"; Model output

[0030]

[0031]

[0032] The rule for extracting local context from natural language premises is as follows: first locate the sentence in which the candidate entity is mentioned, then include it in the context in the following order until the token budget limit is reached. (1) The sentence itself; (2) Sentences that have a reference chain with the sentence (referring to the same entity); (3) Sentences containing the target query set. Medium entity / predicate or constant set Sentences with constant values; (4) Sentences before and after the constant value; Sentence (default) =1~2). The purpose of truncation is to retain the minimum context necessary to determine the relationship of the entity without exceeding the token budget. , Sentences that are irrelevant and have no referential relationship are not included.

[0033] As an optional implementation, the constrained prompt word can be represented as: ; in, The sentence is a natural language sentence to be labeled. and These are constant candidates and predicate candidates related to the sentence and the query, respectively. Output format constraints.

[0034] The rule for determining "relevance" is: for constants... If and only if its canonical name or any alias appears in the sentence In the middle, or Appears in the target query set The same applies to predicates. Therefore, from this... , Filter out subsets related to the sentence and the query. , This avoids the problem of cramming the entire word list into the prompt words, which can cause interference and excessively long prompt words.

[0035] The constrained prompt is used as follows: input it as a whole into the annotation model, and limit the output with three hard constraints: (1) The relation can only be taken from Entities can only be aligned to Or create a new candidate; (2) The number of parameters and the type of each relation / instance type triple must match. (3) The output must conform to the format constraints. (e.g., specifying JSON mode). Any output that violates the above constraints will be considered invalid and will trigger a regeneration or be discarded directly, thus ensuring that the annotation results naturally fall within the vocabulary of symbols available to the solver.

[0036] The annotation results output format can be configured to produce three types of results in JSON or tabular form: entity recognition (entity annotation results), entity relation triples (relation annotation results), and entity type / attribute annotations (attribute annotation results). Because the prompts use a symbolic language vocabulary, the annotation model will prioritize using predicates that can be recognized by the solver, such as Novel, Book, Person, madeOf, causes, and partOf, rather than generating free text labels that cannot be directly used downstream.

[0037] Compared to directly performing knowledge completion on the translated symbolic language, the annotation task of this invention is itself a natural language understanding task, which can typically capture more semantics when performed within the original natural language text. For example, in natural language, "Work A is classified as a novel and was created by B" simultaneously includes genre classification and authorship relationship; if the translator only generates... ,but It may be lost. This invention utilizes cue words containing symbolic language to re-annotate natural language, which can recover the symbolic language relations that are useful for reasoning.

[0038] After annotation, a typified knowledge graph is constructed, specifically including: The annotation results output by the annotation model are uniformly written into the typed knowledge graph. Regarding the relation labeling results The system checks relationship tags. Does it exist in Or whether it can be mapped to For predicates, if they exist or can be mapped, create constant-constant edges (such as binary relations like createdBy, writtenBy, partOf, and causes). For entity type or attribute annotations... The system creates constant-predicate attribution edges (isA / type / attribute attribution, such as Book(Walden)). In addition to the above annotation results, logical rules are extracted from the natural language premise set. For semantics expressed in natural language such as "all A are B", "A belongs to a kind of B", "if A is satisfied, then B is satisfied", the system creates predicate-predicate edges (subclassOf, implies, equivalentTo, disjointWith, etc.).

[0039] Among them, inspection Does it exist in Or can be mapped to The specific method is as follows: First, perform an exact name match (including case sensitivity and comparison after ASCII normalization); if no match is found, then... Perform word form restoration, bilingual dictionary mapping, and then calculate... and The cosine similarity of the embedding vectors of each predicate name is taken, and the highest similarity is considered; when the highest similarity is greater than or equal to the threshold... (Default 0.85) If the signatures of the two primitives / parameter types are consistent, they are determined to be mappable and merged into the existing predicate; otherwise, they will be... Record them as candidate new predicates.

[0040] To avoid the same entity or predicate being represented by multiple names, the above mapping table is used. Redundancy processing is performed. If the natural language surface form... With existing constants If the similarity, consensus resolution confidence, or contextual consistency exceed the threshold, then... merge into Otherwise, a new constant node is created. For predicate nodes, merging is also performed through lexical normalization, bilingual mapping, predicate signature consistency, and target query relevance. After the graph is constructed, the system calculates graph features for each node and edge, including node degree, shortest path to constants or predicates in the target query, edge label confidence, predicate centrality, and edge relationship type.

[0041] Target query set The constants and predicates in the graph do not appear in the graph only because of the annotations, but rather during the initialization phase of building the structured knowledge graph, that is, when the constants and predicates are defined in the graph. The parsed constants, predicates and The symbol in the code is directly injected into the node set as a "seed node". If a certain... If a node has no labeled evidence in the natural language, it is an isolated node, and its shortest path with other nodes is denoted as ∞ (in practice, this is represented by a large constant). This path itself is a useful feature, indicating that the query component lacks supporting information and needs to be supplemented.

[0042] Existing technologies for logical reasoning typically employ two approaches: the first directly extracts entity relations from natural language text to construct a knowledge graph, which is then converted into logical clauses and fed into the solver; the second uses neural symbolic translation to translate natural language into first-order logic, Datalog, or ASP symbolic clauses, which are then directly input into the logic solver to complete the reasoning. This invention does not directly convert the translated symbolic language set into a knowledge graph. Instead, it uses the symbolic set parsed from the symbolic language set as constraints, returns to the original natural language premise to perform knowledge element annotation, and constructs a typified knowledge graph based on the annotation results. The knowledge graph and the translated symbolic language set are then used together in the reasoning process to compensate for the semantic loss caused by the natural language-symbolic language translation process, thereby improving the accuracy of the reasoning. Furthermore, traditional named entity recognition and relation extraction models often use general labeling systems, with extraction results independent of the symbolic system of the reasoning task. This often results in extraction results that cannot be accurately mapped to the solver, leading to poor adaptability and low usability. This invention utilizes target querying combined with symbolic language to construct exclusive annotation prompts for reasoning tasks, so that the knowledge element annotation process no longer relies on a general tag system, but is adapted to the current reasoning task, thus solving the problem of incompatibility between general model extraction results and symbolic reasoning.

[0043] This invention further utilizes knowledge graphs Generate candidate augmented fact sets After screening, the following results were obtained. This makes the target query set The reasoning satisfies higher provability and lower solution cost. Formally, if... Indicates the solver in Top Based on the reasoning results and proof statistics, this invention aims to select... Make: ; It also reduces cost metrics such as proof depth, number of expanded nodes, or solution time.

[0044] from Generate a set of candidate augmented facts Candidate augmented facts include both atomic facts at the instance level and logical rule formulas for subclasses, implications, and equivalence classes. Specifically, for constant-constant edges... ,like If it is a binary predicate, then generate ( Meta-relations are thus expanded to For constant-predicate belonging edges); or attribute edge Generate facts or For predicate-predicate edges, generate respectively: , , (By element) → , That is, each predicate type corresponds to a fixed formula template. When adding a new predicate type, you only need to register its template to generate the corresponding logical rules as candidate facts.

[0045] To ensure compatibility between candidate facts and the solver, the system performs syntax and type checks on them. If a candidate fact's predicate does not exist in the allowed vocabulary and cannot be mapped to an existing predicate, it can be added to the candidate extended predicate set, but it will only proceed to subsequent scoring if it has a graph path association with the target query and its type signature is determinable. If the number of candidate fact parameters is inconsistent with the number of predicate elements, or if the parameter types do not conform to the signature constraints, they are discarded directly.

[0046] Syntax and type checks are performed step by step as follows, and any failure results in rejection or downgrading (entering the candidate list): (1) Syntax: Parse the candidate facts into predicates and parameter lists, requiring that they be parsable and that the predicate names are valid; (2) Lexicon: The predicates must exist in the vocabulary. Or can be mapped to (3) Number of elements: The number of parameters must be equal to the number of elements in the predicate signature; (4) Type: The type of each parameter (given by the type annotation of its constant node) must fall within the range of elements in the predicate signature. The corresponding parameter type field of the declaration. If the atom or type does not match, it is discarded directly; if the predicate is not in the vocabulary but is related to... If there are path relationships in the graph and the type can be determined, then add them to the candidate extended predicate set for scoring.

[0047] After determining the candidate augmentation fact set, the candidate augmentation fact set is filtered for novelty.

[0048] For each candidate augmentation fact The system first determines whether it has already been used in the original symbolic language set. Implies: judge ( Whether it is implied by a set of symbolic languages ​​is tested using the implied meaning of the solver: for a refutable backend (FOL prover, SAT / SMT), join in Post-test Is it unsatisfiable? If it is unsatisfiable, then... For the Datalog / ASP backend, direct query Is it possible to obtain from Derived from this. If true, then... Redundancy is rejected; if the condition is not met, it is retained for utility and risk assessment. To control costs, it can be included only in the utility and risk assessment. Perform local implication checks on the relevant set of clauses.

[0049] like This means This is a redundant fact for the solver; adding it to the candidate augmentation fact set will not expand the provable knowledge boundary, therefore it is rejected. If If the result is not satisfactory, it will be retained and included in the utility and risk assessment. This filtering can be implemented using a first-order logic prover, a SAT / SMT backend, or a Datalog query. For example, when... It already contains and At that time, candidates It can be judged as redundant; while if Only contains But lacking The rules, This may not be a redundant fact and requires further evaluation.

[0050] This invention measures inference utility by the reduction of uncertainty in the target query by candidate facts. Specifically, it employs a utility evaluation in the form of information gain.

[0051] Candidate augmentation facts Information gain is defined as: ; in Entropy represents the result of solving the target query or the posterior confidence. If added... If the provability of the target query is improved, the true / false state is more certain, or the proof path is more stable, then... Larger.

[0052] In practical implementations, accurately calculating entropy can be costly. This invention provides three alternative implementations. The first is a variational approximation: Convert to a factor plot, for The marginal probabilities are approximated, and then the entropy difference is calculated. The second method is to prove the cost proxy: respectively in and Run the solver, record the changes in proof depth and number of unfolded nodes, and estimate the information gain using the following formula: ;in Indicates joining Subsequent evidence showed that the depth reduction was significant. This indicates the amount by which the number of expanded nodes is reduced. , The weights are non-negative. The third type is learned estimation: using graph features (shortest paths to constants or predicates in the target query, edge label confidence, predicate centrality) and novelty features to train a regression model. predict By default, a proof cost surrogate is used because it is more efficient and directly compatible with common symbolic solvers because it does not rely on specific probability calculations.

[0053] Since candidate augmented facts are derived from natural language annotations or large language model-assisted annotations, they are not necessarily correct. This invention addresses the issue of candidate augmented facts... Define total risk : ,like For constants—constant facts; ,like For constants—predicate facts; ,like For predicate—predicate fact. (Among them) That is, according to (Only one item is taken from the edge type.) Once a risk falls into a certain category, only that category of risk is multiplied by its corresponding weight. Risk aversion weights are set separately for the three types of edges (for example, predicate-predicate rules have a large impact, so a larger weight can be chosen). (With stricter control). Equivalently, it can be written as... The indicator function 1[·] guarantees that only the expression is related to the function 1[·]. Only one item of the type matches.

[0054] The above This represents the risk of a constant-constant relationship. For binary relationships... The system comprehensively measures model confidence, consistency across multiple samplings, presence of external knowledge base, and compatibility with [other factors]. Based on the contradiction detection results of known facts, estimate the probability that the relationship is not valid or unreliable, and obtain... If multiple annotations yield the same relationship, and this does not conflict with existing facts, then... Lower.

[0055] The four evaluation parameters were obtained as follows: (1) Labeled model confidence—the log probability or self-consistency rate of the model output relation; (2) Consistency of multiple sampling—multiple sampling of the same sentence, and statistical analysis of the proportion of triples appearing in the relation; (3) Existence check of external knowledge base—using Searching the domain knowledge base returns 1 if a match is found, 0 if no match is found, or a matching score; (4) with Contradiction detection—incorporating this relationship The solver then determines whether a conflict has occurred. The four terms are normalized and then weighted or synthesized using a logistic function to obtain the probability that the relation is not true / unreliable. When multiple annotations are consistent and do not conflict with existing facts. Lower.

[0056] This indicates the risk of constant-predicate attribution. For System check Does it meet the requirements? The parameter type domain must be considered, whether it conflicts with existing type facts, and whether the natural language context supports the attribution. For example, if the parameter type of the predicate Person is "person," and the candidate Person(Walden) conflicts with the entity type "work," then... Relatively high.

[0057] The three evaluation parameters are obtained as follows: (1) Type domain check, Does the entity type fall under (1) Within the declared parameter type field (e.g., Person requires the parameter to be a person); (2) Conflict check, whether a conflict already exists. Mutually exclusive type facts (e.g., Work(Walden) and Person(Walden) are mutually exclusive); (3) Contextual support, whether the natural language context and annotation confidence support the attribution. Three-item normalized weighted synthesis When the type does not match or conflicts with an existing type Significantly increased.

[0058] This indicates the risk associated with predicate-predicate relationships. For predicate relationships of type `subclassOf`, `implies`, or `equivalentTo`, the system checks whether they lead to a loop hierarchy, conflict with existing backimplications, change the meaning of the predicate in the target query, and whether multiple natural language annotations support the direction of the relationship. For domains with high security requirements, this can be increased. To more strictly control predicate level risks.

[0059] The four evaluation parameters are obtained as follows: (1) Cycle detection: whether a cycle level is formed after adding the edge to the subclassOf / implies directed graph (determined by the directed graph cycle detection algorithm); (2) Reverse implication conflict: whether there is already an implication in the opposite direction (e.g., if there is already an implication in the opposite direction). Add (3) Semantic change check: Does the rule change the target query set? (4) Directional consistency: whether multiple natural language annotations consistently support the relation and its direction. Four-item weighted generation Any anomaly will increase the risk, and areas with high security requirements can increase the weighting. .

[0060] In obtaining and Subsequently, this invention uses the utility-risk ratio to screen candidate augmenting facts: ; in This is a smoothing term to prevent the denominator from being zero. The system follows... Sort the candidates from highest to lowest and add them to the augmentation fact set in sequence. until the candidate budget is reached. Or the cumulative risk exceeds the threshold This mechanism is similar to risk-adjusted payoff selection: facts with high information gain and low risk are prioritized in the solver; facts with high confidence but no help to the target query are not prioritized; facts with high risk, although they may shorten the proof path, are postponed or rejected.

[0061] Ultimately, the system will Submitted to the symbolic solver for the target query set The system performs reasoning. The solver can be a first-order logic prover, a SAT / SMT solver, an ASP solver, or a Datalog engine. If the solver outputs a true, false, or undecidable result for the target query, the system simultaneously outputs the proof path, the augmenting facts used, the source sentence for each augmenting fact, and a risk score. This output mechanism makes the reasoning results interpretable and traceable.

[0062] Compared to methods that only use a set of symbolic languages, It retains entity and relation information from the original natural language text that was discarded during the translation stage but is useful for proof. Compared to ordinary RAG or unconstrained knowledge completion, Each fact comes from annotations aligned with the symbolic language set and undergoes syntax and type checks, redundancy filtering, utility evaluation, and risk control, making it more suitable for use in conjunction with symbolic solvers.

[0063] Taking a simplified reasoning task as an example, the natural language premises include: "Walden is a novel. All novels are works of literature. Works of literature are usually written by authors. Thoreau wrote Walden." The target query is: "Is Walden a work of literature, and does it have an author?" Traditional natural language-symbolic language translation might only yield... and And missed or In this case, the solver can prove... However, it cannot be proven that an author exists.

[0064] Using the method of this invention, the system first obtains the constant Walden and predicates Novel, LiteraryWork, etc., from the set of translated symbolic language, and writes these symbols and their type signatures into relation annotation prompts. The relation annotation module returns to the original natural language text and identifies... , or Relationships such as 'etc.' are established. Subsequently, the typed knowledge graph uses Walden and Thoreau as constant nodes, Novel, LiteraryWork, and Author as predicate nodes, and constructs edges such as 'createdBy', 'isA', and 'subclassOf'. If the candidate... or If a fact is not implied by the symbolic language set, reduces the proof depth of the target query, and has a risk score below a threshold, it is added to the augmented fact set. The final solver is in The above completes the proof and outputs the augmented facts used and their source sentences. This embodiment illustrates that the present invention does not simply add external knowledge, but rather addresses the information loss caused by translation by recapturing entities and relations useful for reasoning from the original natural language text under the constraints of symbolic language. This allows the supplementary information to have both a natural language semantic basis and be directly used by the symbolic reasoning system.

[0065] Compared with existing technologies, it can bring at least the following beneficial effects: 1) Compensating for the loss of reasoning information caused by natural language-symbolic language translation. Existing neural symbolic reasoning assumes that the translation result fully carries the semantic meaning of the original text. However, real texts often contain cross-sentence references, implicit relationships, and modifying semantics, making information compression and omissions easy to occur during the translation process. This invention utilizes the lexicon and predicate signatures obtained from the translation to return to the original natural language text for knowledge element annotation, supplementing the key semantics lost in the translation and avoiding the problem of reasoning failure caused by translation.

[0066] 2) Achieving compatibility between annotation results and the solver. Ordinary entity and relation extraction outputs open labels, often resulting in mismatches between predicate names, parameter types, and the solver, requiring extensive manual alignment and even rendering them unusable for logical proof. This invention employs a symbolic language vocabulary, predicate signatures, and target query constraint annotation process, extracting semantics relevant to the reasoning task. The generated knowledge graph facts can be transformed into facts and rules usable by the solver, reducing the overhead of semantic alignment and format conversion.

[0067] 3) Enhances the interpretability and verifiability of reasoning results. This invention can trace the textual source, graphical basis, and legality verification status of each augmented fact, and record its specific role in logical proof. Compared to the fuzzy thought chain output of large models, the evidence is clear, verifiable, and easier for manual verification.

[0068] 4) Reduce the manual construction cost of domain-specific reasoning systems. Traditional symbolic reasoning heavily relies on domain experts manually designing entities, predicates, and organizing facts, resulting in high migration costs, and the content extracted from general knowledge graphs often does not meet the requirements of the reasoning task. This invention uses the results of natural language-symbolic language translation to guide annotation and knowledge graph construction, producing a structured knowledge graph adapted to the current reasoning task, thus reducing manual workload.

[0069] 5) Balancing reasoning ability and reliability. In legal, financial, and scientific research scenarios, blindly supplementing extracted knowledge can easily lead to erroneous reasoning. This invention addresses this by supplementing missing semantics in translation through annotation, and by employing multi-layered screening methods including redundancy filtering, grammatical type verification, information gain utility evaluation, and categorized risk scoring. Only candidate facts with strong relevance, high gain, and controllable risk are fed into the solver, avoiding indiscriminate acceptance of annotation results and adapting to high-reliability business scenarios.

[0070] In summary, this invention is applied to natural language logical reasoning tasks. It no longer relies solely on one-time natural language-symbolic language translation output. Instead, it adds a typed knowledge graph aligned with the reasoning task as an intermediate layer. This allows it to retain entity, attribute, relation, and conceptual axiom information that is easily lost during the translation stage through annotation and participate in reasoning. This adapts to unstructured real-world text scenarios and improves the accuracy, stability, and interpretability of logical reasoning.

[0071] A specific embodiment of the present invention discloses a large-scale model logical reasoning enhancement system based on knowledge graph construction, comprising: The translation module is used to obtain a set of natural language premises and translate the set of natural language premises into a set of symbolic language. The prompting module is used to obtain the target query set to be solved, and construct labeled prompt words for reasoning tasks based on the symbolic language set and the target query set; The annotation module is used to annotate knowledge elements of a natural language premise set based on annotation prompts; the knowledge element annotation includes at least entity annotation, relation annotation, and attribute annotation. The building module is used to construct a typed knowledge graph based on the knowledge element annotation results, and generate an enhanced fact set based on the typed knowledge graph; The reasoning module is used to input the enhanced fact set and the symbolic language set into the logic solver to perform logical reasoning on the target query set.

[0072] The above-described method and system embodiments are based on the same principles, and their related aspects can be referenced from each other to achieve the same technical effects. For specific implementation processes, please refer to the foregoing embodiments, which will not be repeated here.

[0073] The above description is only a preferred embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any changes or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in the present invention should be included within the scope of protection of the present invention.

Claims

1. A method for enhancing logical reasoning in large models based on knowledge graph construction, characterized in that, include: Obtain a set of natural language premises and translate the set of natural language premises into a set of symbolic language; Obtain the target query set to be solved, and construct annotation prompts for reasoning tasks based on the symbolic language set and the target query set; The natural language premise set is labeled with knowledge elements based on the labeled prompt words; wherein, the knowledge element labeling includes at least entity labeling, relation labeling and attribute labeling; A typified knowledge graph is constructed based on the knowledge element annotation results, and an enhanced fact set is generated based on the typified knowledge graph; The augmented fact set and the symbolic language set are input together into the logic solver to perform logical reasoning on the target query set.

2. The method for enhancing logical reasoning in large models based on knowledge graph construction according to claim 1, characterized in that, Translating the set of natural language premises into a set of symbolic languages ​​includes: For each natural language premise sentence in the natural language premise set, a large model is invoked for structured translation, and the structured translation results of each natural language premise sentence are summarized to obtain the symbolic language set; wherein, the symbolic language set includes at least: a set of constant symbols, a set of predicate symbols, and signature information of each predicate.

3. The method for enhancing logical reasoning in large models based on knowledge graph construction according to claim 2, characterized in that, Based on the symbolic language set and the target query set, labeled prompts for reasoning tasks are constructed, including: For each natural language premise sentence, candidate constants related to the sentence and the target query set are selected from the set of constant symbols, and candidate predicates related to the sentence and the target query set are selected from the set of predicate symbols. Summarize each predicate and its corresponding signature information in the symbolic language set to construct a predicate signature set; wherein, the signature information includes at least the number of signature elements and parameter types; Based on the candidate constants, the candidate predicates, and the set of predicate signatures, a tagging prompt word is constructed for each natural language premise sentence.

4. The method for enhancing logical reasoning in large models based on knowledge graph construction according to claim 3, characterized in that, The annotation prompts for each natural language premise sentence include: the corresponding candidate constant, the corresponding candidate predicate, the predicate signature set, the target query set, and the format constraints of the annotation output.

5. The method for enhancing logical reasoning in large models based on knowledge graph construction according to claim 4, characterized in that, Based on the labeled prompts, knowledge element annotation is performed on the natural language premise set, including: The labeled prompts are input into the large model, and knowledge element labeling is performed on the natural language premise set according to the following constraints: the entity nodes obtained by entity labeling are selected from the constant symbol set or aligned with the constant symbol set; the predicates in the relation triples obtained by relation labeling are selected from the predicate symbol set or aligned with the predicate symbol set; the entity type triples and entity attribute triples obtained by attribute labeling satisfy the constraints of the predicate signature set; and the labeling results are output according to the format constraints.

6. The method for enhancing logical reasoning in large models based on knowledge graph construction according to claim 5, characterized in that, Based on the knowledge element annotation results, a typified knowledge graph is constructed, including: Write the entity node, the relation triplet, the entity type triplet, and the entity attribute triplet into the typed knowledge graph; Constant-constant edges are created based on the relation triples, constant-predicate edges are created based on the entity type triples and the entity attribute triples, and logical rules are extracted from the natural language premise set to create predicate-predicate edges.

7. The method for enhancing logical reasoning in large models based on knowledge graph construction according to claim 1, characterized in that, Based on the typed knowledge graph, an enhanced set of facts is generated, including: Based on the type of edges in the knowledge graph, a set of candidate augmented facts is generated using a preset formula template. The candidate augmented fact set is filtered to obtain the augmented fact set.

8. The method for enhancing logical reasoning in large models based on knowledge graph construction according to claim 7, characterized in that, The candidate augmented fact set is filtered to obtain the augmented fact set, which includes: The candidate augmented fact set is subjected to syntactic and predicate checks item by item to obtain a preliminary set of augmented facts; The symbolic language set implication test is performed on the initially screened enhanced fact set to obtain the second-screened enhanced fact set; A benefit-risk score is calculated for the candidate augmented facts in the augmented fact set obtained from the secondary screening. The augmented fact set is then selected from the augmented fact set obtained from the secondary screening based on the benefit-risk score.

9. The method for enhancing logical reasoning in large models based on knowledge graph construction according to claim 8, characterized in that, The benefit-risk score is calculated for the candidate augmented facts in the augmented fact set obtained from the secondary screening, including: Calculate the information gain of the candidate augmented facts; Determine the type of the edge corresponding to the candidate augmentation fact, determine the risk coefficient and risk aversion weight of the candidate augmentation fact based on the edge type, and calculate the risk of the candidate augmentation fact based on the risk coefficient and the risk aversion weight; Calculate the benefit-risk score of the candidate augmentation fact based on its information gain and risk.

10. A large-scale model logical reasoning enhancement system based on knowledge graph construction, characterized in that, include: The translation module is used to obtain a set of natural language premises and translate the set of natural language premises into a set of symbolic language. The prompting module is used to obtain the target query set to be solved, and construct labeled prompt words for reasoning tasks based on the symbolic language set and the target query set; The annotation module is used to annotate the natural language premise set with knowledge elements based on the annotation prompts; wherein the knowledge element annotation includes at least entity annotation, relation annotation and attribute annotation; The construction module is used to construct a typed knowledge graph based on the knowledge element annotation results, and generate an enhanced fact set based on the typed knowledge graph; The reasoning module is used to input the augmented fact set and the symbolic language set into the logic solver to perform logical reasoning on the target query set.

Images