Knowledge enhanced large model question answering method, system, device and storage medium

Abstract

This invention discloses a knowledge-enhanced large-scale model question-answering method, system, device, and storage medium. These are corresponding solutions. In each solution: before answer generation, counterfactual masking scoring is performed on each candidate causal path to distinguish between evidence paths that truly influence the target conclusion and redundant paths that only have semantic relevance; simultaneously, by jointly selecting evidence subgraphs through various constraints, the key factors necessary to influence the conclusion can be retained while compressing the input length; furthermore, through causal evidence verification records, the answer range of the large language model can be constrained before generation, reducing the probability of causal direction errors and absolute statements; in addition, through post-generation consistency verification and failure regeneration mechanisms, the question-answering results can be kept consistent with the evidence subgraph, facilitating manual review, process auditing, and subsequent parameter adjustments.

Description

Technical Field

[0001] This invention relates to the fields of artificial intelligence and knowledge processing technology, and in particular to a knowledge-enhanced large-scale model question-answering method, system, device and storage medium. Background Technology

[0002] Retrieval-enhanced generation techniques have been widely used in scenarios such as enterprise knowledge question answering, R&D assistance, equipment operation and maintenance, and risk interpretation. Existing systems typically recall document fragments based on keywords, vector similarity, or graph adjacency relationships, and then input them into a large language model to generate answers. This approach is effective for fact-based question answering, but it still has shortcomings in causal questions.

[0003] First, existing methods are prone to misinterpreting co-occurrence, statistical correlation, or empirical descriptions as causal evidence, leading to overly certain conclusions in the model output. Second, general knowledge graph retrieval primarily expands entity neighbors or semantic paths, lacking the calculation of counterfactual differences after removing, masking, or replacing candidate causal paths, thus failing to determine whether a path truly influences the target conclusion. Third, inputting all candidate evidence into the model results in contextual redundancy, increased inference costs, and may allow key confounding variables or counterfactual paths to be overwhelmed by noise. Furthermore, causal question answering requires simultaneous consideration of processing variables, target variables, contextual conditions, confounding closures, evidence sources, and counterfactual constraints. Existing solutions lack a minimum sufficient evidence selection mechanism addressing these factors. Current answer verification often stops at post-generation fact-checking, failing to filter path contributions before evidence enters the model, and unable to distinguish which evidence is necessary to support the conclusion and which is merely background noise.

[0004] Therefore, there is a need for a knowledge-enhanced large model question answering method that can compute evidence contributions from candidate causal paths and select the minimum sufficient causal evidence subgraph under various constraints.

[0005] In view of this, the present invention is hereby proposed. Summary of the Invention

[0006] The purpose of this invention is to provide a knowledge-enhanced large-scale model question answering method, system, device, and storage medium that can perform counterfactual occlusion scoring on candidate paths, select the minimum sufficient causal evidence subgraph under various constraints, and finally perform consistency verification on the output answer of the large language model, thereby improving the interpretability and traceability of causal question answering.

[0007] The objective of this invention is achieved through the following technical solution: A knowledge-enhanced large-scale model question answering method includes: A causal knowledge graph with direction, time window, condition, and evidence source is constructed using multi-source data, and a corresponding evidence library is built; wherein, the causal edges in the causal knowledge graph are associated with the evidence library through the evidence source; Analyze user questions to identify causal query elements; Based on the causal query elements, candidate causal paths related to the user's question are recalled in the causal knowledge graph. For each recalled candidate causal path, perform deletion, masking, or permutation test interventions and calculate the contribution of each candidate causal path. By combining the contribution of each candidate causal path, one or more candidate causal paths are selected under preset constraints to form a minimum sufficient causal evidence subgraph. The minimum sufficient causal evidence subgraph is converted into a causal evidence verification record and input into the large language model to guide the large language model to generate an answer under the constraints of the causal evidence verification record; The answers generated by the large language model are subjected to consistency verification. If the verification passes, the answers generated by the large language model are taken as the question-and-answer results. The answers generated by the large language model include conclusions, basis, preconditions, and explanations of uncertainties.

[0008] A knowledge-enhanced large-scale question-answering system, used to implement the aforementioned method, includes: The causal knowledge graph and evidence base construction module utilizes multi-source data to construct a causal knowledge graph with direction, time window, condition, and evidence source, and builds a corresponding evidence base; among them, the causal edges in the causal knowledge graph are associated with the evidence base through the evidence source. The causal query parsing module is used to parse user questions and identify causal query elements; The candidate causal path recall module is used to recall candidate causal paths related to the user's question in the causal knowledge graph based on the causal query elements. The counterfactual impact scoring module performs deletion, masking, or substitution test interventions on each recalled candidate causal path and calculates the contribution of each candidate causal path. The minimum sufficient causal evidence subgraph optimization module combines the contribution of each candidate causal path and selects one or more candidate causal paths under preset constraints to form a minimum sufficient causal evidence subgraph. The constraint generation module is used to convert the minimum sufficient causal evidence subgraph into a causal evidence verification record and input it into the large language model, guiding the large language model to generate an answer under the constraint of the causal evidence verification record; The consistency verification module is used to verify the consistency of the answers generated by the large language model. If the verification passes, the answer generated by the large language model is taken as the question-and-answer result. The answer generated by the large language model includes a conclusion, basis, preconditions and uncertainty explanation.

[0009] A processing device includes: one or more processors; and a memory for storing one or more programs; When the one or more programs are executed by the one or more processors, the one or more processors implement the aforementioned method.

[0010] A readable storage medium storing a computer program that, when executed by a processor, implements the aforementioned method.

[0011] As can be seen from the technical solution provided by the present invention: 1) Before answer generation, counterfactual masking scoring is performed on each candidate causal path, which can distinguish between evidence paths that truly affect the target conclusion and redundant paths that only have semantic relevance; 2) By jointly selecting evidence subgraphs through various constraints, the key factors necessary to affect the conclusion can be retained while compressing the input length; 3) By using causal evidence verification records, the answer range of the large language model can be constrained before generation, reducing the probability of causal direction errors and absolute statements; 4) Through consistency verification and failure regeneration mechanisms after generation, the question-and-answer results can be kept consistent with the evidence subgraph, which is convenient for manual review, process auditing and subsequent parameter adjustment; 5) It can be deployed on existing knowledge graphs, rule bases, log bases and large language model services, with a clear engineering implementation path, which is convenient for implementation in equipment operation and maintenance, enterprise knowledge services and risk question-and-answer scenarios. Attached Figure Description

[0012] To more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings used in the following description of the embodiments will be briefly introduced. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0013] Figure 1 A flowchart of a knowledge-enhanced large-scale model question-answering method provided in an embodiment of the present invention.

[0014] Figure 2 This is a schematic diagram of a knowledge-enhanced large-scale model question-answering method provided in an embodiment of the present invention.

[0015] Figure 3 This is a schematic diagram of a knowledge-enhanced large-scale question-answering system provided in an embodiment of the present invention.

[0016] Figure 4 This is a schematic diagram of a processing device provided in an embodiment of the present invention. Detailed Implementation

[0017] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the protection scope of the present invention.

[0018] First, the following explanations are provided for the terms that may be used in this article: The terms “comprising,” “including,” “containing,” “having,” or other similar semantic descriptions should be interpreted as non-exclusive inclusion. For example, “comprising a technical feature element (such as raw material, component, ingredient, carrier, dosage form, material, size, part, component, mechanism, device, step, process, method, reaction conditions, processing conditions, parameter, algorithm, signal, data, product or article of manufacture, etc.)” should be interpreted as including not only the expressly listed technical feature element, but also other technical feature elements that are not expressly listed and are well-known in the art.

[0019] The term "composed of" excludes any technical features not expressly listed. When used in a claim, it closes the claim to exclude all technical features other than those expressly listed, except for associated conventional impurities. If the term appears only in a clause of a claim, it limits the claim to the elements expressly listed in that clause; elements recited in other clauses are not excluded from the overall claim.

[0020] The following is a detailed description of the knowledge-enhanced large-scale model question-answering method, system, device, and storage medium provided by this invention. Contents not described in detail in the embodiments of this invention are prior art known to those skilled in the art. Where specific conditions are not specified in the embodiments of this invention, conventional conditions in the art or conditions recommended by the manufacturer shall apply. Reagents or instruments used in the embodiments of this invention, unless otherwise specified by the manufacturer, are all commercially available conventional products.

[0021] Example 1 This invention provides a knowledge-enhanced large-scale model question-answering method. While existing methods like retrieval-enhanced generation, general knowledge graph expansion, and post-generation fact-checking can improve answer relevance, they still suffer from the following problems: First, they cannot quantify the impact of a single causal path on the target conclusion before the evidence enters the large language model; second, they cannot compress the context while retaining confounding factors and high-confidence counterfactual evidence; and third, they cannot transform the evidence screening results into generation constraints that can be verified by a computer. The technical improvement of this invention lies in organizing the counterfactual masking test, constraint subgraph solution, and pre- and post-generation dual verification into a continuous data processing flow. For example... Figure 1 As shown, the method mainly includes the following steps: Step 1: Construction of causal knowledge graph and evidence base.

[0022] In this embodiment of the invention, a causal knowledge graph with direction, time window, condition and evidence source is constructed using multi-source data, and a corresponding evidence library is constructed; wherein, the causal edges in the causal knowledge graph are associated with the evidence library through the evidence source.

[0023] Specifically, causal relationships are extracted from multi-source data (e.g., documents, logs, tables, and expert rules), and the directional attributes, time window features, and evidence sources of each causal relationship are recorded to form causal edges. All causal edges are then combined to form a causal knowledge graph.

[0024] A causal edge e can be represented as: ; in, and These represent the causal variable and the outcome variable, respectively. The causal relationship corresponding to the causal edge. Indicates the applicable conditions. Indicates the allowed time interval between cause and effect. Identifiers indicating the source of evidence. This indicates the identifier of the corresponding evidence text or evidence fragment; both are used to trace the source of evidence for the causal boundary e. This represents the source confidence of the causal edge e.

[0025] Step 2, Causal query analysis.

[0026] In this embodiment of the invention, the user's question is analyzed to identify causal query elements, which mainly include: identifying intervention variables, target variables, contextual conditions, and intervention expressions.

[0027] Step 3: Candidate causal path recall.

[0028] In this embodiment of the invention, candidate causal paths related to a user's question are recalled in a causal knowledge graph based on the causal query elements. The candidate causal paths include: supporting paths, hybrid paths, mediating paths, and counter-evidence paths; each candidate causal path is formed by connecting one or more causal edges in the causal knowledge graph in a specific direction.

[0029] In this embodiment of the invention, a combination of graph path recall and evidence base recall can be used. Graph path recall is used to obtain explicit directional relationships between variables, while evidence base recall is used to supplement evidence fragments in logs, reports, tables, and expert rules. For the same causal edge, if there are multiple sources of evidence, they are merged according to source credibility, time freshness, and scene matching to avoid the same evidence appearing repeatedly in the prompt words.

[0030] Step 4: Counterfactual effect on scoring.

[0031] For each recalled candidate causal path, perform deletion, masking, or permutation test interventions and calculate the contribution of each candidate causal path.

[0032] Let the set of candidate causal paths be denoted. The complete candidate graph formed by the set of candidate causal paths is G; for the i-th path The contribution of the path is calculated using the following formula. : ; in, This represents the inference result of the target variable obtained based on the complete candidate graph G. ( ) indicates the deletion, masking, or displacement test intervention path. The inference results are given by d, where i = 1, 2, ..., n, and n is the number of candidate causal paths. The larger the value, the better the path. The higher the contribution to the conclusion.

[0033] In one possible implementation, Output the normalized conclusion vector r: ; in, This represents the probability of supporting the target causal conclusion. Indicates the probability of proof by contradiction or rebuttal. Indicates the probability of insufficient evidence. This indicates the risk value of the target event.

[0034] Difference measurement It can be used to measure probability differences, conclusion differences, or semantic differences, and can be calculated as a weighted combination: ; in, Reference ( The normalized conclusion vector output. express The target event risk value; express The first three items, namely , and ; express The first three items; Represents the distribution of three types of conclusions (i.e. , and The Jensen-Shannon difference, where Sem represents the semantic difference in conclusions obtained from sentence vectors or rule labels. , and The weights are preset or adjusted based on the validation set. Therefore, the counterfactual impact score is not subjectively labeled by humans, but is calculated based on the results of two inferences of the candidate path.

[0035] Step 5: Minimum sufficient causal evidence subgraph optimization.

[0036] By combining the contribution of each candidate causal path, one or more candidate causal paths are selected under preset constraints to form a minimum sufficient causal evidence subgraph.

[0037] In this embodiment of the invention, the minimum sufficient causal evidence subgraph S satisfies the following constraints: (1) Hybrid closure constraint, i.e., confounding variables that are simultaneously related to the intervention variable and the target variable must be included or described; (2) Path coverage constraint, i.e., the main support chain and the necessary intermediate chain are covered; (3) Rebuttal retention constraint, i.e., high-confidence reverse evidence must not be deleted; (4) Evidence diversity constraint, i.e., evidence sources in text, structured data or expert rules are reasonably covered; (5) Context budget constraint, i.e., the length of evidence input to the model does not exceed the budget.

[0038] Based on the above constraints, the binary selection variable is solved on the candidate causal path set P, and the minimum sufficient causal evidence subgraph is formed by all candidate causal paths with a value of 1.

[0039] The binary selection variables are obtained by solving the following optimization problem: ; in, For the i-th path Binary selection variables, Representing a path The token length or storage length required to access the prompt word. For the i-th path Contribution Let S represent the penalty term for violating constraints (hybrid closure constraint, evidence-retaining constraint, evidence diversity constraint, or path coverage constraint), and let S be the minimum sufficient causal evidence subgraph. and The preset weighting coefficients, Used to adjust the weights of path contribution items. Used to adjust the weight of the penalty for constraint violation.

[0040] For example: if path contribution Normalize to [0,1], The number of tokens required to enter the prompt word is calculated based on the path; a single path typically requires tens to hundreds of tokens. Calculated based on the number of items that violate constraints, it can be set The range is 100 to 500. The range is 500 to 2000; for example, one could take... , This prioritizes high-contribution paths during the selection process while imposing greater penalties for violations of constraints.

[0041] When the number of candidate paths is small, integer programming can be used to solve the problem; when the number of candidate paths is large, a greedy selection algorithm can be used based on counterfactual contribution, evidence confidence, and constraint priority. First, hybrid closures and high-confidence counterfactual evidence should be forcibly included, then... Select a supporting chain until the context budget is reached.

[0042] Step 6: Generate constraints.

[0043] The minimum sufficient causal evidence subgraph is converted into a causal evidence verification record and input into the large language model to guide the large language model to generate an answer under the constraints of the causal evidence verification record.

[0044] Set evidence pruning thresholds and evidence-of-contrast retention thresholds; when the contribution of a candidate causal path is lower than the evidence pruning threshold and it does not belong to the hybrid closure or evidence-of-contrast set, it is excluded from the model context (i.e., the input evidence content provided to the large language model for generating answers); when a candidate causal path is an evidence-of-contrast path and its confidence level is higher than the evidence-of-contrast retention threshold, it is saved to the causal evidence verification record to avoid the model generating answers based solely on one-way evidence.

[0045] The causal evidence verification record includes: variable definitions, support chains, confounding factors, chains of rebuttal, applicable conditions, sources of evidence, contribution scores, and prohibited out-of-bounds conclusions. The large language model generates an answer based on this record and outputs the conclusion, supporting evidence, preconditions, and an explanation of uncertainties.

[0046] Step 7: Consistency check.

[0047] In this embodiment of the invention, the answers generated by the large language model are subjected to consistency verification. If the verification passes, the answers generated by the large language model are taken as the question-and-answer results. The answers generated by the large language model include conclusions, basis, preconditions, and explanations of uncertainties.

[0048] In this embodiment of the invention, the consistency verification includes three types of checks: checking whether the answer uses a key causal edge that is not present in the causal evidence verification record; checking whether the answer ignores high-confidence disproven paths or confounding factors; and checking whether the answer rewrites conditional conclusions as absolute conclusions. A high-confidence disproven path refers to a candidate causal path in the minimum sufficient causal evidence subgraph with a confidence level higher than a preset disprovenance retention threshold. When any check fails, the causal evidence verification record is regenerated and input into the large language model, which then regenerates the answer based on retaining the minimum sufficient causal evidence subgraph.

[0049] In this embodiment of the invention, the causal evidence verification record can use structured fields: QueryID, Intervention, Outcome, Context, SelectedPaths, Confounders, CounterEvidence, ForbiddenClaims, BudgetUsed, and TraceID; where QueryID represents the query identifier; Intervention represents the intervention variable or intervention expression; Outcome represents the target variable; Context represents the context conditions; SelectedPaths represents the selected candidate causal paths; Confounders represents confounding factors; CounterEvidence represents counter-evidence information; ForbiddenClaims represents out-of-bounds conclusions that are prohibited from being output; BudgetUsed represents the context budget that has been used; and TraceID represents the traceability identifier for this question and answer. The ForbiddenClaims field records absolute conclusions that must not be output, such as prohibiting the output of "inevitably leads to" when only conditional evidence exists. During consistency verification, this record is read, and the causal edges, directional words, conditional words, and counter-evidence references in the answer are matched; when the answer adds a key causal edge that does not appear in SelectedPaths, or omits high-confidence counter-evidence in CounterEvidence, the answer is marked as failing and a rewrite is triggered.

[0050] In this embodiment of the invention, during each question-and-answer session, the input question, candidate causal paths, path contributions, selected subgraphs, model versions, and output answers are recorded to form a traceable log. This log can be used for expert review, as well as for subsequent threshold adjustments and offline evaluation, enabling continuous optimization of the evidence selection strategy based on feedback from real-world applications.

[0051] The above-mentioned solutions provided by the embodiments of the present invention mainly achieve the following beneficial effects: 1) Before generating the answer, counterfactual masking scores are performed on each candidate causal path, which can distinguish between evidence paths that truly affect the target conclusion and redundant paths that are only semantically relevant.

[0052] 2) By jointly selecting evidence subgraphs through various constraints, the key factors necessary to influence the conclusion can be retained while compressing the input length.

[0053] 3) By verifying causal evidence records, the range of responses of the large language model can be constrained before generation, reducing the probability of causal direction errors and absolute statements.

[0054] 4) Through the consistency verification and failure regeneration mechanism after generation, the question and answer results can be kept consistent with the evidence subgraph, which facilitates manual review, process auditing and subsequent parameter adjustment.

[0055] 5) It can be deployed on top of existing knowledge graphs, rule bases, log bases and large language model services. The engineering implementation path is clear and it is easy to implement in equipment operation and maintenance, enterprise knowledge services and risk Q&A scenarios.

[0056] To more clearly demonstrate the technical solution and its effects provided by the present invention, the method provided by the embodiments of the present invention will be described in detail below with reference to specific embodiments. In this specific embodiment, the entire solution is illustrated using a compressor shutdown risk Q&A example, such as... Figure 2 As shown, the overall process of the solution is illustrated.

[0057] I. Construction of Causal Knowledge Graph and Evidence Base.

[0058] Candidate causal relationships are extracted from multi-source data such as equipment manuals, alarm logs, maintenance work orders, and expert rules, and the directional attributes, time window characteristics, and evidence sources of each causal relationship are recorded simultaneously. The candidate causal relationships include, but are not limited to, the influence of environmental variables (such as temperature and humidity) on sensor false alarm rate and equipment downtime risk.

[0059] II. The issue of receiving causality.

[0060] In this embodiment, the system receives questions from users: Will an increase in ambient temperature lead to a higher risk of compressor shutdown, and is it necessary to shut down the compressor immediately?

[0061] III. Analysis of variables and conditions.

[0062] The system analyzes the user's questions and identifies the causal variable as rising ambient temperature and the outcome variable as compressor shutdown risk. The contextual conditions include ambient humidity, equipment model, sensor calibration status, and maintenance cycle.

[0063] IV. Candidate Path Recall.

[0064] Based on candidate causal relationships, recall causal paths related to the problem, including direct paths from the target variable to the risk event, mediated paths via sensor false alarms, confounding paths caused by common causes such as environmental factors, and counter-evidence paths that do not trigger actual risk events.

[0065] 5. Perform counterfactual perturbations and calculate path contributions.

[0066] The recalled candidate paths are sequentially masked, and the inference calculation is re-performed. The contribution of each path is quantified based on the magnitude of the change in the inference conclusion: if masking a specific path leads to a substantial reversal or change in the inference conclusion (e.g., from a false alarm to a genuine fault requiring confirmation), then that path is assigned a high contribution weight; if masking a specific sample path keeps the inference conclusion unchanged, then that path is determined to be low contribution evidence.

[0067] VI. Minimum sufficient subgraph selection.

[0068] Within a preset threshold of computing resources or the number of nodes, high-contribution supporting paths, necessary confounding variables, and high-confidence counter-evidence paths are selected and retained to construct a causal evidence verification record. The applicable boundaries for triggering specific conditions (such as a specific humidity threshold) are then explicitly output. In a specific implementation scenario, supporting paths representing direct risks are... Characterizing the mediation path of false alarms caused by multi-factor coupling. Hybrid pathways characterizing common cause influences and historical counter-evidence sample paths Both are included in the candidate set. (If the intermediate path...) The counterfactual score (path contribution) is high and Even if it is judged as a high-confidence proof, The direction of the argument contradicts the conclusion of the main reasoning, yet it will still be... Write the counterevidence field; if If it applies to both alarm nodes and risk nodes, it will be written as a confounders factor in the confounders field.

[0069] VII. Causal evidence verification record conversion and binding content generation.

[0070] The minimum sufficient causal evidence subgraph is converted into a causal evidence verification record. The large language model then generates a constrained inference response (answer) based on this record. This response includes applicable premises, a causal evidence chain, and operational suggestions (e.g., a suggestion to prioritize verifying sensor status under specific environmental variables to eliminate false alarms). Subsequently, the generated response is automatically reviewed by a consistency verification module to verify its completeness in including applicable conditions, evidence chains, and uncertainty statements.

[0071] 8. Consistency check.

[0072] The consistency check and failure regeneration mechanism ensure that the question-and-answer results are consistent with the evidence subgraph.

[0073] IX. Closed-loop feedback and weight update.

[0074] By acquiring actual feedback data from on-site handling, if the identified actual triggering factors differ from historical inferences (e.g., the risk is caused by maintenance cycle exceeding limits rather than temperature fluctuations), the path corresponding to the actual triggering factor is added to the counter-evidence set, and the contribution weight of the original misleading path in similar scenarios is dynamically reduced. Based on this feedback mechanism, when processing similar inputs subsequently, the minimum sufficient evidence subgraph that conforms to the physical laws of the scene can be retrieved and matched more efficiently.

[0075] 10. Cache reuse and audit trail.

[0076] For batch causal question answering tasks, candidate causal paths and their contribution scores for frequently asked questions are pre-cached. When similar query requests are received, incremental calculation is only performed on causal paths where the context conditions have significantly shifted, in order to reduce online inference overhead. Furthermore, when outputting inference results, an audit log is generated and output synchronously. This audit log includes evidence subgraph numbers, a list of removed paths, the criteria for retaining counter-evidence, and the version information of the model call.

[0077] XI. Expert intervention and adaptive threshold adjustment.

[0078] It provides a manual review mechanism and intervention interface. When domain experts determine that a specific low-scoring path has critical business value, they can respond to instructions and configure it to the mandatory retention set; for evidence types that have been repeatedly verified as invalid by business, it supports dynamically increasing their pruning threshold. Through the above mechanisms, the efficiency of automated inference can be maintained while continuously injecting and accumulating expert experience into the minimum sufficient subgraph selection process.

[0079] Through the above description of the embodiments, those skilled in the art can clearly understand that the above embodiments can be implemented by software, or by using software plus necessary general-purpose hardware platforms. Based on this understanding, the technical solutions of the above embodiments can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (such as a CD-ROM, USB flash drive, mobile hard drive, etc.), including several instructions to cause a computer device (such as a personal computer, server, or network device, etc.) to execute the methods described in the various embodiments of the present invention.

[0080] Example 2 This invention also provides a knowledge-enhanced large-scale question-answering system, which is mainly used to implement the methods provided in the foregoing embodiments, such as... Figure 3 As shown, the system mainly includes: The causal knowledge graph and evidence base construction module utilizes multi-source data to construct a causal knowledge graph with direction, time window, condition, and evidence source, and builds a corresponding evidence base; among them, the causal edges in the causal knowledge graph are associated with the evidence base through the evidence source. The causal query parsing module is used to parse user questions and identify causal query elements; The candidate causal path recall module is used to recall candidate causal paths related to the user's question in the causal knowledge graph based on the causal query elements. The counterfactual impact scoring module performs deletion, masking, or substitution test interventions on each recalled candidate causal path and calculates the contribution of each candidate causal path. The minimum sufficient causal evidence subgraph optimization module combines the contribution of each candidate causal path and selects one or more candidate causal paths under preset constraints to form a minimum sufficient causal evidence subgraph. The constraint generation module is used to convert the minimum sufficient causal evidence subgraph into a causal evidence verification record and input it into the large language model, guiding the large language model to generate an answer under the constraint of the causal evidence verification record; The consistency verification module is used to verify the consistency of the answers generated by the large language model. If the verification passes, the answer generated by the large language model is taken as the question-and-answer result. The answer generated by the large language model includes a conclusion, basis, preconditions and uncertainty explanation.

[0081] Since the main technical details of the above system have been described in detail in the previous embodiments, they will not be repeated here.

[0082] Those skilled in the art will understand that, for the sake of convenience and brevity, the above-described division of functional modules is used as an example. In practical applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the system can be divided into different functional modules to complete all or part of the functions described above.

[0083] Example 3 The present invention also provides a processing device, such as Figure 4 As shown, it mainly includes: one or more processors; a memory for storing one or more programs; wherein, when the one or more programs are executed by the one or more processors, the one or more processors implement the method provided in the foregoing embodiments.

[0084] Furthermore, the processing device also includes at least one input device and at least one output device; in the processing device, the processor, memory, input device, and output device are connected via a bus.

[0085] In this embodiment of the invention, the specific types of the memory, input device, and output device are not limited; for example: Input devices can be touchscreens, image acquisition devices, physical buttons, or mice, etc. The output device can be a display terminal; The memory can be random access memory (RAM) or non-volatile memory, such as disk storage.

[0086] Example 4 The present invention also provides a readable storage medium storing a computer program that, when executed by a processor, implements the method provided in the foregoing embodiments.

[0087] In this embodiment of the invention, the readable storage medium is a computer-readable storage medium and can be disposed in the aforementioned processing device, for example, as a memory in the processing device. Furthermore, the readable storage medium can also be any medium capable of storing program code, such as a USB flash drive, portable hard drive, read-only memory (ROM), magnetic disk, or optical disk.

[0088] The above description is merely a preferred embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in the present invention should be included within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the scope of the claims. The information disclosed in the background section is intended only to enhance the understanding of the overall background technology of the present invention and should not be construed as an admission or implication in any way that such information constitutes prior art known to those skilled in the art.

Claims

1. A knowledge-enhanced large-scale model question-answering method, characterized in that, include: A causal knowledge graph with direction, time window, condition, and evidence source is constructed using multi-source data, and a corresponding evidence library is built; wherein, the causal edges in the causal knowledge graph are associated with the evidence library through the evidence source; Analyze user questions to identify causal query elements; Based on the causal query elements, candidate causal paths related to the user's question are recalled in the causal knowledge graph. For each recalled candidate causal path, perform deletion, masking, or permutation test interventions and calculate the contribution of each candidate causal path. By combining the contribution of each candidate causal path, one or more candidate causal paths are selected under preset constraints to form a minimum sufficient causal evidence subgraph. The minimum sufficient causal evidence subgraph is converted into a causal evidence verification record and input into the large language model to guide the large language model to generate an answer under the constraints of the causal evidence verification record; The answers generated by the large language model are subjected to consistency verification. If the verification passes, the answers generated by the large language model are taken as the question-and-answer results. The answers generated by the large language model include conclusions, basis, preconditions, and explanations of uncertainties.

2. The knowledge-enhanced large-scale model question-answering method according to claim 1, characterized in that, The construction of a causal knowledge graph with direction, time window, condition, and evidence source using multi-source data includes: Causal relationships are extracted from multi-source data, and the directional attributes, time window features, and evidence sources of each causal relationship are recorded to form causal edges. All causal edges are then used to form a causal knowledge graph. A causal edge e can be represented as: ； in, and These represent the causal variable and the outcome variable, respectively. The causal relationship corresponding to the causal edge. Indicates applicable conditions. Indicates the allowed time interval between cause and effect. Identifiers indicating the source of evidence. This indicates the identifier of the corresponding evidence text or evidence fragment; both are used to trace the source of evidence for the causal boundary e. This represents the source confidence of the causal edge e.

3. The knowledge-enhanced large-scale model question-answering method according to claim 1, characterized in that, The candidate causal paths related to recall and user issues include: recall support path, heterogeneous path, mediation path, and evidence-of-contrast path; Each candidate causal path is formed by connecting one or more causal edges in the causal knowledge graph in a specific direction.

4. The knowledge-enhanced large-scale model question-answering method according to claim 1, characterized in that, The step of performing deletion, masking, or permutation tests on each recalled candidate causal path and calculating the contribution of each candidate causal path includes: Let the set of candidate causal paths be denoted. The complete candidate graph formed by the set of candidate causal paths is G; For the i-th path The contribution of the path is calculated using the following formula. : ； in, This represents the inference result of the target variable obtained based on the complete candidate graph G. ( ) indicates the deletion, masking, or displacement test intervention path. The inference results are given by d, where i = 1, 2, ..., n, and n is the number of candidate causal paths.

5. The knowledge-enhanced large-scale model question-answering method according to claim 1, characterized in that, The step of combining the contribution of each candidate causal path and selecting one or more candidate causal paths under preset constraints to form a minimum sufficient causal evidence subgraph includes: The preset constraints include: hybrid closure constraint, path coverage constraint, evidence retention constraint, evidence diversity constraint, and context budget constraint. Combining the above constraints, the binary selection variable is solved on the candidate causal path set P, and the minimum sufficient causal evidence subgraph is formed by all candidate causal paths with a value of 1. The binary selection variables are obtained by solving the following optimization problem: ； in, For the i-th path Binary selection variables, Representing a path The token length or storage length required to access the prompt word. For the i-th path Contribution S represents the penalty term for violating the constraints, and S is the minimum sufficient causal evidence subgraph. and These are the preset weighting coefficients.

6. The knowledge-enhanced large-scale model question-answering method according to claim 1, characterized in that, Converting the minimum sufficient causal evidence subgraph into a causal evidence verification record includes: Set evidence pruning threshold and evidence-of-contrast retention threshold; when the contribution of a candidate causal path is lower than the evidence pruning threshold and it does not belong to the hybrid closure or evidence-of-contrast set, it is excluded from the model context; when a candidate causal path is an evidence-of-contrast path and its confidence is higher than the evidence-of-contrast retention threshold, it is saved to the causal evidence verification record. The causal evidence verification record includes: variable definition, support chain, confounding factors, chain of counter-evidence, applicable conditions, source of evidence, contribution score, and out-of-bounds conclusions that are prohibited from being output.

7. The knowledge-enhanced large-scale model question-answering method according to claim 1, characterized in that, The consistency check includes three types of checks: checking whether the answer uses a key causal edge that does not exist in the causal evidence check record; checking whether the answer ignores high-confidence disproven paths or confounding factors; and checking whether the answer rewrites conditional conclusions as absolute conclusions. Among them, a high-confidence disproven path refers to a candidate causal path in the minimum sufficient causal evidence subgraph with a confidence level higher than the preset disproven evidence retention threshold. If any check fails, the causal evidence verification record is regenerated and input into the large language model, which then regenerates the answer based on retaining the minimum sufficient causal evidence subgraph.

8. A knowledge-enhanced large-scale question-answering system, characterized in that, To implement the method according to any one of claims 1 to 7, comprising: The causal knowledge graph and evidence base construction module utilizes multi-source data to construct a causal knowledge graph with direction, time window, condition, and evidence source, and builds a corresponding evidence base; among them, the causal edges in the causal knowledge graph are associated with the evidence base through the evidence source. The causal query parsing module is used to parse user questions and identify causal query elements; The candidate causal path recall module is used to recall candidate causal paths related to the user's question in the causal knowledge graph based on the causal query elements. The counterfactual impact scoring module performs deletion, masking, or substitution test interventions on each recalled candidate causal path and calculates the contribution of each candidate causal path. The minimum sufficient causal evidence subgraph optimization module combines the contribution of each candidate causal path and selects one or more candidate causal paths under preset constraints to form a minimum sufficient causal evidence subgraph. The constraint generation module is used to convert the minimum sufficient causal evidence subgraph into a causal evidence verification record and input it into the large language model, guiding the large language model to generate an answer under the constraint of the causal evidence verification record; The consistency verification module is used to verify the consistency of the answers generated by the large language model. If the verification passes, the answer generated by the large language model is taken as the question-and-answer result. The answer generated by the large language model includes a conclusion, basis, preconditions and uncertainty explanation.

9. A processing device, characterized in that, include: One or more processors; Memory, used to store one or more programs; Wherein, when the one or more programs are executed by the one or more processors, the one or more processors cause the one or more processors to implement the method as described in any one of claims 1 to 7.

10. A readable storage medium storing a computer program, characterized in that, When a computer program is executed by a processor, it implements the method as described in any one of claims 1 to 7.

Images