A model inference method, device, equipment and computer readable storage medium

Abstract

The application discloses a model reasoning method and device, equipment and a computer readable storage medium, applied to the technical field of intelligent question and answer, comprising: determining the search path in the knowledge graph corresponding to the target reasoning question; determining the knowledge fragment metadata corresponding to each hop path in the search path, and performing compliance verification on the searched knowledge fragment based on the compliance rule graph in the search process; when reaching the last hop of the search path, generating the target reasoning result by using a large language model based on the complete knowledge fragment searched, and determining the final credibility corresponding to the target reasoning result; the final credibility is determined based on the final cumulative path credibility, the credible knowledge fragment coverage and the average credible contribution degree. The application fully embeds the compliance rules into each link of the multi-hop search based on the credibility, reduces the compliance loopholes in the knowledge fragment search process, improves the credibility of the model reasoning result, and accurately evaluates the credibility of the target reasoning result.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of intelligent question answering, and in particular relates to a model reasoning method and device, equipment and a computer readable storage medium. BACKGROUND

[0002] Large language models (LLM) rely on the self-attention mechanism of the Transformer architecture and the "pre-training-fine-tuning" paradigm to achieve the understanding and generation of general natural language, and can process basic tasks such as syntax analysis and text creation. However, in the fields of finance, law and other fields with high requirements for facts and relevance, LLM has obvious shortcomings: knowledge is fixed in training data (there is a time cutoff point and information cannot be updated in real time), it is easy to generate "illusory content" without basis (such as fictitious financial product yield), it is difficult to capture deep connections between knowledge (cannot connect the logic of "customer overdue" and "guarantor default"), and it is difficult to meet actual business needs.

[0003] Therefore, how to improve the credibility of the model reasoning result is a technical problem that needs to be solved by those skilled in the art. SUMMARY

[0004] Therefore, the purpose of the present application is to provide a model reasoning method, device, equipment and computer readable storage medium, which solves the technical problem of low credibility of the model reasoning result in the prior art.

[0005] To solve the above technical problems, the present application provides a model reasoning method, comprising:

[0006] determining a retrieval path in a knowledge graph corresponding to a target reasoning question;

[0007] determining knowledge fragment metadata corresponding to each hop path in the retrieval path, and performing compliance verification on the retrieved knowledge fragments based on a compliance rule graph during the retrieval process; wherein the knowledge fragment metadata includes knowledge fragment identification, knowledge fragment credibility, knowledge fragment credibility contribution, and cumulative path credibility; and the compliance rule graph is a graph constructed based on compliance rule metadata;

[0008] when the last hop of the retrieval path is reached, generating a target reasoning result based on the complete knowledge fragments retrieved using a large language model, and determining a final credibility corresponding to the target reasoning result; wherein the final credibility is determined based on the final cumulative path credibility, the coverage of credible knowledge fragments, and the average credible contribution.

[0009] Optionally, before determining the knowledge fragment metadata corresponding to each hop path in the retrieval path and performing compliance verification on the retrieved knowledge fragments based on the compliance rule graph during the retrieval process, the method further comprises:

[0010] obtaining a knowledge consistency result based on cross verification of the current knowledge fragment and other source knowledge fragments;

[0011] determining a knowledge fragment credibility corresponding to each knowledge fragment based on a knowledge source authority of each knowledge fragment, a knowledge timeliness of the knowledge fragment, and the knowledge consistency result; wherein the knowledge source authority is determined based on a source authority level corresponding to the knowledge fragment; and the knowledge timeliness is determined based on a difference between a publication time of the knowledge fragment and a current time.

[0012] Optionally, before determining the knowledge fragment metadata corresponding to each hop path in the retrieval path and performing compliance verification on the retrieved knowledge fragments based on the compliance rule graph during the retrieval process, the method further comprises:

[0013] constructing the compliance rule graph based on the compliance rule metadata; wherein the compliance rule metadata comprises a rule unique identifier, a rule content, an applicable business scenario, a trigger condition, a violation operation, and a rule update time;

[0014] embedding the compliance rule graph into the knowledge graph in a retrieval-enhanced form.

[0015] Optionally, determining the knowledge fragment metadata corresponding to each hop path in the retrieval path and performing compliance verification on the retrieved knowledge fragments based on the compliance rule graph during the retrieval process comprises:

[0016] generating a unique path identifier of the retrieval path and determining the knowledge fragment metadata corresponding to each hop path in the retrieval path;

[0017] wherein the knowledge fragment credibility contribution is determined by the knowledge fragment credibility of the current hop and a path attenuation coefficient;

[0018] the cumulative path credibility is determined by the knowledge fragment credibility contribution.

[0019] Optionally, determining the knowledge fragment metadata corresponding to each hop path in the retrieval path and performing compliance verification on the retrieved knowledge fragments based on the compliance rule graph during the retrieval process comprises:

[0020] before performing retrieval, pulling pre-filtering rules from the compliance rule graph according to a business scenario, generating retrieval filtering conditions, and filtering the target reasoning problem based on the retrieval filtering conditions to obtain a filtered reasoning problem;

[0021] determining a to-be-verified knowledge fragment based on the filtered reasoning problem, and matching a verification condition corresponding to a business scenario of the to-be-verified knowledge fragment;

[0022] checking the to-be-checked knowledge segment based on the checking condition;

[0023] If it is determined based on the checking condition that the knowledge segment violates the rule, a violation operation is performed, the violation operation including terminating the search and returning a compliance prompt, or triggering a re-search until the compliance rule is met or a re-search upper limit is reached.

[0024] Optionally, when the last hop of the search path is reached, a target reasoning result is generated based on the complete knowledge segment searched by using a large language model, and a final credibility corresponding to the target reasoning result is determined, including:

[0025] Determining a key fact point and a logic chain of the target reasoning result;

[0026] Removing the current to-be-evaluated knowledge segment from the complete knowledge segment, and regenerating an answer based on the remaining searched knowledge segment by using the large language model to obtain a removed reasoning result;

[0027] Determining a semantic difference degree between the target reasoning result and the removed reasoning result, and determining an accuracy attenuation degree of the removed reasoning result based on the key fact point and the logic chain;

[0028] Determining a knowledge segment reference degree corresponding to the current to-be-evaluated knowledge segment based on the semantic difference degree and the accuracy attenuation degree;

[0029] Determining a knowledge segment contribution degree based on the knowledge segment credibility of the current to-be-evaluated knowledge segment and the knowledge segment reference degree, and determining the average credibility contribution degree based on the knowledge segment contribution degrees corresponding to all knowledge segments;

[0030] Determining the final credibility based on the final cumulative path credibility corresponding to the complete knowledge segment, the average credibility contribution degree, and the credible knowledge segment coverage.

[0031] Optionally, when the last hop of the search path is reached, a target reasoning result is generated based on the complete knowledge segment searched by using a large language model, and a final credibility corresponding to the target reasoning result is determined, including:

[0032] Determining a credibility level corresponding to each knowledge segment in the complete knowledge segment, and determining a credibility weight based on the credibility level;

[0033] Determining the credible knowledge segment coverage based on the credibility weight of each knowledge segment and the corresponding coverage.

[0034] The application further provides a model reasoning device, including:

[0035] The retrieval path determination module is configured to determine a retrieval path in a knowledge graph corresponding to the target reasoning question;

[0036] The compliance verification module is configured to determine knowledge segment metadata corresponding to each hop path in the retrieval path, and perform compliance verification on the retrieved knowledge segments based on a compliance rule graph during the retrieval process; wherein the knowledge segment metadata comprises a knowledge segment identifier, a knowledge segment credibility, a knowledge segment credibility contribution, and an accumulated path credibility; and the compliance rule graph is a graph constructed based on compliance rule metadata.

[0037] The target reasoning module is configured to generate a target reasoning result using a large language model based on the complete knowledge segments retrieved when the last hop of the retrieval path is reached, and determine a final credibility corresponding to the target reasoning result; wherein the final credibility is determined based on a final accumulated path credibility, a credible knowledge segment coverage, and an average credible contribution degree.

[0038] The present application also provides a model reasoning device, comprising:

[0039] A memory is configured to store a computer program.

[0040] A processor is configured to execute the computer program to implement the steps of the above-mentioned model reasoning method.

[0041] The present application also provides a computer readable storage medium, wherein the computer readable storage medium stores a computer program, and the computer program is executed by a processor to implement the steps of the above-mentioned model reasoning method.

[0042] The present application also provides a computer program product, comprising computer programs / instructions, which are executed by a processor to implement the steps of the above-mentioned model reasoning method.

[0043] It can be seen that the application determines the retrieval path corresponding to the target reasoning problem in the knowledge graph, determines the knowledge fragment metadata corresponding to each hop path in the retrieval path, and performs compliance verification on the retrieved knowledge fragments based on the compliance rule graph in the retrieval process. The knowledge fragment metadata includes knowledge fragment identification, knowledge fragment credibility, knowledge fragment credibility contribution, and cumulative path credibility. The compliance rule graph is a graph constructed based on compliance rule metadata. When the last hop of the retrieval path is reached, the target reasoning result is generated based on the complete knowledge fragment retrieved by using a large language model, and the final credibility corresponding to the target reasoning result is determined. The final credibility is determined based on the final cumulative path credibility, the credible knowledge fragment coverage, and the average credible contribution. The beneficial effects of the application are that the compliance rules are fully embedded in each link of the multi-hop retrieval based on credibility, the compliance control of the retrieval process is realized from the source, the compliance loopholes in the knowledge fragment retrieval process are reduced, the risk of generating illegal content is reduced, the credibility of the model reasoning result is improved, and the credibility of the target reasoning result is accurately evaluated.

[0044] In addition, the application also provides a model reasoning method, device, equipment and computer readable storage medium, which also have the beneficial effects described above. BRIEF DESCRIPTION OF DRAWINGS

[0045] In order to more clearly illustrate the technical solutions in the embodiments of the application or the prior art, the following will briefly introduce the drawings needed to be used in the embodiments or prior art description. Obviously, the drawings in the following description are only embodiments of the application, and for those skilled in the art, other drawings can be obtained without creative labor on the basis of the provided drawings.

[0046] Figure 1 A flowchart of a model reasoning method provided by an embodiment of the application;

[0047] Figure 2 A flowchart of a model reasoning method provided by an embodiment of the application;

[0048] Figure 3 A structural schematic diagram of a model reasoning device provided by an embodiment of the application;

[0049] Figure 4 A structural schematic diagram of a model reasoning device provided by an embodiment of the application. DETAILED DESCRIPTION

[0050] In order to make the purposes, technical solutions and advantages of the embodiments of the present application clearer, the technical solutions in the embodiments of the present application will be described clearly and completely below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative work fall within the scope of protection of the present application.

[0051] Reference is made to Figure 1 , Figure 1 A flowchart of a model reasoning method provided by the embodiments of the present application. The method can include:

[0052] S101, determining a search path in a knowledge graph corresponding to a target reasoning question.

[0053] The various steps in the embodiments can be executed by a specified electronic device, which can be a server, a portable terminal or other forms. The embodiments are not limited to specific target reasoning questions, for example, the target reasoning question in the embodiments can be a reasoning question in the financial field, or the target reasoning question in the embodiments can also be a reasoning question in the medical field. The knowledge graph in the embodiments is a structured semantic network for representing and storing knowledge, which liberates knowledge from unstructured text and converts it into structured, interconnected data that machines can understand and reason about. This structure is the basis for supporting multi-hop search and complex reasoning. The search path refers to a chain sequence composed of a series of interconnected entities and relationships in the knowledge graph for answering a specific question. It describes the reasoning route from the starting point of the question (usually a known entity) to the target point where the answer is located.

[0054] S102, determining knowledge segment metadata corresponding to each hop path in the search path, and performing compliance verification on the searched knowledge segments based on a compliance rule graph in the search process; wherein the knowledge segment metadata includes knowledge segment identification, knowledge segment credibility, knowledge segment credibility contribution and cumulative path credibility; the compliance rule graph is a graph constructed based on compliance rule metadata.

[0055] In this embodiment, when searching based on the search path, a knowledge fragment is obtained at each hop, and the knowledge fragment metadata corresponding to the knowledge fragment is determined. In this embodiment, the compliance checking of the searched knowledge fragment based on the compliance rule graph during the search process refers to the compliance checking of the knowledge fragment at each hop, and the compliance checking of the target reasoning problem. The compliance rule graph in this embodiment is a knowledge graph specially used for storing, managing and associating compliance rules. It is different from the "main knowledge graph" that stores business data, but a meta-knowledge graph that stores "rules about rules". It converts the originally scattered and isolated compliance provisions and risk control strategies into a structured network that can be understood and executed by machines. The rules in the rule graph in this embodiment are constraints that must be followed by the target reasoning problem and the knowledge fragment under certain conditions. The knowledge fragment identifier in this embodiment is the identifier corresponding to each knowledge fragment. The knowledge fragment credibility in this embodiment is determined based on the source authority, timeliness and consistency of each knowledge fragment. The knowledge fragment credibility contribution of each hop in this embodiment is determined by the knowledge fragment credibility score and the path attenuation coefficient. The cumulative path credibility in this embodiment is the cumulative path credibility from the initial to the hop.

[0056] It needs to be further explained that based on any of the above embodiments, before determining the knowledge fragment metadata corresponding to each hop of the search path, and performing compliance checking of the searched knowledge fragment based on the compliance rule graph during the search process, it can further include: obtaining a knowledge consistency result based on cross-validation of the current knowledge fragment and knowledge fragments from other sources; determining the knowledge fragment credibility corresponding to each knowledge fragment based on the knowledge source authority, knowledge timeliness and knowledge consistency result of each knowledge fragment; wherein the knowledge source authority is determined based on the source authority level corresponding to the knowledge fragment; the knowledge timeliness is determined based on the difference between the publication time of the knowledge fragment and the current time. The knowledge fragment credibility in this embodiment is determined by the knowledge source authority, knowledge timeliness and knowledge consistency result, and the embodiment does not limit the level of the knowledge source authority, knowledge timeliness and knowledge consistency result, for example, the level corresponding to different knowledge fragments can be 5 levels; or the level corresponding to different knowledge fragments in this embodiment can be 3 levels. When determining the knowledge fragment credibility, the knowledge source authority, knowledge timeliness and knowledge consistency result can be assigned weights, and the knowledge fragment credibility is obtained based on the weights and the corresponding scores. For the sake of understanding, taking a three-dimensional grading standard as an example (5 levels, level 1 is the lowest and level 5 is the highest), the source authority (the weight can be set to 40%): distinguish whether the knowledge comes from a national authority (such as A bank), an industry authority (such as B bank annual report) or a general source (such as a personal blog).

[0057] Table 1 Source authority of knowledge fragment

[0058]

[0059] The timeliness in this embodiment (which can be set to a weight of 30%): measures the interval between the knowledge publishing time and the current time, and the higher the timeliness is, the higher the timeliness is.

[0060] Table 2 Timeliness of knowledge fragments

[0061]

[0062] The consistency in this embodiment (weight 30%) is used to evaluate the cross verification of knowledge and other sources, and the higher the consistency score is, the higher the score is.

[0063] Table 3 Consistency of knowledge fragments

[0064]

[0065] The knowledge fragment credibility score in this embodiment = (authority score x 40%) + (timeliness score x 30%) + (consistency score x 30%), and is divided into A (85-100), B (60-84), and C (<60) three levels according to the score, which correspond to “high credibility”, “medium credibility”, and “low credibility” respectively. This embodiment determines the credibility of the knowledge fragment through the three indexes, which improves the accuracy of the credibility determination. In the knowledge fragment credibility evaluation, the prior art only relies on a single dimension (such as source authority) to score statically, and it is difficult to handle multi-source conflicts and dynamic changes. The three-dimensional grading mechanism of “authority x 40% + timeliness x 30% + consistency x 30%” of the scheme can comprehensively cover the credibility influencing factors: for example, in the financial scenario, the 2024 annual report of A bank (authority 4 levels, timeliness 5 levels, consistency 4 levels, score 4.3) and the 2022 ordinary financial article (authority 2 levels, timeliness 2 levels, consistency 1 level, score 1.7), can accurately filter high-quality knowledge through multi-dimensional weighting; at the same time, the timeliness is automatically adjusted with time, and the consistency supports multi-source verification, such as when the bond rating information is updated from “B bank AA report 1 month ago + announcement” to “2 institutions AA report within 3 days”, the new information can be selected preferentially according to the high timeliness and consistency, solving the problems of static evaluation and conflict processing.

[0066] It needs to be further explained that based on any of the above embodiments, before determining the knowledge fragment metadata corresponding to each hop path in the retrieval path and performing compliance verification on the retrieved knowledge fragments based on the compliance rule graph during the retrieval process, it can also include: constructing a compliance rule graph based on compliance rule metadata; wherein the compliance rule metadata includes rule unique identifier, rule content, applicable business scenario, trigger condition, violation operation and rule update time; embedding the compliance rule graph into the knowledge graph in the form of retrieval enhancement. This embodiment designs metadata for each compliance rule and stores it in the compliance rule graph. The compliance rule metadata corresponding to each compliance rule can include rule unique identifier, rule content, applicable business scenario, trigger condition, violation operation and rule update time, please refer to Table 4 for details. Table 4 is a compliance rule metadata table provided by an embodiment of the application. This embodiment embeds dynamic compliance rules into the knowledge graph, thereby realizing full-link compliance risk blocking.

[0067] Table 4: A compliance rule metadata table

[0068]

[0069] It needs to be further explained that based on any of the above embodiments, the above determination of knowledge fragment metadata corresponding to each hop path in the retrieval path and the compliance verification on the retrieved knowledge fragments based on the compliance rule graph during the retrieval process can include: generating a unique path identifier of the retrieval path and determining the knowledge fragment metadata corresponding to each hop path in the retrieval path; wherein the knowledge fragment reliability contribution is determined by the current hop knowledge fragment reliability and the path attenuation coefficient; the cumulative path reliability is determined by the knowledge fragment reliability contribution. This embodiment generates a unique path_id (path identifier) for each round of multi-hop retrieval, and records knowledge fragment metadata for each hop path (hop_id). Please refer to Table 5 for details.

[0070] Table 5: A knowledge fragment metadata table

[0071]

[0072] The hop_reliability reliability contribution of the knowledge fragment corresponding to each hop path in this embodiment is determined by its source_reliability source reliability score and path attenuation coefficient. Assuming that the historical reliability decays by 10% and the current hop contributes by 10% with each additional hop:

[0073] hop_reliability(n) reliability contribution = source_reliability(n) source reliability score x 0.1;

[0074] path_reliability(n) is the cumulative path reliability from the initial to the hop = path_reliability(n-1) x 0.9 + hop_reliability(n).

[0075] The attenuation coefficient in this embodiment can be adjusted according to the business scenario. After the end of each round of multi-hop retrieval, the knowledge_id, the reliability contribution hop_reliability, and the final cumulative path reliability path_reliability of each hop in the path are stored completely, and the result reliability can be traced back. Through multi-hop retrieval reliability tracking, this embodiment realizes full-path reliability and traceability.

[0076] It should be further explained that, based on any of the above embodiments, the knowledge fragment metadata corresponding to each hop in the retrieval path is determined, and the retrieved knowledge fragments are checked for compliance based on the compliance rule graph during the retrieval process, which can include:

[0077] S1021, before the retrieval, pre-filtering rules are pulled from the compliance rule graph according to the business scenario, retrieval filtering conditions are generated, and the target reasoning problem is filtered based on the retrieval filtering conditions to obtain a filtered reasoning problem;

[0078] S1022, determine the knowledge fragment to be checked based on the filtered reasoning problem, and match the checking conditions corresponding to the business scenario of the knowledge fragment to be checked;

[0079] S1023, check the knowledge fragment to be checked based on the checking conditions;

[0080] S1024, if it is determined that the knowledge fragment violates the rules based on the checking conditions, a violation operation is performed, which includes terminating the retrieval and returning a compliance prompt, or triggering a re-retrieval until the compliance rules are met or the re-retrieval upper limit is reached.

[0081] The embodiment realizes the embedding of compliance rule verification in each stage in the multi-hop retrieval process. For example, before retrieval: rule pre-filtering. According to the business scenario (such as “financial product Q&A”), the multi-hop retrieval controller pulls the pre-filtering rules from the compliance rule atlas, and generates retrieval filtering conditions. For example, if the rule contains “prohibit ‘guaranteed return’”, the first-hop retrieval automatically excludes knowledge containing the keyword. During retrieval: intermediate result verification. After each hop retrieval is completed, the core content of the knowledge fragment is extracted, and the corresponding intermediate verification rule (such as “risk prompt needs to contain ‘past performance does not represent the future’”) is matched. If the rule is violated, perform the rule_action operation: terminate: terminate the retrieval and return the compliance prompt. Recheck: trigger recheck until the rule is met or the recheck upper limit is reached. It can be understood that the compliance rules in the embodiment need to be updated in real time, and the rule atlas receives regulatory or internal compliance updates through a real-time data pipeline. The controller synchronizes the latest rules at regular intervals (such as every minute) to ensure that the retrieval always adapts to the latest requirements.

[0082] S103, when reaching the last hop of the retrieval path, generating a target reasoning result based on the complete knowledge fragment retrieved by using a large language model, and determining a final confidence corresponding to the target reasoning result; wherein the final confidence is determined based on the final cumulative path confidence, the coverage of the reliable knowledge fragment, and the average reliable contribution.

[0083] In the embodiment, when reaching the last hop of the retrieval path, all knowledge fragments based on the target reasoning question are obtained, so that the large language model is used for reasoning based on the knowledge fragments to obtain the target reasoning result. The final confidence in the embodiment is the cumulative path confidence corresponding to the last hop of the retrieval path; the coverage of the reliable knowledge fragment in the embodiment is the proportion of the reliable knowledge fragment in the entire target reasoning result in the entire target reasoning result, and the reliable knowledge fragment is the knowledge fragment reaching the set confidence level. The average reliability contribution in the embodiment is the average value of the reliability contribution of each knowledge fragment in the process of generating the target reasoning result. It needs to be further explained that the embodiment is different in the reference strategy of knowledge fragments of different confidence levels when generating the target reasoning result based on the complete knowledge fragment (i.e. all knowledge fragments retrieved). Therefore, the accuracy of generating the target reasoning result is improved. For details, please refer to Table 6.

[0084] Table 6: Reference strength table of knowledge fragments of different levels

[0085]

[0086] It needs to be further explained that based on any of the above embodiments, when the last hop of the retrieval path is reached, generating a target reasoning result based on the retrieved complete knowledge fragment using a large language model and determining the final credibility corresponding to the target reasoning result can include:

[0087] S1031, determining the key fact points and the logic chain of the target reasoning result.

[0088] The key fact points in this embodiment refer to atomic and indivisible data units retrieved from the knowledge graph to support the final answer. Each key fact point represents a minimum information unit, which usually corresponds to a triple in the knowledge graph. The logic chain in this embodiment is to sequentially link the dispersed key fact points according to their internal causal, logical or chronological relationships, thereby forming a complete and coherent reasoning path. It explains the process of "how to derive the final answer step by step from known facts", and is the skeleton and story line of reasoning.

[0089] S1032, removing the current to-be-evaluated knowledge fragment from the complete knowledge fragment, and regenerating the answer based on the remaining retrieved knowledge fragment using the large language model to obtain the removed reasoning result.

[0090] In this embodiment, the current to-be-evaluated knowledge fragment is removed, and the answer is regenerated based on the remaining retrieved knowledge fragment to obtain the removed reasoning result.

[0091] S1033, determining the semantic difference degree between the target reasoning result and the removed reasoning result, and determining the accuracy decay degree of the removed reasoning result based on the key fact points and the logic chain.

[0092] The embodiment calculates the reference score of the current knowledge segment from two dimensions of semantic difference degree and accuracy attenuation degree. The semantic difference degree in the embodiment is a numerical index for quantifying the difference between two texts in the semantic level. It is obtained by calculating the distance between the vector representations of the two texts in a high-dimensional space. The embodiment does not limit the specific method for calculating the semantic difference degree. For example, the embodiment can determine the semantic difference degree based on semantic similarity; or the embodiment can also determine the semantic difference degree based on Euclidean / Manhattan distance. The semantic difference degree in the embodiment is the complement of the cosine similarity. It measures the degree of deviation in direction between two vectors, and the value closer to 1 indicates a greater semantic difference. The greater the semantic difference degree in the embodiment, the greater the intervention and influence of the current knowledge segment on the generation of the target reasoning result by the large language model. For example, the cosine similarity between the target reasoning result and the removed reasoning result is calculated by Sentence-BERT (a feature vector is generated for the reasoning result). The lower the similarity, the greater the difference, and the higher the reference degree (formula: semantic difference degree = 1-similarity). The embodiment can use the large language model to compare the differences between the two in key fact integrity, logical coherence, and question relevance. If the removed reasoning result has fact missing (such as missing “guarantor default record”) or logical break, the attenuation degree is high, and the reference degree is higher.

[0093] S1034, determine the knowledge segment reference degree corresponding to the current knowledge segment to be evaluated based on the semantic difference degree and the accuracy attenuation degree.

[0094] The embodiment can assign different weights to the semantic difference degree and the accuracy attenuation degree when determining the knowledge segment reference degree. For example, knowledge segment reference degree = semantic difference degree x 50 + accuracy attenuation degree x 50. The higher the score, the more critical the knowledge is to the answer.

[0095] S1035, determine the knowledge segment contribution degree based on the knowledge segment credibility and the knowledge segment reference degree of the current knowledge segment to be evaluated, and determine the average credible contribution degree based on the knowledge segment contribution degrees corresponding to all knowledge segments.

[0096] The embodiment constructs a final result credibility calculation model by taking the knowledge segment credibility and the knowledge segment reference degree as two dimensions. For each knowledge segment, the knowledge segment contribution degree = knowledge segment credibility x (reference score / 100). For example, a certain knowledge segment has a credibility of 80 points (B level) and a knowledge segment reference degree of 90 points, so the contribution degree = 80 x 0.9 = 72 points. If another knowledge has a credibility of 90 points (A level) but a reference degree of 30 points, the contribution degree = 90 x 0.3 = 27 points. The latter has a high credibility but limited actual impact.

[0097] S1036, determine the final credibility based on the final accumulated path credibility, average credibility contribution degree and credible knowledge fragment coverage corresponding to the complete knowledge fragment.

[0098] The final credibility corresponding to the target reasoning result in this embodiment is (final accumulated path credibility x 0.3) + (average credibility contribution degree x 0.5) + (credible knowledge fragment coverage x 0.2), wherein 0.3, 0.5 and 0.2 are weights. The final accumulated path credibility in this embodiment is the original multi-hop accumulated credibility (reflecting the reliability of the search link); the average credibility contribution degree is the average value of the contribution degrees of all knowledge fragments (reflecting the credibility of the key knowledge); and the credible knowledge fragment coverage is the original "proportion of credible knowledge support content" (reflecting the overall knowledge coverage quality). By quantitatively analyzing the actual influence weight of each search result on the generated answer, a more accurate overall evaluation is formed in combination with the knowledge credibility. This mechanism extends the credibility evaluation from "knowledge itself credibility" to "actual contribution value of knowledge to the answer", solving the problem of "high credibility but low reference degree knowledge excessively influencing the result" in the traditional evaluation.

[0099] It needs to be further explained that based on any of the above embodiments, when reaching the last hop of the search path, the target reasoning result is generated based on the complete knowledge fragment retrieved by using a large language model, and the final credibility corresponding to the target reasoning result is determined, which can include: determining the credibility level corresponding to each knowledge fragment in the complete knowledge fragment, determining the credibility weight based on the credibility level; determine the credible knowledge fragment coverage based on the credibility weight and the corresponding coverage of each knowledge fragment. This embodiment no longer treats all retrieved knowledge fragments equally, but rather performs weighted calculation according to their inherent credibility, thereby obtaining a more accurate and more reflective "coverage" index of the reliability of the answer, thereby improving the accuracy of the determination of the credible knowledge fragment coverage.

[0100] The model reasoning method provided by the embodiment of the application can comprise: S101, determining a retrieval path in a knowledge graph corresponding to a target reasoning question; S102, determining knowledge segment metadata corresponding to each hop path in the retrieval path, and performing compliance verification on the retrieved knowledge segments based on a compliance rule graph in a retrieval process; wherein the knowledge segment metadata comprises a knowledge segment identifier, a knowledge segment credibility, a knowledge segment credibility contribution, and an accumulated path credibility; the compliance rule graph is a graph constructed based on compliance rule metadata; S103, when the last hop of the retrieval path is reached, generating a target reasoning result by using a large language model based on the complete knowledge segments retrieved, and determining a final credibility corresponding to the target reasoning result; wherein the final credibility is determined based on a final accumulated path credibility, a credible knowledge segment coverage, and an average credible contribution degree. The application embeds compliance rules into each link of the multi-hop retrieval based on credibility, realizes compliance control of the retrieval process from the source, reduces compliance loopholes in the knowledge segment retrieval process, reduces the risk of generating illegal content, improves the credibility of the model reasoning result, and accurately evaluates the credibility of the target reasoning result.

[0101] In order to make the application more convenient to understand, please refer to Figure 2 , Figure 2 The flowchart example of the model reasoning method provided by the embodiment of the application can specifically comprise:

[0102] S201, determining a retrieval path based on a target reasoning question by using a knowledge graph, generating a unique path identifier for the retrieval path, and determining knowledge segment metadata corresponding to each hop path in each retrieval path.

[0103] The knowledge segment metadata in this embodiment comprises a knowledge identifier corresponding to each hop path, a knowledge credibility, a knowledge credibility contribution, and an accumulated path credibility from the initial to the hop; wherein the knowledge credibility contribution of each hop is determined by the knowledge credibility of the current hop and a path attenuation coefficient.

[0104] S202, while retrieving the knowledge segments, pulling pre-filtering rules from the compliance rule graph according to the business scenario, generating retrieval filtering conditions, filtering the target reasoning question based on the retrieval filtering conditions, and obtaining a filtered reasoning question.

[0105] S203, when the knowledge segments are obtained based on the filtered reasoning question, and the next knowledge segment is retrieved based on the current knowledge segment, matching the verification filtering conditions corresponding to the business scenario, and if illegal, performing illegal operation.

[0106] The illegal operation in this embodiment comprises terminating the retrieval, returning a compliance prompt, and triggering re-retrieval until the rules are met or the re-retrieval upper limit is reached.

[0107] S204, after the end of the multi-hop retrieval, the knowledge fragment identifier, the knowledge fragment credibility contribution and the final cumulative path credibility of each hop in the complete storage path are stored.

[0108] S205, based on all the retrieved knowledge fragments, a large language model is used to generate a target reasoning result, and key fact points and logical chains of the target reasoning result are determined.

[0109] S206, remove the current knowledge fragment to be evaluated from the complete knowledge fragment, and use a large language model to regenerate the answer based on the remaining retrieved knowledge fragments to obtain the removed reasoning result.

[0110] S207, determine the semantic difference degree between the target reasoning result and the removed reasoning result, and determine the accuracy attenuation degree of the removed reasoning result based on the key fact points and the logical chains.

[0111] S208, determine the knowledge fragment reference degree corresponding to the current knowledge fragment to be evaluated based on the semantic difference degree and the accuracy attenuation degree.

[0112] S209, determine the knowledge fragment contribution degree based on the knowledge fragment credibility of the current knowledge fragment to be evaluated and the knowledge fragment reference degree, and determine the average credibility contribution degree based on the knowledge fragment contribution degrees corresponding to all knowledge fragments.

[0113] S210, determine the final credibility corresponding to the current target reasoning result based on the final cumulative path credibility corresponding to the complete knowledge fragment, the average credibility contribution degree and the credibility knowledge fragment coverage.

[0114] For ease of understanding, it can specifically include:

[0115] I. Knowledge source credibility grading implementation steps.

[0116] 1. Knowledge fragment preprocessing and dimension labeling.

[0117] The financial field knowledge (such as regulatory documents, financial reports, news, etc.) is structured and processed, and each piece of knowledge (knowledge_id) is labeled with three-dimensional original information:

[0118] Authority labeling: match the level according to the source type (such as "B Bank 2024 C Monetary Policy Report"→ authority 5 levels);

[0119] Time effectiveness labeling: calculate the interval between the publication time and the current time (such as "published in June 2024, current September 2024"→ interval 3 months→ time effectiveness 5 levels);

[0120] Consistency annotation: Through multi-source comparison, the number of high-level sources supporting this knowledge is counted (e.g., "a certain enterprise overdue data" is supported by B bank credit report (5 levels) and A bank annual report (4 levels) → consistency level 3).

[0121] 2. Total reliability calculation.

[0122] Formula: Knowledge piece reliability source_reliability= (auth_score*0.4) + timeliness_score*0.3) + consistency_score*0.3).

[0123] Example: The knowledge about "personal housing loan non-performing rate" in "A bank's 2024 semi-annual report": authority: A bank annual report → 4 levels (auth_score=4); timeliness: published in July 2024, current September 2024 → interval 2 months → 5 levels (timeliness_score=5); consistency: consistent with B bank's financial stability report (5 levels) data → 3 levels (consistency_score=3); total reliability: 4*0.4+5*0.3+3*0.3=1.6+1.5+0.9=4.0 (normalized to 0-100 points as 80 points → B level, medium trust).

[0124] II. Implementation steps of multi-hop search reliability tracking.

[0125] 1. Path metadata initialization.

[0126] When each round of multi-hop search starts, a unique path_id (such as path_20240903_001) is generated, and path_reliability (0)=100 (initial reliability) is initialized.

[0127] 2. Each hop reliability calculation and transmission.

[0128] Formula: Knowledge piece reliability contribution hop_reliability(n)=knowledge piece reliability (knowledge piece source reliability) source_reliability(n)*0.1; cumulative path reliability path_reliability(n)=cumulative path reliability path_reliability(n-1)*0.9+knowledge piece reliability contribution hop_reliability(n);

[0129] Example: User query "whether there is risk in customer A's housing loan", multi-hop search path is:

[0130] 1st hop (hop_1): Client A→ Loan Contract B (knowledge_id=doc_001, source_reliability=90 points);

[0131] hop_reliability(1)=90×0.1=9;

[0132] path_reliability(1)=100×0.9+9=99;

[0133] 2nd hop (hop_2): Contract B→ Guarantor Enterprise C (knowledge_id=doc_002, source_reliability=85 points);

[0134] hop_reliability(2)=85×0.1=8.5;

[0135] path_reliability(2)=99×0.9+8.5=97.6;

[0136] 3rd hop (hop_3): Enterprise C→ 2024 Q2 Default Record (knowledge_id=doc_003, source_reliability=92 points);

[0137] hop_reliability(3)=92×0.1=9.2;

[0138] path_reliability(3)=97.6×0.9+9.2=97.04;

[0139] The final path reliability is 97.04 points.

[0140] 3. Path storage and backtracking.

[0141] Store the complete metadata (hop_id, knowledge_id, source_reliability, hop_reliability, path_reliability) of path_id=path_20240903_001 in the graph database, supporting backtracking of each step of "Client A risk judgment" during auditing.

[0142] III. Dynamic compliance rule embedding implementation steps.

[0143] 1. Compliance rule graph construction.

[0144] Create rule nodes in the graph database and store compliance rule metadata, as shown in Table 7:

[0145] Table 7: An illustrative table of compliance rule metadata

[0146]

[0147] 2. Retrieve the full-process compliance verification.

[0148] (1) Pre-filtering before retrieval:

[0149] Identify the user query scenario as "financial product promotion", and pull rules with rule_scope = financial product promotion from the rule graph;

[0150] Generate filter conditions: exclude knowledge containing ["guaranteed return"], and filter content with ">15%" when retrieving "annual yield" related knowledge.

[0151] (2) Intermediate verification during retrieval:

[0152] The second hop retrieves "the annual yield of a certain product is 16%", triggering r_002 (rule_trigger = ["annual yield", ">15%"]);

[0153] Perform rule_action = recheck, re-retrieve the product yield information, and finally obtain the compliance knowledge of "annual yield 14%".

[0154] (3) Real-time rule update:

[0155] On September 5, 2024, new rule r_003 ("need to prompt'market volatility risk'") is added through Kafka push to the rule graph;

[0156] The controller polls synchronously every minute, and the update is completed at 10:01 on the same day, and subsequent retrieval automatically includes the rule verification.

[0157] Four, reference strength of large language model and result generation implementation steps.

[0158] 1. Reference strategy based on credibility.

[0159] A-level knowledge (such as B bank announcement, 90 points): as the core reference, generate the time label "according to the B bank Q2 report in 2024, the guarantor of customer A has a default record";

[0160] B-level knowledge (such as A bank annual report, 80 points): it needs to be cross-verified with another B-level knowledge (such as XX local notification, 75 points), and the generation time is expressed as "combined with A bank annual report and XX local notification, there is potential risk in customer A's loan";

[0161] C-level knowledge (e.g., XX Finance Blog, 50 points): As a supplement, marked as "XX Finance Blog mentions Client A's historical loan records, but the information has not been verified by an authoritative source."

[0162] 2. Reference degree calculation of search results.

[0163] Steps:

[0164] (1) Reference generation: Based on the complete search results (doc_001 + doc_002 + doc_003), generate the reference answer answer_original, containing the key fact "Client A's loan is guaranteed by Enterprise C, C has a default record in Q2 2024, and there is a risk";

[0165] (2) Remove and regenerate: Remove doc_003 (Enterprise C default record) to generate answer_removed, which contains "Client A's loan is guaranteed by Enterprise C, and no risk has been found";

[0166] (3) Difference quantification:

[0167] Semantic difference degree: The similarity calculated by Sentence-BERT is 0.4 → semantic difference degree = 1-0.4 = 0.6;

[0168] Accuracy attenuation degree: answer_removed lacks the key fact "Enterprise C default" → attenuation degree = 0.9;

[0169] Reference degree score: 0.6*50 + 0.9*50 = 30 + 45 = 75 points.

[0170] 3. Confidence and reference degree fusion evaluation.

[0171] Formula: The contribution degree of each piece of knowledge = source_reliability * (reference degree score / 100);

[0172] Final result confidence = (path_reliability * 0.3) + (average contribution degree * 0.5) + (coverage of reliable knowledge pieces * 0.2).

[0173] Example:

[0174] The source_reliability of three-hop knowledge is 90, 85, and 92 points, respectively, and the reference degree score is 60, 50, and 75 points, respectively;

[0175] Single knowledge contribution degree:

[0176] doc_001: 90*60 / 100 = 54;

[0177] doc_002: 85*50 / 100 = 42.5;

[0178] doc_003: 92*75 / 100 = 69;

[0179] Average contribution: (54 + 42.5 + 69) / 3 = 55.17;

[0180] Path credibility: 97.04 points;

[0181] Credible knowledge fragment coverage: 90% (90% of the content in the answer is supported by B-level and above knowledge);

[0182] Final result credibility: 97.04*0.3 + 55.17*0.5 + 90*0.2 = 29.11 + 27.59 + 18 = 74.7 points → medium credibility, generate prompt "the result is based on multi-source credible knowledge, the core basis is the default record of enterprise C, and it is recommended to verify combined with the latest credit data".

[0183] The present application aims at the core defects of the prior art in knowledge credibility evaluation, single static, multi-hop retrieval cannot be traced, compliance verification is lagging behind. The overall solution of credible multi-hop retrieval and generation is formed through three improvements, realizing the unity of technology and business value.

[0184] In the knowledge credibility evaluation, the prior art only relies on single dimension (such as source authority) static scoring, which is difficult to handle multi-source conflicts and dynamic changes. The three-dimensional grading mechanism of "authority x 40% + timeliness x 30% + consistency x 30%" can comprehensively cover the credibility influencing factors.

[0185] In the field of multi-hop retrieval, the prior art has no path record and cumulative credibility calculation, and complex scenes are easy to be confused. The scheme solves this problem through the "path metadata + attenuation weighting" mechanism: a unique path_id is generated for each round of retrieval, and the knowledge ID, source credibility and single-hop contribution of each hop are recorded. For example, the risk control conclusion "customer A is high risk" can be traced back to "customer A → guarantor B (B bank credit, 90 points) → B default (national financial supervision and administration general office notification, 95 points)", realizing full link traceability; design the formula "path credibility = previous hop x 0.9 + current hop x 0.1", the final path credibility of 3-hop retrieval (score 90, 85, 92) is 88.75, combined with the credibility of knowledge coverage quantification, and based on the "entity-relation" structured jump of graph, it avoids the path deviation of traditional text multi-hop, and adapts to the complex retrieval scene across sources.

[0186] In terms of compliance management, the prior art has the problems of post-filtering, rule updating lag and lack of full-link management. The scheme breaks through the mechanism of "rule atlas + full-process embedding": the scene rules are pulled before retrieval (such as the prohibition of "guaranteed return" in financial propaganda), and the illegal documents are directly filtered. When querying "18% annual return guaranteed product", the illegal information is blocked at the first hop; the rule atlas synchronizes the new rules through the real-time pipeline, and the controller updates every minute, automatically adding risk prompt verification; the compliance verification is embedded in the whole process of "before-retrieval, during-retrieval and after-retrieval", the risk prompt is triggered for re-checking when the intermediate hop misses, and the result is re-checked for illegal expressions, so as to prevent illegal information from penetrating from the source and reduce compliance risks.

[0187] Each improvement of the scheme is aimed at the explicit defects of the prior art, and forms a closed loop through "mechanism design → logical verification → effect landing": the source credibility classification is solved by multi-dimensional dynamic calculation, the "inaccurate credibility evaluation" is solved, the multi-hop credibility tracking is solved by full-link recording and weighted calculation, the "path is not traceable and credibility is not quantifiable" is solved, and the dynamic compliance embedding is solved by full-process rule management. "Compliance lag and risk postposition" is solved. Finally, in the financial and other scenes, "knowledge credibility can be screened, path can be traced and quantified, and compliance can be controlled and followed", without the need for a large number of additional hardware costs (based on existing graph database and vector database modification), clear structure (module cooperation and linkage), convenient use (rules and credibility are automatically calculated without manual intervention), and true realization of the unity of technical improvement and business value.

[0188] The model reasoning device provided by the embodiment of the application will be introduced below. The model reasoning device described below can be correspondingly referred to the model reasoning method described above.

[0189] For details, please refer to Figure 3 , Figure 3 The structural schematic diagram of the model reasoning device provided by the embodiment of the application can include:

[0190] The retrieval path determination module 100 is configured to determine a retrieval path in a knowledge graph corresponding to a target reasoning question;

[0191] The compliance verification module 200 is configured to determine knowledge segment metadata corresponding to each hop path in the retrieval path, and perform compliance verification on the retrieved knowledge segments based on a compliance rule atlas during the retrieval process; wherein the knowledge segment metadata includes knowledge segment identification, knowledge segment credibility, knowledge segment credibility contribution and cumulative path credibility; and the compliance rule atlas is an atlas constructed based on compliance rule metadata;

[0192] The target reasoning module 300 is configured to generate a target reasoning result based on the complete knowledge fragment retrieved by using a large language model when reaching the last hop of the retrieval path, and determine a final confidence degree corresponding to the target reasoning result, wherein the final confidence degree is determined based on a final accumulated path confidence degree, a confident knowledge fragment coverage degree, and an average confident contribution degree.

[0193] Further, based on any of the above embodiments, the model reasoning device can further include:

[0194] A knowledge consistency result determination module is configured to obtain a knowledge consistency result based on cross-validation of the current knowledge fragment and knowledge fragments from other sources.

[0195] A knowledge fragment confidence degree determination module is configured to determine a knowledge fragment confidence degree corresponding to each knowledge fragment based on a knowledge source authority of each knowledge fragment, a knowledge timeliness, and the knowledge consistency result, wherein the knowledge source authority is determined based on a source authority level corresponding to the knowledge fragment, and the knowledge timeliness is determined based on a difference between a publication time of the knowledge fragment and a current time.

[0196] Further, based on any of the above embodiments, the model reasoning device can further include:

[0197] A compliance rule knowledge graph construction module is configured to construct the compliance rule graph based on the compliance rule metadata, wherein the compliance rule metadata includes a rule unique identifier, a rule content, an applicable business scenario, a trigger condition, a violation operation, and a rule update time.

[0198] An embedding module is configured to embed the compliance rule graph into the knowledge graph in a retrieval-enhanced form.

[0199] Further, based on any of the above embodiments, the compliance verification module 200 can include:

[0200] A knowledge fragment metadata determination module is configured to generate a unique path identifier of the retrieval path, and determine the knowledge fragment metadata corresponding to each hop path in the retrieval path.

[0201] The knowledge fragment confidence degree contribution is determined by the knowledge fragment confidence degree of the current hop and a path attenuation coefficient, and the accumulated path confidence degree is determined by the knowledge fragment confidence degree contribution.

[0202] Further, based on any of the above embodiments, the compliance verification module 200 can include:

[0203] The reasoning problem compliance processing unit is configured to, before the searching, pull pre-filtering rules from the compliance rule atlas according to a business scenario, generate a search filtering condition, and filter the target reasoning problem based on the search filtering condition to obtain a filtered reasoning problem.

[0204] The knowledge segment compliance processing unit is configured to determine a to-be-verified knowledge segment based on the filtered reasoning problem, and match a verification condition corresponding to a business scenario of the to-be-verified knowledge segment.

[0205] The verification unit is configured to verify the to-be-verified knowledge segment based on the verification condition.

[0206] The violation operation execution unit is configured to, if it is determined that the knowledge segment is in violation based on the verification condition, perform a violation operation, the violation operation including returning a compliance prompt by terminating the searching, or triggering re-search until the compliance rule is met or a re-search upper limit is reached.

[0207] Further, based on any of the above embodiments, the target reasoning module 300 can include:

[0208] The key fact point and logic chain determination unit is configured to determine a key fact point and a logic chain of the target reasoning result.

[0209] The removed reasoning result determination unit is configured to remove a current to-be-evaluated knowledge segment from the complete knowledge segment, regenerate an answer based on remaining search knowledge segments by using the large language model, and obtain a removed reasoning result.

[0210] The accuracy attenuation degree determination unit is configured to determine a semantic difference degree between the target reasoning result and the removed reasoning result, and determine an accuracy attenuation degree of the removed reasoning result based on the key fact point and the logic chain.

[0211] The knowledge segment reference degree determination unit is configured to determine a knowledge segment reference degree corresponding to the current to-be-evaluated knowledge segment based on the semantic difference degree and the accuracy attenuation degree.

[0212] The average credible contribution degree determination unit is configured to determine a knowledge segment contribution degree based on a knowledge segment credibility of the current to-be-evaluated knowledge segment and the knowledge segment reference degree, and determine the average credible contribution degree based on knowledge segment contribution degrees corresponding to all knowledge segments.

[0213] The final credibility determination unit is configured to determine the final credibility based on a final cumulative path credibility corresponding to the complete knowledge segment, the average credible contribution degree, and a credible knowledge segment coverage degree.

[0214] Further, based on any of the above embodiments, the target reasoning module 300 can include:

[0215] a weight determination unit configured to determine a confidence level corresponding to each knowledge fragment in the complete knowledge fragment, and determine a confidence weight based on the confidence level;

[0216] a trusted knowledge fragment coverage determination unit configured to determine the trusted knowledge fragment coverage based on the confidence weight and the corresponding coverage of each knowledge fragment.

[0217] It should be noted that the order of the modules and units in the above model inference device can be changed without affecting the logic.

[0218] The model inference device provided by the embodiment of the present application can comprise: a retrieval path determination module 100 configured to determine a retrieval path in a knowledge graph corresponding to a target inference question; a compliance verification module 200 configured to determine knowledge fragment metadata corresponding to each hop path in the retrieval path, and perform compliance verification on the retrieved knowledge fragments based on a compliance rule graph during the retrieval process; wherein the knowledge fragment metadata comprises knowledge fragment identification, knowledge fragment confidence, knowledge fragment confidence contribution, and cumulative path confidence; the compliance rule graph is a graph constructed based on compliance rule metadata; a target inference module 300 configured to generate a target inference result based on the complete knowledge fragments retrieved by using a large language model when reaching the last hop of the retrieval path, and determine a final confidence corresponding to the target inference result; wherein the final confidence is determined based on the final cumulative path confidence, the trusted knowledge fragment coverage, and the average confidence contribution degree. The present application fully embeds compliance rules into each link of the multi-hop retrieval based on confidence, realizes compliance control of the retrieval process from the source, reduces compliance loopholes in the knowledge fragment retrieval process, reduces the risk of generating illegal content, improves the confidence of the model inference result, and accurately evaluates the confidence of the target inference result.

[0219] A model inference device provided by an embodiment of the present application will be described below. The model inference device described below can be referred to in correspondence with the model inference method described above.

[0220] Please refer to Figure 4 , Figure 4 A structural schematic diagram of a model inference device provided by an embodiment of the present application can comprise:

[0221] A memory 10 configured to store a computer program;

[0222] A processor 20 configured to execute the computer program to implement the model inference method described above.

[0223] The memory 10, the processor 20, and the communication interface 30 communicate with each other through the communication bus 40.

[0224] In the embodiment of the present application, the memory 10 stores one or more programs, which can include program codes including computer operation instructions. In the embodiment of the present application, the memory 10 can store programs for implementing the following functions:

[0225] determining a search path in the knowledge graph corresponding to the target reasoning question;

[0226] determining knowledge fragment metadata corresponding to each hop path in the search path, and performing compliance verification on the searched knowledge fragments based on the compliance rule graph in the search process; wherein the knowledge fragment metadata includes knowledge fragment identifier, knowledge fragment credibility, knowledge fragment credibility contribution, and cumulative path credibility; the compliance rule graph is a graph constructed based on compliance rule metadata;

[0227] when reaching the last hop of the search path, generating a target reasoning result based on the complete knowledge fragment searched by using a large language model, and determining a final credibility corresponding to the target reasoning result; wherein the final credibility is determined based on the final cumulative path credibility, the credible knowledge fragment coverage, and the average credible contribution degree.

[0228] In a possible implementation, the memory 10 can include a program storage area and a data storage area, wherein the program storage area can store an operating system, and application programs required by at least one function, etc.; and the data storage area can store data created in the use process.

[0229] In addition, the memory 10 can include read-only memory and random access memory, and provide instructions and data for the processor. A part of the memory can also include NVRAM. The memory stores an operating system and operation instructions, executable modules or data structures, or subsets thereof, or extended sets thereof, wherein the operation instructions can include various operation instructions for implementing various operations. The operating system can include various system programs for implementing various basic tasks and processing hardware-based tasks.

[0230] The processor 20 can be a central processing unit (CPU), an application-specific integrated circuit, a digital signal processor, a field programmable gate array, or other programmable logic device. The processor 20 can be a microprocessor or any conventional processor, etc. The processor 20 can invoke programs stored in the memory 10.

[0231] The communication interface 30 can be an interface of a communication module, used for connecting with other devices or systems.

[0232] Of course, it needs to be explained that, Figure 4 The structure shown does not constitute a limitation on the model inference device in the embodiments of the application, and in actual applications, the model inference device can include more or fewer components than Figure 4 shown, or combine certain components.

[0233] The computer readable storage medium provided by the embodiments of the application is introduced below, and the computer readable storage medium described below can be referred to each other with the model inference method described above.

[0234] The application also provides a computer readable storage medium, which stores a computer program, and the computer program is executed by a processor to realize the steps of the model inference method described above.

[0235] The computer readable storage medium can include: a U disk, a mobile hard disk, a read-only memory (Read-Only Memory, ROM), a random access memory (Random Access Memory, RAM), a magnetic disk or an optical disk, and various media that can store program codes.

[0236] Each embodiment in the specification is described in a progressive manner, and each embodiment focuses on the difference from other embodiments. The same or similar parts of each embodiment can be referred to each other. For the device disclosed in the embodiment, since it corresponds to the method disclosed in the embodiment, the description is relatively simple, and the related parts can be referred to the method part.

[0237] The skilled person can further realize that the units and algorithm steps of each example described in combination with the embodiments disclosed herein can be realized by electronic hardware, computer software or a combination of the two. In order to clearly show the interchangeability of hardware and software, the composition and steps of each example have been described in the above description. Whether the functions are executed in hardware or software depends on the specific application and design constraints of the technical solution. The skilled person can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of the application.

[0238] Finally, it needs to be explained that, in this document, relationships such as first and second, and the like, are intended to distinguish one entity or action from another, without necessarily requiring or implying any actual such relationship or order between entities or actions. Also, the terms "comprises", "comprising", or any other variations thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but can include other elements not expressly listed or inherent to such process, method, article, or apparatus.

[0239] The model reasoning method, device, equipment and computer readable storage medium provided by the present application are introduced in detail above, the principle and implementation mode of the present application are described in this document, the above embodiment is only used to help understand the method and core idea of the present application; At the same time, for those skilled in the art, according to the idea of the present application, the specific implementation mode and application range will be changed; In view of the above, the content of the specification should not be understood as the limitation of the present application.

Claims

1. A model reasoning method, characterized in that, include: Determine the retrieval path in the knowledge graph corresponding to the target reasoning question; The metadata of the knowledge fragment corresponding to each hop in the retrieval path is determined, and compliance verification of the retrieved knowledge fragments is performed based on the compliance rule graph during the retrieval process; wherein, the metadata of the knowledge fragment includes the knowledge fragment identifier, the knowledge fragment credibility, the knowledge fragment credibility contribution, and the cumulative path credibility; wherein, the knowledge fragment credibility contribution is determined by the credibility of the knowledge fragment in the current hop and the path decay coefficient; the cumulative path credibility is determined by the knowledge fragment credibility contribution; When the last hop of the retrieval path is reached, a target reasoning result is generated based on the retrieved complete knowledge fragment using a large language model, and the final credibility corresponding to the target reasoning result is determined; wherein, the final credibility is determined based on the final cumulative path credibility, the credibility knowledge fragment coverage, and the average credibility contribution. The process includes, before determining the metadata of the knowledge fragment corresponding to each hop in the retrieval path, and before performing compliance verification on the retrieved knowledge fragments based on the compliance rule graph during the retrieval process, the following steps are also included: Based on the cross-validation of the current knowledge fragment with knowledge fragments from other sources, a knowledge consistency result is obtained; The credibility of each knowledge fragment is determined based on the authority of its source, the timeliness of its information, and the consistency of the information. The authority of the source is determined based on the authority level of the source corresponding to the knowledge fragment. The timeliness of the information is determined based on the difference between the publication time of the knowledge fragment and the current time. The compliance rule graph is constructed based on compliance rule metadata; the compliance rule metadata includes a unique rule identifier, rule content, applicable business scenario, triggering condition, violation operation, and rule update time. The compliance rule graph is embedded into the knowledge graph in a retrieval-enhanced form.

2. The model reasoning method according to claim 1, characterized in that, Determine the metadata of the knowledge fragment corresponding to each hop in the retrieval path, and perform compliance verification on the retrieved knowledge fragments based on the compliance rule graph during the retrieval process, including: Generate a unique path identifier for the retrieval path and determine the metadata of the knowledge fragment corresponding to each hop in the retrieval path.

3. The model reasoning method according to claim 1, characterized in that, Determine the metadata of the knowledge fragment corresponding to each hop in the retrieval path, and perform compliance verification on the retrieved knowledge fragments based on the compliance rule graph during the retrieval process, including: Before performing a search, pre-filtering rules are retrieved from the compliance rule graph based on the business scenario to generate search filtering conditions. The target reasoning question is then filtered based on the search filtering conditions to obtain the filtered reasoning question. Determine the knowledge fragment to be verified based on the filtered reasoning question, and match the verification conditions of the business scenario corresponding to the knowledge fragment to be verified. The knowledge fragment to be verified is verified based on the verification conditions. If a knowledge fragment is determined to be in violation based on the verification conditions, then a violation operation is performed. The violation operation includes terminating the search and returning a compliance prompt, or triggering a re-examination, until the compliance rules are met or the re-examination limit is reached.

4. The model reasoning method according to claim 1, characterized in that, Upon reaching the last hop of the retrieval path, a target inference result is generated based on the retrieved complete knowledge fragment using a large language model, and the final credibility corresponding to the target inference result is determined, including: Identify the key fact points and logical chains of the target reasoning result; Remove the current knowledge fragment to be evaluated from the complete knowledge fragment, and regenerate the answer based on the remaining retrieved knowledge fragment using the large language model to obtain the reasoning result after removal; Determine the semantic difference between the target inference result and the removed inference result, and determine the accuracy decay of the removed inference result based on the key fact points and the logical chain; The reference degree of the knowledge segment corresponding to the current knowledge segment to be evaluated is determined based on the semantic difference degree and the accuracy decay degree. The contribution of a knowledge segment is determined based on the credibility of the knowledge segment to be evaluated and the reference value of the knowledge segment, and the average credibility contribution is determined based on the contribution of all knowledge segments. The final credibility is determined based on the final cumulative path credibility corresponding to the complete knowledge fragment, the average credibility contribution, and the credibility knowledge fragment coverage.

5. The model reasoning method according to claim 1, characterized in that, Upon reaching the last hop of the retrieval path, a target inference result is generated based on the retrieved complete knowledge fragment using a large language model, and the final credibility corresponding to the target inference result is determined, including: Determine the credibility level of each knowledge segment in the complete knowledge segment, and determine the credibility weight based on the credibility level; The coverage of the trusted knowledge fragment is determined based on the credibility weight and the corresponding coverage of each knowledge fragment.

6. A model reasoning device, characterized in that, include: The retrieval path determination module is used to determine the retrieval path in the knowledge graph corresponding to the target reasoning question; The compliance verification module is used to determine the metadata of the knowledge fragment corresponding to each hop in the retrieval path, and to perform compliance verification on the retrieved knowledge fragments based on the compliance rule graph during the retrieval process. The knowledge fragment metadata includes a knowledge fragment identifier, knowledge fragment credibility, knowledge fragment credibility contribution, and cumulative path credibility. The compliance rule graph is a graph constructed based on the compliance rule metadata. The knowledge fragment credibility contribution is determined by the credibility of the knowledge fragment at the current hop and the path decay coefficient. The cumulative path credibility is determined by the knowledge fragment credibility contribution. The target reasoning module is used to generate a target reasoning result based on the retrieved complete knowledge fragment using a large language model when the last hop of the retrieval path is reached, and to determine the final credibility corresponding to the target reasoning result; wherein, the final credibility is determined based on the final cumulative path credibility, the credibility knowledge fragment coverage, and the average credibility contribution. The knowledge consistency result determination module is used to obtain the knowledge consistency result based on the cross-validation of the current knowledge fragment with knowledge fragments from other sources; The knowledge fragment credibility determination module is used to determine the credibility of each knowledge fragment based on the authority of the knowledge source, the timeliness of the knowledge, and the consistency result of the knowledge; wherein, the authority of the knowledge source is determined based on the authority level of the source corresponding to the knowledge fragment; and the timeliness of the knowledge is determined based on the difference between the publication time of the knowledge fragment and the current time. The compliance rule knowledge graph construction module is used to construct the compliance rule graph based on the compliance rule metadata; wherein, the compliance rule metadata includes a unique rule identifier, rule content, applicable business scenarios, triggering conditions, violation operations, and rule update time; An embedding module is used to embed the compliance rule graph into the knowledge graph in a retrieval-enhanced form.

7. A model reasoning device, characterized in that, include: Memory, used to store computer programs; A processor for executing the computer program to implement the steps of the model inference method as described in any one of claims 1 to 5.

8. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, implements the steps of the model inference method as described in any one of claims 1 to 5.

Images