A legal large model hallucination elimination method, system, electronic device, computer readable storage medium and computer program product

Abstract

A legal large model illusion elimination method, system, electronic device, computer readable storage medium and computer program product, wherein the legal large model illusion elimination method comprises the following steps: performing element extraction on a legal problem text, generating a structured sub-question with a sub-question identifier and retrieving an evidence record with an evidence identifier; providing the sub-question and the evidence as common inputs to two independently configured reasoning engines, respectively, converting the processing results into assertion records containing sub-question identifiers, processing stage identifiers, assertion contents and evidence identifier sets, and performing alignment comparison; generating correction control information for determining target sub-questions, target processing stages and processing parameters according to the difference records, selectively re-executing the corresponding stages until the conditions are met, and outputting a legal reply result. The beneficial effects of the present application are that the generated result difference can be structured and positioned to drive the previous processing stage to automatically backtrack, and the illusion is gradually eliminated through closed-loop iteration.

Description

Technical Field

[0001] This invention relates to the fields of artificial intelligence, natural language processing, information retrieval, and computer data processing, specifically to a method, system, electronic device, computer-readable storage medium, and computer program product for eliminating illusions in a large legal model. Background Technology

[0002] Large language models can generate highly readable responses to natural language questions, but their output is generated probabilistically. In legal information processing scenarios, input facts may lack information such as the identity of the subject, the time of the action, the applicable region, or the conditions for validity. Furthermore, legal norms may differ in version, level of validity, and scope of application. If the retrieval input is incomplete, the evidence selection is inappropriate, or the reasoning process incorrectly connects premises and conclusions, the computer system may output non-existent evidence, invalid norms, omitted conditions for validity, or conclusions inconsistent with the evidence.

[0003] Existing retrieval-enhanced generation techniques typically input the retrieved document fragments along with the user's question into the generative model. This approach can expand the model's context, but it may still produce inconsistent outputs when the retrieval request itself omits key elements, candidate evidence has temporal or scope conflicts, or the model does not correctly utilize the evidence.

[0004] Existing multi-model scoring, voting, or cross-validation techniques can detect differences between generated results. However, if only natural language answers are compared or an overall score is given, differences are difficult to reliably pinpoint to specific sub-questions, evidence records, or preceding processing stages. After discovering differences, the system often re-executes the entire process or only rewrites the final text, failing to convert the differences into structured control data that directly controls retrieval parameters and processing paths. This leads to redundant calculations and potential continuation of generation based on erroneous premises. The current legal large-scale model processing suffers from difficulties in structurally locating differences in generated results, the inability of validation results to directly drive corrections in preceding processing stages, and the waste of computational resources caused by repeatedly executing the entire process after differences occur.

[0005] Therefore, it is necessary to develop a multi-stage closed-loop scheme for computer information processing flow, which converts the output of different inference channels into assertion records that can be aligned and computed, establishes the association between the difference records and sub-problems, evidence and processing stages, and generates machine-executable correction control information from the difference records to selectively re-execute the affected processing stages, and achieves the elimination of illusion round by round through a closed-loop iterative mechanism. Summary of the Invention

[0006] The present invention aims to solve the above-mentioned technical problems.

[0007] To address the aforementioned technical problems, the first objective of this invention is to propose a method for eliminating illusions in a large legal model. A computing device extracts legal elements from the legal issue text, generating a structured set of sub-issues with sub-issue identifiers. Based on this structured sub-issue set, a legal knowledge base is retrieved to form evidence records associated with each structured sub-issue and identified by evidence identifiers. This invention employs a multi-stage closed-loop scheme oriented towards computer information processing flows. It converts the outputs of different reasoning channels into assertion records that can be aligned and computed. It establishes associations between the difference records and sub-issues, evidence, and processing stages, and generates machine-executable correction control information from the difference records to selectively re-execute affected processing stages. Through a closed-loop iterative mechanism, the illusion is eliminated round by round.

[0008] The second objective of this invention is to propose a legal large-scale illusion elimination system.

[0009] The third objective of this invention is to provide an electronic device.

[0010] The fourth objective of this invention is to provide a computer-readable storage medium.

[0011] The fifth objective of this invention is to provide a computer program product.

[0012] To achieve the above objectives, a first aspect of the present invention discloses a method for eliminating the illusion of a legal big model, comprising the following steps: Step S1: Obtaining legal question text, extracting legal elements from the legal question text, and generating a structured sub-question set with sub-question identifiers; Step S2: Retrieving a legal knowledge base based on the structured sub-question set to obtain evidence records associated with the structured sub-question set and having evidence identifiers; Step S3: Using the structured sub-question set and the evidence records as common inputs, providing them to a first inference engine and a second inference engine configured independently, respectively, the first inference engine generating a first processing result, and the second inference engine generating a second processing result; Step S4: Converting the first processing result and the second processing result into assertion records, the assertion records including sub-question identifiers, processing stage identifiers, assertion content, and a set of evidence identifiers, for those with... Step S5: Align and compare the assertion records corresponding to the sub-problem identifiers to generate a difference record; Step S6: When the difference record does not meet the consistency condition, generate correction control information based on the difference type, sub-problem identifier, and processing stage identifier in the difference record. The correction control information is used to determine the target sub-problem to be re-executed, the target processing stage, and processing parameters. The target processing stage is selected from at least one of the structured sub-problem generation stage, the evidence retrieval stage, and the reasoning processing stage; Step S7: Re-execute the target sub-problem starting from the target processing stage according to the correction control information to obtain an updated assertion record; Step S8: Repeat steps S4 to S6, perform alignment and comparison, determine the consistency condition, and re-execute until the consistency condition or the iteration termination condition is met; Step S9: Output the legal response result generated based on the final assertion record and associated with the evidence identifier.

[0013] In addition, the legal big model illusion elimination method according to the above-mentioned technical solution of this application may also have the following additional technical features: Optionally, extracting legal elements from the legal issue text includes: extracting entities, attributes, behaviors, conditions, and relationships between entities; writing the extraction results into element slots of a preset legal element structure; generating structured sub-problems based on the filled element slots; and generating a missing mask for the unfilled element slots, wherein the missing mask is used to restrict the output of deterministic assertions that depend on the corresponding missing elements.

[0014] Optionally, the legal knowledge base includes at least two of the following: a set of legal norms, a set of judgments, a set of normative documents, and a set of logical rules, each with its own retrieval index; the evidence record includes at least three of the following: evidence identifier, source type, source location information, standardized text, version identifier, effective time range, and scope of application label; retrieving the legal knowledge base includes performing deduplication, time validity filtering, and scope of application filtering on candidate evidence, and establishing a mapping between structured sub-questions and evidence records according to sub-question identifiers.

[0015] Optionally, the first inference engine and the second inference engine differ in at least one of the following: model structure, model parameters, parameter adaptation layer, prompt template, or context configuration; before generating the first processing result and the second processing result, neither inference engine receives the processing result of the other inference engine.

[0016] Optionally, the assertion content includes subject items, relation items, object items or judgment values, establishment conditions, and polarity; the alignment comparison includes aligning the assertion records according to sub-problem identifiers, legal element identifiers, and semantic representations, and comparing the differences between the mutually aligned assertion records in at least one of the judgment values, establishment conditions, polarity, and evidence identifier sets.

[0017] Optionally, the difference type includes at least one of the following: difference of missing elements, difference of missing evidence, difference of conflicting evidence, difference of time validity, difference of condition range, difference of logical relationship, and difference of conclusion polarity; the difference record also includes evidence identifiers corresponding to the difference content.

[0018] Optionally, generating the correction control information includes: when the difference type is a missing element difference, determining the target processing stage as a structured sub-problem generation stage, and updating the element slots, missing masks, or structured sub-problems; when the difference type is a difference in missing evidence, a difference in conflicting evidence, or a difference in time validity, determining the target processing stage as an evidence retrieval stage, and generating retrieval correction parameters; when the difference type is a difference in condition range, a difference in logical relationship, or a difference in conclusion polarity, determining the target processing stage as an inference processing stage, and generating inference processing parameters.

[0019] Optionally, the retrieval correction parameters include at least one of the following: query term expansion parameters, source type parameters, applicable geographical parameters, time range parameters, normative level parameters, result quantity parameters, and ranking weight parameters; the inference processing parameters include at least one of the following: assertion identifier to be reprocessed, validity condition, evidence identifier set, context configuration, and output structure constraints.

[0020] Optionally, the alignment comparison includes an analysis phase comparison and a conclusion phase comparison; if the analysis phase comparison does not meet the analysis consistency condition, the evidence record is updated according to the analysis difference record and the reasoning process is re-executed; if the conclusion phase comparison does not meet the conclusion consistency condition, the target processing phase is determined according to the conclusion difference record and re-executed from the target processing phase.

[0021] Optionally, when re-executing according to the correction control information, only the target sub-problem and its associated evidence records are reprocessed, and the processing results of other structured sub-problems that are not affected by the difference records are retained; the evidence records used in each re-executing have an iteration version identifier, and the first inference engine and the second inference engine use the same iteration version of the evidence records in the same iteration.

[0022] Optionally, the iteration termination condition includes at least one of reaching a preset maximum number of iterations, the difference improvement amount of consecutive iterations being lower than a preset improvement threshold, or exceeding a preset computing resource limit; when the legal response result is output due to meeting the iteration termination condition, at least one of the following is appended to the legal response result: a low confidence indicator, an unresolved difference record, or a missing element list.

[0023] Optionally, after step 7, step 9 is also included: recording the relationship between structured sub-questions, retrieval requests, evidence records, assertion records, difference records, correction control information, and re-execution paths, and generating audit records based on the relationship; each conclusion item in the legal response result is associated with the source location information, version identifier, and effective time range of the corresponding evidence record through evidence identifier.

[0024] To achieve the above objectives, a second aspect of the present invention proposes a legal big model illusion elimination system, comprising: a structured processing module, used to acquire legal question text, extract legal elements from the legal question text, and generate a structured sub-question set with sub-question identifiers; an evidence retrieval module, used to retrieve a legal knowledge base based on the structured sub-question set to obtain evidence records associated with the structured sub-question set and having evidence identifiers; a dual-channel reasoning module, including a first reasoning engine and a second reasoning engine configured independently, used to take the structured sub-question set and the evidence records as common inputs to generate a first processing result and a second processing result respectively; an assertion comparison module, used to convert the first processing result and the second processing result into assertion records including sub-question identifiers, processing stage identifiers, assertion content, and a set of evidence identifiers respectively, and to perform alignment comparison on assertion records with corresponding sub-question identifiers to generate difference records; and a correction control module, used to generate correction control based on the difference type, sub-question identifier, and processing stage identifier in the difference records when the difference records do not meet the consistency conditions. The system comprises: a correction control information module, a re-execution module, and a backtracking control module. The correction control information is used to determine the target sub-problem to be re-executed, the target processing stage, and processing parameters. The target processing stage is selected from at least one of the structured sub-problem generation stage, the evidence retrieval stage, and the reasoning processing stage. A re-execution module is used to re-execute the target sub-problem starting from the target processing stage according to the correction control information, obtaining an updated assertion record. A backtracking control module is used to repeatedly control the assertion comparison module to perform alignment comparison, control the correction control module to judge consistency conditions, and control the re-execution module to re-execute until the consistency conditions or iteration termination conditions are met. An output module is used to output a legal response result generated based on the final assertion record and associated with the evidence identifier. An audit module is used to record the association between the structured sub-problem, retrieval request, evidence record, assertion record, difference record, correction control information, and re-execution path, and generate an audit record based on the association. Each conclusion item in the legal response result is associated with the source location information, version identifier, and effective time range of the corresponding evidence record through the evidence identifier.

[0025] To achieve the above objectives, a third aspect of the present invention provides an electronic device including a processor and a memory, wherein the memory stores a computer program, and when the computer program is executed by the processor, the electronic device performs the method described in any one of the first aspects of the present invention.

[0026] To achieve the above objectives, a fourth aspect of the present invention provides a computer-readable storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the method described in any one of the first aspects of the present invention.

[0027] To achieve the above objectives, a fifth aspect of the present invention provides a computer program product comprising a computer program that, when executed by a processor, implements the method described in any one of the first aspects of the present invention.

[0028] This invention achieves the following beneficial effects: 1. It uniformly converts the natural language or structured output of the inference engine into assertion records with sub-problem identifiers, processing stage identifiers, and evidence identifier sets, enabling fine-grained alignment and difference calculation of results from different inference channels in the computer; 2. It converts the difference type and its associated identifiers into target sub-problems, target processing stages, and processing parameters, making the verification results machine-executable data that can directly control preceding modules, rather than just being used to give scores or rewrite the final text; 3. It selectively re-executes sub-problems affected by differences starting from the corresponding processing stage, while retaining the processing results of other sub-problems, which can reduce irrelevant retrieval, model calls, and data transmission, avoid continuing to generate based on erroneous premises, and gradually eliminate illusions through multi-stage closed-loop iteration; 4. Through the association between conclusion items and evidence identifiers, as well as the recording of retrieval, assertion comparison, and backtracking paths, the output results can be traced back to the source location, version, and validity period of the corresponding evidence. Attached Figure Description

[0029] Figure 1 A flowchart of a method for eliminating illusions in a legal big model, provided as an embodiment of the present invention.

[0030] Figure 2 This is a schematic diagram of a legal large-scale illusion elimination system provided as an embodiment of the present invention. Detailed Implementation

[0031] The present invention will be further described below with reference to the accompanying drawings and embodiments. The following embodiments are used to illustrate the technical principles of the present invention and are not intended to limit the scope of protection. Where there is no conflict, the technical features of the various embodiments can be combined with each other.

[0032] The following describes the method for eliminating the illusion of a large legal model according to an embodiment of the present invention, with reference to the accompanying drawings.

[0033] Figure 1 A flowchart illustrating a method for eliminating the illusion of a large legal model, as provided in one embodiment of the present invention. Figure 1As shown, the method for eliminating the illusion of the legal big model includes the following steps: Step S1: Obtain legal question text, extract legal elements from the legal question text, and generate a structured sub-question set with sub-question identifiers; Step S2: Search the legal knowledge base based on the structured sub-question set to obtain evidence records associated with the structured sub-question set and with evidence identifiers; Step S3: Use the structured sub-question set and the evidence records as common inputs, and provide them to a first inference engine and a second inference engine that are configured independently, respectively. The first inference engine generates a first processing result, and the second inference engine generates a second processing result; Step S4: Convert the first processing result and the second processing result into assertion records, respectively. The assertion record includes a sub-question identifier, a processing stage identifier, assertion content, and a set of evidence identifiers. For assertions with corresponding sub-question identifiers... Step S5: When the difference record does not meet the consistency condition, generate correction control information based on the difference type, sub-problem identifier, and processing stage identifier in the difference record. The correction control information is used to determine the target sub-problem to be re-executed, the target processing stage, and processing parameters. The target processing stage is selected from at least one of the structured sub-problem generation stage, the evidence retrieval stage, and the reasoning processing stage. Step S6: According to the correction control information, re-execute the target sub-problem starting from the target processing stage to obtain the updated assertion record. Step S7: Repeat steps S4 to S6 to perform alignment comparison, determine the consistency condition, and re-execute until the consistency condition or the iteration termination condition is met. Step S8: Output the legal response result generated based on the final assertion record and associated with the evidence identifier.

[0034] Specifically, the terminology and data objects are explained first: In this specification, "illusion" refers to content in the generated result that is inconsistent with the input facts, retrieved evidence, or preset inference constraints. This includes non-existent evidence, incorrect application time of evidence, missing conditions, logical conflicts, and conflicting conclusion polarities. "Elimination" refers to the process of gradually excluding the aforementioned inconsistent content from the final output through automatic verification, closed-loop backtracking, and iterative correction, reducing or even eliminating the possibility of illusory content entering the final response. "Mutually independent configuration" in this specification means that the first inference engine and the second inference engine differ in at least one of the following: model structure, model parameters, parameter adaptation layer, prompt template, or context configuration. Furthermore, before forming the processing result to be compared, neither inference engine uses the processing result of the other as a necessary input. The two inference engines can be deployed on the same computing device or on different computing nodes. The "multi-stage closed-loop reasoning" referred to in this specification refers to dividing legal question-and-answer processing into multiple stages, including a structured sub-question generation stage, an evidence retrieval stage, and a reasoning processing stage. Through inter-stage difference detection and selective backtracking, a closed-loop iterative process of "processing → verification → difference location → correction control → selective backtracking → reprocessing" is formed until the consistency condition is met or the termination condition is reached. Structured sub-question records are used to convert the processing objects in legal questions into searchable and locateable data objects. Evidence records are used to store the content of standardized evidence obtained from retrieval, along with its source, version, time, and scope of application. Assertion records are used to represent the comparable judgments made by the reasoning engine for a structured sub-question. Difference records are used to represent the differences between aligned assertion records and the corresponding sub-questions, evidence, and processing stages. Correction control information is machine-executable control data generated from difference records, used to determine the target sub-question, target processing stage, and processing parameters.

[0035] The overall processing flow is as follows: Figure 1 As shown, a method for eliminating illusions in a large legal model based on multi-stage closed-loop reasoning can be executed by a server, computer cluster, cloud computing node, or local electronic device, including steps S1 to S8. Each step is explained in detail below: Step S1: Obtain the legal question text and generate a set of structured sub-questions. The legal question text can come from a web page input box, client, speech recognition module, document parsing module, or application programming interface. The system performs entity recognition, event extraction, attribute extraction, negation recognition, condition recognition, and relation extraction on the text, writes the extraction results into the legal element structure, and assigns sub-question identifiers to each generated structured sub-question.

[0036] Step S2: Retrieve Evidence Records. The system generates retrieval requests based on each structured sub-question, retrieves one or more sets of legal knowledge, and standardizes the retrieval results to form evidence records with evidence identifiers. The system establishes a mapping between structured sub-questions and evidence records according to the sub-question identifiers.

[0037] Step S3: Perform independent dual-channel processing. The structured sub-problem set and corresponding evidence records serve as common inputs, provided to the first inference engine and the second inference engine, respectively. Under the condition of output isolation between the two inference engines, they generate the first processing result and the second processing result, respectively. The processing result can be natural language text, structured objects, function call results, or judgment data generated by the rule engine.

[0038] Step S4: Convert the first and second processing results into assertion records and perform alignment comparison. Each assertion record carries at least a sub-problem identifier, a processing stage identifier, assertion content, and a set of evidence identifiers. The system first identifies the corresponding assertions in the two channels, then compares their judgment values, validity conditions, polarity, and set of evidence identifiers to generate a difference record.

[0039] Step S5: Generate correction control information based on the difference record. The system determines the target sub-problem that needs to be re-executed, as well as the target processing stage in the structured sub-problem generation stage, evidence retrieval stage, or reasoning processing stage, based on the difference type, sub-problem identifier, and processing stage identifier, and generates the corresponding processing parameters.

[0040] Step S6: Selectively re-execute based on the corrected control information. The system only reprocesses the target sub-problem and its associated data; structured sub-problems, evidence records, and assertion records unaffected by the differences can be retained.

[0041] Step S7 involves re-execution to generate updated assertion records. The system then performs alignment and comparison again until the consistency condition or iteration termination condition is met. The above "difference detection → correction control → selective backtracking → reprocessing → re-detection" constitutes the core loop of multi-stage closed-loop inference.

[0042] Step S8: Generate output. The system synthesizes the final assertion records into a natural language response or a structured report, and associates each conclusion with an evidence identifier supporting that conclusion. If processing ends due to an iteration termination condition, an identifier indicating unresolved discrepancies, missing features, or low confidence is appended to the output.

[0043] According to the legal large-scale model illusion elimination method of this application, a computing device takes a structured set of sub-problems and evidence records as common inputs, and provides them to a first inference engine and a second inference engine configured independently. The processing results of the two inference engines are converted into assertion records containing sub-problem identifiers, processing stage identifiers, assertion content, and a set of evidence identifiers, respectively. Assertion records with corresponding sub-problem identifiers are aligned and compared to generate difference records. Based on the difference type, sub-problem identifier, and processing stage identifier in the difference records, the computing device generates correction control information, determines the target sub-problem to be re-executed, the target processing stage, and the processing parameters of that stage, and selectively re-executes starting from the target processing stage. Assertion comparison and correction processing are repeated until a consistency condition or iteration termination condition is met, forming a multi-stage closed-loop inference, and outputting a legal response result generated based on the final assertion record and associated with the evidence identifiers. Through the above closed-loop iterative mechanism, various inconsistencies in the inference process are gradually eliminated, reducing the possibility of illusory content entering the final output.

[0044] According to one embodiment of this application, extracting legal elements from the legal issue text includes: extracting entities, attributes, behaviors, conditions, and relationships between entities; writing the extraction results into element slots of a preset legal element structure; generating structured sub-problems based on the filled element slots; and generating a missing mask for the unfilled element slots, wherein the missing mask is used to restrict the output of deterministic assertions that depend on the corresponding missing elements.

[0045] Specifically, this embodiment explains the generation of structured sub-problems: the legal element structure can include subject elements, object elements, rights and obligations elements, subjective state elements, objective fact elements, and subjective-objective relationship elements. Different types of legal business can be configured with different sets and levels of elements, and it is not required that each problem have the same number of elements.

[0046] In one embodiment, a structured sub-problem record is represented as Q={qid, dtype, slots, missing_mask, query_template, stage_state}. Here, qid is the sub-problem identifier, dtype is the feature type, slots is the set of extracted feature slots, missing_mask is the missing mask, query_template is the retrieval template, and stage_state is the processing stage status.

[0047] When a legal issue text fails to provide a certain element, the system sets a missing flag in the corresponding slot. The missing mask can be used to generate a request for supplementary information, or it can serve as a constraint on inference output, preventing the inference engine from automatically filling in the missing fact as a definite fact, or marking conclusions that depend on the missing fact as conditional conclusions.

[0048] According to one embodiment of this application, the legal knowledge base includes at least two of the following: a set of legal norms, a set of judgments, a set of normative documents, and a set of logical rules, each with its own retrieval index; the evidence record includes at least three of the following: evidence identifier, source type, source location information, standardized text, version identifier, effective time range, and scope of application label; retrieving the legal knowledge base includes performing deduplication, time validity filtering, and scope of application filtering on candidate evidence, and establishing a mapping between structured sub-questions and evidence records according to sub-question identifiers.

[0049] Specifically, this embodiment explains multi-source evidence retrieval and evidence recording: The legal knowledge base may include sets of legal norms, judgments, normative documents, logical rules, contract templates, or authorized internal organizational regulations, each with its own index. The index can be an inverted index, vector index, graph index, or a hybrid index. Each set can return candidate evidence based on its own index, without requiring the retrieval results of another set as necessary input.

[0050] A structured subquestion can be converted into one or more search requests. Search requests can include keyword expressions, vector representations, entity identifiers, relational paths, time ranges, geographic ranges, canonical levels, source types, and the number of results. Multiple search requests can be executed in parallel.

[0051] Evidence records can be represented as E={eid, qid, source_type, locator, content, version, valid_time, scope, score, iteration_id}. Here, eid is the evidence identifier, qid is the associated sub-question identifier, locator is the original source location information, content is the normalized text, version is the version identifier, valid_time is the valid time range, scope is the geographical, subject, or business application scope, score is the retrieval relevance, and iteration_id is the iteration version identifier.

[0052] The retrieval module filters candidate evidence based on the target time, region, subject, and business tags corresponding to the question, considering validity and scope of application. For multiple versions of the same specification, the version matching the target time is retained; for evidence with hierarchical or scope-of-application conflicts, the evidence record retains the conflict relationship for assertion comparison and correction control.

[0053] According to one embodiment of this application, the first inference engine and the second inference engine differ in at least one of the following: model structure, model parameters, parameter adaptation layer, prompt template, or context configuration; before generating the first processing result and the second processing result, neither inference engine receives the processing result of the other inference engine.

[0054] According to one embodiment of this application, the assertion content includes a subject item, a relation item, an object item or a judgment value, a condition for establishment, and a polarity; the alignment comparison includes aligning the assertion records according to sub-problem identifiers, legal element identifiers, and semantic representations, and comparing the differences between mutually aligned assertion records in at least one of the judgment value, condition for establishment, polarity, and evidence identifier set.

[0055] Specifically, this embodiment explains assertion record transformation and difference calculation: the first inference engine and the second inference engine process the structured sub-problems and evidence records of the same iterative version, respectively. The first inference engine can be a hybrid expert model, a domain-wide language model, or a rule-based inference engine; the second inference engine can be another large language model, a model with different parameter versions, or an inference engine with different context configurations.

[0056] The processing results are converted into assertion records A={aid, qid, stage, subject, relation, value, condition, polarity, evidence_ids, confidence} through structured output constraints, information extraction models, function call interfaces, or rule parsers. Here, aid is the assertion identifier, qid is the sub-problem identifier, stage is the processing stage identifier, subject, relation, and value constitute the assertion content, condition is the condition for its validity, polarity indicates affirmation, negation, or uncertainty, evidence_ids is the set of evidence identifiers, and confidence is the confidence level.

[0057] In another embodiment of this application, assertion alignment can first form candidate pairs based on sub-problem identifiers and legal element types, and then determine the corresponding assertions based on the semantic similarity of the subject, relation, and object items. An exemplary alignment value is M = λ1 × Sim(subject) + λ2 × Sim(relation) + λ3 × Sim(value) + λ4 × Overlap(evidence_ids), where λ1 to λ4 are non-negative weights, Sim represents semantic similarity, and Overlap represents the overlap of evidence identifier sets. When M is not lower than the alignment threshold, the corresponding assertions are determined to be mutually aligned. This formula is only an example; rule matching, graph structure matching, or natural language inference models can also be used.

[0058] According to one embodiment of this application, the difference type includes at least one of the following: difference of missing elements, difference of missing evidence, difference of conflicting evidence, difference of time validity, difference of condition range, difference of logical relationship, and difference of conclusion polarity; the difference record also includes an evidence identifier corresponding to the difference content.

[0059] Specifically, for mutually aligned assertions, the system calculates the differences in judgment values, conditions for validity, polarity, and evidence sets. The difference record D may include the difference type, degree of difference, sub-problem identifier, assertion identifier, evidence identifier, and processing stage identifier. Difference types may include differences in missing elements, missing evidence, conflicting evidence, time validity, range of conditions, logical relationship, and conclusion polarity.

[0060] Consistency conditions can be defined as follows: all key assertions are aligned and the weighted difference value does not exceed a threshold; or there can be no high-risk difference types. For different types of legal business, separate rules can be set for the existence of evidence, temporal validity, scope of conditions, and polarity of conclusions.

[0061] According to one embodiment of this application, generating the correction control information includes: when the difference type is a missing element difference, determining the target processing stage as a structured sub-problem generation stage, and updating the element slots, missing masks, or structured sub-problems; when the difference type is a difference in missing evidence, a difference in conflicting evidence, or a difference in time validity, determining the target processing stage as an evidence retrieval stage, and generating retrieval correction parameters; when the difference type is a difference in condition range, a difference in logical relationship, or a difference in conclusion polarity, determining the target processing stage as an inference processing stage, and generating inference processing parameters.

[0062] According to one embodiment of this application, the retrieval correction parameters include at least one of the following: query term expansion parameters, source type parameters, applicable geographical parameters, time range parameters, normative level parameters, result quantity parameters, and ranking weight parameters; the inference processing parameters include at least one of the following: assertion identifier to be reprocessed, validity condition, evidence identifier set, context configuration, and output structure constraints.

[0063] According to one embodiment of this application, when re-execution is performed based on the correction control information, only the target sub-problem and its associated evidence records are reprocessed, while the processing results of other structured sub-problems unaffected by the difference records are retained; the evidence records used in each re-execution have an iteration version identifier, and the first inference engine and the second inference engine use the same iteration version of the evidence records in the same iteration.

[0064] Specifically, this embodiment explains the correction control information and selective backtracking: The backtracking control module maintains the mapping between difference types and target processing stages. Differences involving missing elements correspond to the structured sub-problem generation stage; differences involving missing evidence, conflicting evidence, and time validity correspond to the evidence retrieval stage; differences involving condition range, logical relationship, and conclusion polarity correspond to the reasoning processing stage. A difference record can point to multiple processing stages, and the system can select the earliest affected stage as the backtracking starting point.

[0065] When the target processing stage is the structured sub-problem generation stage, the correction control information may include the feature slots to be updated, missing masks, sub-problem relationships, and search templates. When the target processing stage is the evidence retrieval stage, the correction control information may include query term expansion parameters, source type parameters, applicable geographic parameters, time range parameters, specification level parameters, result quantity parameters, and ranking weight parameters. When the target processing stage is the inference processing stage, the correction control information may include assertion identifiers to be reprocessed, validity conditions, evidence identifier sets, context configuration, and output structure constraints.

[0066] The correction control information can be represented as C={target_qids, target_stage, stage_parameters, retained_result_ids, iteration_id}. Here, target_qids is the set of target sub-problem identifiers, target_stage is the target processing stage, stage_parameters are the processing parameters, retained_result_ids are the identifiers of unaffected processing results that should be retained, and iteration_id is the identifier for the next iteration.

[0067] The system re-executes the processing of only the affected subproblems based on `target_qids`, and reuses other processing results corresponding to `retained_result_ids`. New evidence records or assertion records are written to the new iteration version. The first and second inference engines use the same version of data in the same iteration, thereby avoiding additional differences caused by inconsistent input versions.

[0068] The aforementioned cyclical process of "difference detection → correction control generation → selective backtracking → re-execution → further difference detection" constitutes the core loop of multi-stage closed-loop reasoning. Each round of closed-loop iteration locates the difference to a specific sub-problem and processing stage. Through precise correction control information, selective re-execution is driven, gradually narrowing the inconsistency range until the consistency condition is met or the termination condition is reached, thereby achieving the elimination of illusion round by round.

[0069] In addition, this application also provides an embodiment for targeted correction of discrepancies in time validity: The system receives a legal question related to contract performance obligations, including the contract signing and performance dates. The structured processing module generates structured sub-questions such as the parties involved, contractual obligations, performance actions, target time, and liability conditions.

[0070] The evidence retrieval module retrieves two versions of the same legal norm. The first inference engine uses the first version to form an analytical assertion, while the second inference engine uses the second version to form an analytical assertion with different conditions for validity. The assertion comparison module generates a record of time validity differences based on the valid time range in the evidence record and the target time in the question.

[0071] The backtracking control module defines the target processing stage as the evidence retrieval stage. It incorporates the target time range, specification version conditions, and version matching sorting parameters into the correction control information, re-retrieving only the corresponding sub-problems. The updated evidence records retain versions valid at the target time, and both inference engines regenerate assertions based on evidence records from the same iteration version. Thus, differences in time validity are transformed into executable retrieval control parameters, eliminating the illusion caused by version confusion through closed-loop iteration.

[0072] In addition, this application also provides an example for interpreting missing factual constraints: User input only includes a description of the behavior, without providing the identity of the actor, the time of the behavior, the result, or any exceptions. The structured processing module sets a missing mask in the corresponding feature slot and generates structured sub-questions for retrieving general rules.

[0073] In the conclusion phase, one inference engine generates a deterministic conclusion, while the other generates a conditional conclusion. The assertion comparison module identifies differences in conclusion polarity and missing elements, while the backtracking control module defines the target processing phase as the structured sub-problem generation phase and uses the missing mask as the output structural constraint. If no supplementary facts are obtained, the final assertion record retains the conditional expression, and upon reaching the iteration termination condition, it appends a list of missing elements and a low-confidence flag to prevent the model from automatically filling in missing facts as a deterministic conclusion, thereby eliminating the illusion caused by missing facts.

[0074] According to one embodiment of this application, the alignment comparison includes an analysis phase comparison and a conclusion phase comparison; when the analysis phase comparison does not meet the analysis consistency condition, the evidence record is updated according to the analysis difference record and the reasoning process is re-executed; when the conclusion phase comparison does not meet the conclusion consistency condition, the target processing phase is determined according to the conclusion difference record and re-executed from the target processing phase.

[0075] Specifically, this embodiment explains the phased verification implementation method: In one implementation, assertion comparison includes analysis phase comparison and conclusion phase comparison. In the analysis phase, two inference engines respectively generate analytical assertions to describe normative applicability, factual conditions, and evidentiary relationships. If the first discrepancy record does not meet the analytical consistency condition, the system corrects the evidence retrieval or analysis processing based on the first discrepancy record to obtain the target analysis result.

[0076] In the conclusion phase, the two inference engines generate conclusion assertions based on the target analysis results. The conclusion phase allows for setting higher weights for conclusion polarity, key validity conditions, and evidence relevance than for general analysis assertions. If the second difference record does not meet the conclusion consistency condition, the system locates the structured sub-problem generation, evidence retrieval, or inference processing stage according to the difference type, and re-executes from the corresponding stage.

[0077] Phased verification is not the only implementation of the scheme defined in claim 1. Other implementations may perform assertion comparison only in one processing stage, or perform assertion comparison separately in multiple stages such as fact extraction, norm retrieval, rule application, and conclusion generation, as long as the difference record can generate machine-executable correction control information and control the re-execution of the preceding processing stages to form a multi-stage closed-loop reasoning.

[0078] In one implementation, assertion comparisons are set in both the analysis phase and the conclusion phase: differences in the analysis phase are used to correct evidence retrieval and intermediate analysis, while differences in the conclusion phase are used to select a backtracking starting point in the structured sub-problem generation, evidence retrieval, or reasoning processing phases.

[0079] According to one embodiment of this application, the iteration termination condition includes at least one of reaching a preset maximum number of iterations, the difference improvement amount of consecutive iterations being lower than a preset improvement threshold, or exceeding a preset computing resource limit; when the legal response result is output due to the fulfillment of the iteration termination condition, at least one of the following is appended to the legal response result: a low confidence indicator, an unresolved difference record, or a missing element list.

[0080] Specifically, this embodiment explains alternative implementation methods and security measures: the dimensions, names, and levels of structured sub-problems can be configured according to the type of legal business; the number and type of evidence sets can be changed; assertion alignment can adopt rule matching, vector matching, knowledge graph matching, natural language inference models, or combinations thereof; the first inference engine and the second inference engine can be deployed locally, in a private cloud, in a public cloud, or on an edge node.

[0081] Consistency criteria are not limited to a single numerical threshold. The system can use a risk level matrix, a classification model, or multiple rules to determine whether to continue iteration. The maximum number of iterations, computational resource limits, and difference improvement thresholds can be configured based on task complexity, the amount of evidence, and available computational resources.

[0082] The system can perform de-identification, sensitive information detection, access control, and log auditing on user input, court documents, and internal organizational data. Data from different organizations can be isolated through tenant permissions. The above security measures do not change the technical relationship between structured sub-problem generation, evidence retrieval, dual-channel assertion comparison, and difference-driven backtracking.

[0083] According to one embodiment of this application, after step 7, step 9 is further included: recording the association between structured sub-questions, retrieval requests, evidence records, assertion records, difference records, correction control information, and re-execution paths, and generating audit records based on the association; each conclusion item in the legal response result is associated with the source location information, version identifier, and effective time range of the corresponding evidence record through evidence identifier.

[0084] Specifically, each conclusion item in the legal response is linked to the source location information, version identifier, and effective time range of the corresponding evidence record through evidence identification. This allows each conclusion item to be traced back to the specific source of the evidence, the effective version of the evidence, and the effective time range, thereby verifying whether the legal norms on which the conclusion is based are currently valid and applicable to a specific region or situation, avoiding illusions caused by citing outdated norms, incorrect versions, or evidence with incompatible scope of application. The system records the relationships between structured sub-questions, search requests, evidence records, assertion records, difference records, correction control information, and re-execution paths, and generates audit logs based on these relationships. These audit logs fully preserve the entire processing chain from the original legal question to the final response, including which sub-questions were re-executed in each iteration, which evidence was updated, how differences were located and corrected, and the selection process for backtracking routes. Audit logs provide a machine-readable audit trail for the output of legal AI systems, enabling system operators, regulators, or end-users to conduct post-process review of the generation process. This allows them to verify whether the system operates according to a pre-defined multi-stage closed-loop reasoning mechanism, quickly pinpointing specific points that produce inconsistencies or illusions (such as evidence filtering errors in the retrieval stage or assertion generation errors in the reasoning stage), and meeting the compliance and explainability requirements of legal technology applications. By retaining structured data and their relationships at each processing stage, when the final response is challenged, a complete chain of evidence can be provided to prove the formation process of the conclusion, clarifying the decision-making basis of each module in the processing, reducing the loss of trust caused by "black box" reasoning, and providing a basis for accountability in automated decision-making in legal scenarios.

[0085] Figure 2 This is a schematic diagram of a legal large-scale illusion elimination system provided as an embodiment of the present invention. Figure 2 As shown, based on the above embodiments, this invention also proposes a legal big model illusion elimination system, comprising: a structured processing module, used to acquire legal question text, extract legal elements from the legal question text, and generate a structured sub-question set with sub-question identifiers; an evidence retrieval module, used to retrieve a legal knowledge base based on the structured sub-question set to obtain evidence records associated with the structured sub-question set and having evidence identifiers; a dual-channel reasoning module, including a first reasoning engine and a second reasoning engine configured independently, used to take the structured sub-question set and the evidence records as common inputs to generate a first processing result and a second processing result respectively; an assertion comparison module, used to convert the first processing result and the second processing result into assertion records including sub-question identifiers, processing stage identifiers, assertion content, and a set of evidence identifiers respectively, and to perform alignment comparison on assertion records with corresponding sub-question identifiers to generate difference records; and a correction control module, used to generate corrections based on the difference type, sub-question identifier, and processing stage identifier in the difference records when the difference records do not meet the consistency conditions. The system comprises: control information, wherein the correction control information is used to determine the target sub-problem to be re-executed, the target processing stage, and processing parameters, wherein the target processing stage is selected from at least one of the structured sub-problem generation stage, the evidence retrieval stage, and the reasoning processing stage; a re-execution module, used to re-execute the target sub-problem starting from the target processing stage according to the correction control information, to obtain an updated assertion record; a backtracking control module, used to repeatedly control the assertion comparison module to perform alignment comparison, control the correction control module to judge consistency conditions, and control the re-execution module to re-execute until the consistency conditions or iteration termination conditions are met; an output module, used to output a legal response result generated based on the final assertion record and associated with the evidence identifier; and an audit module, used to record the association between the structured sub-problem, retrieval request, evidence record, assertion record, difference record, correction control information, and re-execution path, and generate an audit record based on the association; and each conclusion item in the legal response result is associated with the source location information, version identifier, and effective time range of the corresponding evidence record through the evidence identifier.

[0086] Based on the above embodiments, this invention also proposes an electronic device, including a processor and a memory, wherein the memory stores a computer program, and when the computer program is executed by the processor, the electronic device performs the method described in any one of the first aspects of this invention.

[0087] Based on the above embodiments, this invention also proposes a computer-readable storage medium storing a computer program thereon, which, when executed by a processor, implements the method described in any one of the first aspects of this invention.

[0088] Based on the above embodiments, this invention also proposes a computer program product, including a computer program that, when executed by a processor, implements the method described in any one of the first aspects of this invention.

[0089] Specifically, the legal big model illusion elimination system based on multi-stage closed-loop reasoning includes a structured processing module, an evidence retrieval module, an evidence storage module, a dual-channel reasoning module, an assertion comparison module, a backtracking control module, and an output and auditing module. The dual-channel reasoning module includes a first reasoning channel and a second reasoning channel.

[0090] The structured processing module performs legal element extraction, structured sub-problem generation, and missing mask generation. The evidence retrieval module sends retrieval requests to one or more sets of legal knowledge and forms evidence records. The evidence storage module saves evidence records, iterative versions, and their mappings to structured sub-problems. The assertion comparison module performs assertion transformation, assertion alignment, difference calculation, and consistency judgment. The backtracking control module generates correction control information based on the difference records and controls relevant modules to selectively re-execute, achieving multi-stage closed-loop reasoning. The output and auditing module generates evidence-related responses and audit records.

[0091] Each module can be implemented by a software component in the same computing device, or it can be deployed separately on the client, application server, model server, retrieval server and data storage server, and exchange structured data with sub-problem identifiers, evidence identifiers, assertion identifiers and iteration version identifiers through a network interface.

[0092] The electronic device includes at least one processor, memory, network interface, and input / output interface. When a computer program stored in the memory is executed by the processor, it implements the method steps described above. The computer program may be stored on a non-volatile computer-readable storage medium or provided as a computer program product via a network or storage medium.

[0093] The above embodiments are preferred implementations of the present invention. In addition, the present invention can be implemented in other ways. Any obvious substitutions without departing from the concept of the present invention are within the protection scope of the present invention.

Claims

1. A method for eliminating illusions in a legal grand model, characterized in that, Includes the following steps: Step S1: Obtain the legal issue text, extract legal elements from the legal issue text, and generate a structured set of sub-issues with sub-issue identifiers; Step S2: Search the legal knowledge base based on the structured sub-problem set to obtain evidence records that are associated with the structured sub-problem set and have evidence identification; Step S3: The structured sub-problem set and the evidence record are used as common inputs and provided to the first inference engine and the second inference engine, which are configured independently. The first inference engine generates a first processing result and the second inference engine generates a second processing result. Step S4: Convert the first processing result and the second processing result into assertion records respectively. The assertion record includes a sub-problem identifier, a processing stage identifier, assertion content, and a set of evidence identifiers. Align and compare the assertion records with corresponding sub-problem identifiers to generate difference records. Step S5: When the difference record does not meet the consistency condition, generate correction control information based on the difference type, sub-problem identifier and processing stage identifier in the difference record. The correction control information is used to determine the target sub-problem to be re-executed, the target processing stage and processing parameters. The target processing stage is selected from at least one of the structured sub-problem generation stage, evidence retrieval stage and reasoning processing stage. Step S6: Based on the correction control information, re-execute the target sub-problem from the target processing stage to obtain the updated assertion record; Step S7: Repeat steps S4 to S6 to perform alignment comparison, determine consistency conditions, and re-execute until the consistency conditions or iteration termination conditions are met. Step S8: Output the legal response results generated based on the final assertion record and associated with the evidence identifier.

2. The method according to claim 1, characterized in that: Extracting legal elements from the legal issue text includes: extracting entities, attributes, behaviors, conditions, and relationships between entities; writing the extraction results into element slots of a preset legal element structure; generating structured sub-problems based on the filled element slots; and generating a missing mask for the unfilled element slots. The missing mask is used to restrict the output of deterministic assertions that depend on the corresponding missing elements.

3. The method according to claim 1, characterized in that: The legal knowledge base includes at least two of the following: a set of legal norms, a set of judgments, a set of normative documents, and a set of logical rules, each with its own retrieval index. The evidence record includes at least three of the following: evidence identifier, source type, source location information, standardized text, version identifier, effective time range, and scope of application label. Retrieving the legal knowledge base includes performing deduplication, time validity filtering, and scope of application filtering on candidate evidence, and establishing a mapping between structured sub-questions and evidence records according to sub-question identifiers.

4. The method according to claim 1, characterized in that: The first inference engine and the second inference engine differ in at least one of the following: model structure, model parameters, parameter adaptation layer, prompt template, or context configuration; before generating the first processing result and the second processing result, neither inference engine receives the processing result of the other inference engine.

5. The method according to claim 1, characterized in that: The assertion content includes subject items, relation items, object items or judgment values, establishment conditions, and polarity; the alignment comparison includes aligning the assertion records according to sub-problem identifiers, legal element identifiers, and semantic representations, and comparing the differences between the mutually aligned assertion records in at least one of the judgment values, establishment conditions, polarity, and evidence identifier sets.

6. The method according to claim 1, characterized in that: The difference types include at least one of the following: difference of missing elements, difference of missing evidence, difference of conflicting evidence, difference of time validity, difference of condition range, difference of logical relationship, and difference of conclusion polarity; the difference record also includes evidence identifiers corresponding to the difference content.

7. The method according to claim 6, characterized in that: Generating the correction control information includes: when the difference type is a missing element difference, determining the target processing stage as the structured sub-problem generation stage, and updating the element slots, missing masks, or structured sub-problems; when the difference type is a difference in missing evidence, a difference in conflicting evidence, or a difference in time validity, determining the target processing stage as the evidence retrieval stage, and generating retrieval correction parameters; when the difference type is a difference in condition range, a difference in logical relationship, or a difference in conclusion polarity, determining the target processing stage as the reasoning processing stage, and generating reasoning processing parameters.

8. The method according to claim 7, characterized in that: The retrieval correction parameters include at least one of the following: query term expansion parameters, source type parameters, applicable geographical parameters, time range parameters, normative level parameters, result quantity parameters, and ranking weight parameters; the inference processing parameters include at least one of the following: assertion identifier to be reprocessed, validity condition, evidence identifier set, context configuration, and output structure constraints.

9. The method according to claim 1, characterized in that: The alignment comparison includes analysis phase comparison and conclusion phase comparison; when the analysis phase comparison does not meet the analysis consistency condition, the evidence record is updated according to the analysis difference record and the reasoning process is re-executed; when the conclusion phase comparison does not meet the conclusion consistency condition, the target processing phase is determined according to the conclusion difference record and re-executed from the target processing phase.

10. The method according to claim 1, characterized in that: When re-executed based on the corrected control information, only the target sub-problem and its associated evidence records are reprocessed, while the processing results of other structured sub-problems unaffected by the difference records are retained; Each re-execution uses an evidence record with an iteration version identifier, and the first inference engine and the second inference engine use the same iteration version of the evidence record in the same iteration.

11. The method according to claim 1, characterized in that: The iteration termination condition includes at least one of the following: reaching a preset maximum number of iterations, the difference improvement amount of consecutive iterations being lower than a preset improvement threshold, or exceeding a preset computing resource limit; when the legal response result is output due to the fulfillment of the iteration termination condition, at least one of the following is added to the legal response result: a low confidence indicator, an unresolved difference record, or a missing element list.

12. The method according to claim 1, characterized in that: Following step 7, step 9 is also included: recording the relationships between structured sub-questions, retrieval requests, evidence records, assertion records, difference records, correction control information, and re-execution paths, and generating audit records based on the relationships; each conclusion item in the legal response result is associated with the source location information, version identifier, and effective time range of the corresponding evidence record through evidence identifiers.

13. A legal large-scale model illusion elimination system, characterized in that, include: The structured processing module is used to acquire legal issue text, extract legal elements from the legal issue text, and generate a set of structured sub-issues with sub-issue identifiers; The evidence retrieval module is used to search the legal knowledge base based on the structured sub-question set to obtain evidence records that are associated with the structured sub-question set and have evidence identifiers; The dual-channel reasoning module includes a first reasoning engine and a second reasoning engine that are configured independently, and is used to take the structured sub-problem set and the evidence record as common inputs to generate a first processing result and a second processing result respectively. The assertion comparison module is used to convert the first processing result and the second processing result into assertion records that include sub-problem identifiers, processing stage identifiers, assertion content and evidence identifier sets, respectively, and to align and compare the assertion records with corresponding sub-problem identifiers to generate difference records; The correction control module is used to generate correction control information based on the difference type, sub-problem identifier, and processing stage identifier in the difference record when the difference record does not meet the consistency condition. The correction control information is used to determine the target sub-problem, target processing stage, and processing parameters to be re-executed. The target processing stage is selected from at least one of the structured sub-problem generation stage, evidence retrieval stage, and reasoning processing stage. The re-execution module is used to re-execute the target sub-problem from the target processing stage according to the correction control information, and obtain the updated assertion record; The backtracking control module is used to repeatedly control the assertion comparison module to perform alignment comparison, control the correction control module to judge the consistency condition, and control the re-execution module to re-execute until the consistency condition or the iteration termination condition is met. The output module is used to output the legal response results generated based on the final assertion record and associated with the evidence identifier; The audit module is used to record the relationships between structured sub-questions, retrieval requests, evidence records, assertion records, difference records, correction control information, and re-execution paths, and to generate audit records based on these relationships; each conclusion item in the legal response result is associated with the source location information, version identifier, and effective time range of the corresponding evidence record through evidence identifiers.

14. An electronic device comprising a processor and a memory, characterized in that: The memory stores a computer program, which, when executed by the processor, causes the electronic device to perform the method according to any one of claims 1 to 12.

15. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, it implements the method of any one of claims 1 to 12.

16. A computer program product, comprising a computer program, characterized in that: When the computer program is executed by a processor, it implements the method of any one of claims 1 to 12.

Images