A large model question and answer control method and device

Abstract

The application discloses a large model question and answer control method and device, and relates to artificial intelligence question and answer control and information processing technology; including: step 1: receiving user query q and thread identification thread id , case identification case id and token summary related information, initializing or restoring session state; step 2: based on user query q and session state state t , candidate answer r is generated by a large model, step 3: a set of evidence units {E j} is constructed, step 4: the candidate answer r is parsed, and the original assertion set A raw ={a i} is obtained, each assertion contains a verifiable field, evidence matching is performed on each assertion, and a binding relationship M(a i -{E j}) between the assertion and the evidence unit is established, step 5: coverage_score and conflict_score are calculated according to A gate , step 6: stability_score_t is calculated according to historical information, and gate_score is calculated by combining the risk level, and step 7: the candidate answer r is output only when the following conditions are met simultaneously.

Description

Technical Field

[0001] This invention discloses a large-scale question-answering control method and device, which relates to artificial intelligence question-answering control and information processing technology. Background Technology

[0002] Existing large-scale question answering systems typically include stages such as query access, context reading, knowledge retrieval, external tool invocation, answer generation, and result output. Some systems determine whether to output the answer directly by using a fixed confidence threshold, some systems improve recall quality through RAG (Recall and Query) methods, and some systems control the retrieval, invocation, and answering processes through workflow orchestration.

[0003] In multi-turn question-answering scenarios, systems typically need to save thread context, case context, or other session states and resume execution after user interruption. For question-answering systems involving business interface calls, it is also necessary to access databases, retrieval services, or third-party interfaces during the answering process to supplement the factual basis required for the answer. However, many existing systems use fixed thresholds to uniformly control different tasks, making it difficult to adapt to the differentiated requirements of high-risk and low-risk tasks. Although retrieval results or tool call results are used to generate answers, the lack of explicit binding relationships between the various assertions and evidence in the answers leads to insufficient traceability. There is a lack of calculable quantification mechanisms for conflicts between multi-source evidence, making it difficult for the system to determine situations where "evidence exists but is contradictory." When calling external tools, invalid parameters or context pollution can easily trigger erroneous outbound calls, erroneous queries, or invalid retries. During multi-turn session interruption recovery, existing solutions often only restore the cache without verifying thread identifiers, case identifiers, and token digests, which can easily lead to contextual crosstalk. Summary of the Invention

[0004] This invention addresses the problems of existing technologies by providing a large-scale question-and-answer control method and apparatus, applicable to enterprise knowledge base question-and-answer, intelligent customer service, work order question-and-answer, and other large-scale question-and-answer systems that can call retrieval services or external tools.

[0005] The specific solution proposed in this invention is as follows: This invention provides a large-scale question-answering control method, comprising: Step 1: Receive user query q and thread identifier thread id Case Identifier id With token digest information, initialize or restore session state. t ; Step 2: Based on user query q and session state t Candidate answers r are generated by a large model. These candidate answers are not directly output before passing through the gating system. Step 3: Collect at least one of the following: search results and results from external tool calls, to construct the evidence unit set {E} j Each unit of evidence shall include at least the following fields: evidence identifier, request fingerprint, response fingerprint, source identifier, timestamp, and assertion link field. Step 4: Perform assertion parsing on the candidate answer r to obtain the original assertion set A. raw ={a i Each assertion contains a verifiable field. Evidence matching is performed on each assertion to establish a binding relationship M(a) between the assertion and the evidence unit. i ->{E j}), Step 5: From A raw Select the set of assertions A that participate in gating. gate According to A gate Calculate the coverage score and the conflict score. Step 6: Calculate the stability score (stability_score_t) based on historical information, and combine this with the risk level to calculate the gate score (gate_score). Step 7: Output the candidate answer r if and only if the following conditions are met simultaneously: gate_score≥τ(risk_level); coverage_score≥C_min(risk_level); conflict_score≤K_max(risk_level); τ ( risk _ level ) is the gate threshold. C _ my ( risk _ level ) is the minimum coverage threshold. K _ max ( risk _ level ) represents the maximum permissible conflict threshold. risk _ level Risk level, If any condition is not met, the rollback process will be triggered.

[0006] Furthermore, in step 3 of the large model question-answering control method, joint deduplication of evidence units is performed; for the same deduplication key, only the most recent timestamp record is retained, and the hit count hit_count_j is maintained.

[0007] Furthermore, in step 4 of the large-scale model question-answering control method, assertion parsing is performed, including: named entity extraction, relation extraction, and temporal attribute extraction. For assertion a... iLet B be the set of evidence units that are bound to it. i , A valid binding is determined based on the following conditions: When B i When =∅, assert that the indicator I is hit. i =0; When B i When ≠∅, if the field difference d_field i ≤δ f Time difference d_time i ≤δ t And max (j∈Bi) source_conf ij ≥δ s If so, the binding is considered valid and I is allowed. i =1; otherwise let I =1; i =0.

[0008] in: d_field ij =mismatch_count ij / max(comparable_field_count ij ,1); d_time ij =min(1,|t_claim i -t_evidence j | / ΔT_max); source_conf ij =clip(ρ1*cred_level ij +ρ2*success_rate ij +ρ3*freshness ij ,0,1); mismatch_count ij Indicates assertion a i The number of fields that do not match evidence j, comparable_field_count ij δ represents the total number of fields that can be compared between the two. f Indicates the threshold for field differences; t_claim i Indicates assertion a i timestamp, t_evidence j Let be the timestamp of evidence j, ΔT_max be the maximum allowable time difference, and δ be the time stamp. t cred_level is the threshold for time variability. ij success_rate represents the credibility level of evidence j.ij As evidence j Historical success rate, freshness ij As evidence j The freshness of the time is higher the closer it is, and it usually decays over time. ρ1ρ2ρ3 are weighting coefficients, and ρ1+ρ2+ρ3=1.

[0009] Furthermore, step 5 of the large-model question-answering control method specifically includes: When the number of original assertions |A raw When the threshold is 30, the top 30 assertions are truncated in descending order of their assertion weights to form the set A of participating gating assertions. gate Otherwise, let A gate =A raw , When|A gate |=0 or Σw i When the value is ≤0, a rollback is triggered directly. Coverage is calculated using the following formula: coverage_score=(Σw i *I i ) / max(Σw i ,ε), The conflict score is calculated using the following formula: conflict_score=(Σw i *c i ) / max(Σw i ,ε), where: ω ij =source_conf ij / max(Σ(j∈B i )source_conf ij ,ε), d_field i =Σ (j∈Bi) ω ij *d_field ij d_time i =Σ (j∈Bi) ω ij *d_time ij d_source i =Σ (j∈Bi) ω ij *(1-source_conf ij ), c i =min(1,α*d_field i +β*d_time i +γ*d_source i ), When B i When =∅, we can let c i =1 and Hi =1; when B i When ≠∅, let H i =0, the fallback priority is defined as: priority i =c i +μ*H i , For evidence within the assertion, sort it according to the hit count correction term: Δpriority j =ν*log(1+hit_count_j), ε is a minimal constant to prevent division by zero, w i To assert the weight of i, I i To assert the hit indicator of i, c i For the conflict value of assertion i, the value is [0,1], source_conf ij To assert the confidence level of the source of evidence j in i, ω ij d_field is the normalized weight of evidence j within assertion i. ij d_time represents the distance between evidence j and assertion i on the field. ij d_field represents the temporal distance between evidence j and assertion i. i This represents the weighted mean of the field differences for assertion i, d_time i d_source represents the time-difference weighted mean of assertion i. i This represents the weighted mean of the uncertainty in the source of assertion i. α, β, and γ are the weighting coefficients for field differences, time differences, and source uncertainty, respectively; μ is the weighting coefficient for the rollback indication; and priority is... i The backoff priority of assertion i, ν is the weight coefficient of the hit count correction term, Δpriority j This is the priority adjustment amount for evidence j. The more hits, the higher the priority. The logarithmic form prevents monopolization by head evidence.

[0010] Furthermore, step 6 of the large-scale question-answering control method utilizes a formula based on historical information: stability_score t =λ*stability_score (t-1) +(1-λ)*(1-conflict_score t ), Calculate stability_score t λ is the attenuation coefficient. λ ∈[0,1], Using the formula: gate_score=θ1*coverage_score-θ2*conflict_score+θ3*stability_score-θ4*risk_norm, The gate score is calculated as follows: gate_score, risk_norm is the normalized value of risk level, θ1 is the positive weight of coverage (the larger the value, the more coverage is valued), θ2 is the negative weight of conflict (the larger the value, the more conflict is penalized), θ3 is the positive weight of stability (the larger the value, the more historical consistency is valued), and θ4 is the negative weight of risk (the larger the value, the more high risk is penalized).

[0011] Furthermore, the rollback process in step 7 of the large-scale question-answering control method includes at least: retrieval enhancement; multi-source verification; external tool invocation; and regenerating the answer. Before making an external call, parameter pattern validation is performed. The validation items include at least the parameter name, parameter type, and required attributes. If the validation fails, the outbound call is prohibited and a structured error is returned.

[0012] Furthermore, the large-model question-answering control method also includes step 8: in the scenario of resuming the session, performing at least three checks: Verify thread_id consistency; Verify the consistency of case_id; Verify the consistency of token_digest; If any verification fails, recovery is refused, and the context is required to be reconfirmed.

[0013] This invention also provides a large-scale question-answering control device, including a query access module, a generation module, an evidence construction module, an assertion parsing and binding module, a scoring calculation module, a dynamic gating module, and a rollback control module. The query access module receives user query q, as well as thread identifier thread_id, case identifier case_id, and token digest information, and initializes or restores session state state_t. The generation module is based on the user query q and the session state state. t Candidate answers r are generated by a large model. These candidate answers are not directly output before passing through the gating system. The evidence construction module collects at least one of the following: retrieval results and external tool call results, to construct the evidence unit set {E}. j Each unit of evidence shall include at least the following fields: evidence identifier, request fingerprint, response fingerprint, source identifier, timestamp, and assertion link field. The assertion parsing and binding module performs assertion parsing on the candidate answer r to obtain the original assertion set A. raw ={a iEach assertion contains a verifiable field. Evidence matching is performed on each assertion to establish a binding relationship M(a) between the assertion and the evidence unit. i ->{E j}), The scoring calculation module starts from A raw Select the set of assertions A that participate in gating. gate According to A gate Calculate the coverage score and the conflict score. The dynamic gating module calculates the stability score (stability_score_t) based on historical information and combines it with the risk level to calculate the gate score (gate_score). The backoff control module outputs a candidate answer r if and only if the following conditions are met simultaneously: gate_score ≥ τ(risk_level); coverage_score ≥ C_min(risk_level); conflict_score ≤ K_max(risk_level); τ ( risk _ level ) is the gate threshold. C _ my ( risk _ level ) is the minimum coverage threshold. K _ max ( risk _ level ) represents the maximum permissible conflict threshold. risk _ level Risk level, If any condition is not met, the rollback process will be triggered.

[0014] Furthermore, the large-scale question-answering control device also includes a consistency verification module, which performs at least three verifications in the scenario of resuming a session: Verify thread_id consistency; Verify the consistency of case_id; Verify the consistency of token_digest; If any verification fails, recovery is refused, and the context is required to be reconfirmed.

[0015] The advantages of this invention are: 1. By binding assertion-level evidence, a clear correspondence is established between the response and the evidence, thereby improving traceability.

[0016] 2. By linking coverage and conflict scores, it is possible to distinguish between two types of risks: “insufficient evidence” and “contradictory evidence”, and to adopt different rollback strategies.

[0017] 3. By introducing historical stability and risk levels, different release thresholds can be set for different tasks, improving the system's usability.

[0018] 4. By using external call parameter pattern validation and budget constraints, erroneous outbound calls, invalid retries, and resource waste can be reduced.

[0019] 5. By interrupting and resuming consistency checks, the risks of thread crosstalk, case crosstalk, and continued incorrect answers after recovery can be reduced.

[0020] 6. This solution is suitable for both single-machine deployment and multi-service collaborative deployment, and has good engineering feasibility. Attached Figure Description

[0021] Figure 1 This is a schematic diagram of the application process of the method of the present invention. Detailed Implementation

[0022] The present invention will be further described below with reference to the accompanying drawings and specific embodiments, so that those skilled in the art can better understand and implement the present invention. However, the embodiments described are not intended to limit the present invention. Example

[0023] This invention provides a large-scale question-answering control method, comprising: Step 1: Receive user query q and thread identifier thread id Case Identifier id With token digest information, initialize or restore session state. t .

[0024] Step 2: Based on user query q and session state t Candidate answers r are generated by a large model, and the candidate answers are not directly output before passing through the gating.

[0025] Step 3: Collect at least one of the following: search results and results from external tool calls, to construct the evidence unit set {E} j Each evidence unit includes at least: an evidence identifier, a request fingerprint, a response fingerprint, a source identifier, a timestamp, and an assertion link field. Perform joint deduplication on evidence units; for the same deduplication key, only retain the most recent timestamp record and maintain the hit count hit_count_j.

[0026] Step 4: Perform assertion parsing on the candidate answer r to obtain the original assertion set A.raw ={a i Each assertion contains a verifiable field. Evidence matching is performed on each assertion to establish a binding relationship M(a) between the assertion and the evidence unit. i ->{E j}).

[0027] This involves assertion parsing, including named entity extraction, relation extraction, and temporal attribute extraction. For assertion a... i Let B be the set of evidence units that are bound to it. i , A valid binding is determined based on the following conditions: When B i When =∅, assert that the indicator I is hit. i =0; When B i When ≠∅, if the field difference d_field i ≤δ f Time difference d_time i ≤δ t And max (j∈Bi) source_conf ij ≥δ s If so, the binding is considered valid and I is allowed. i =1; otherwise let I =1; i =0.

[0028] in: d_field ij =mismatch_count ij / max(comparable_field_count ij ,1); d_time ij =min(1,|t_claim i -t_evidence j | / ΔT_max); source_conf ij =clip(ρ1*cred_level ij +ρ2*success_rate ij +ρ3*freshness ij ,0,1); mismatch_count ij Indicates assertion a i The number of fields that do not match evidence j, comparable_field_count ij δ represents the total number of fields that can be compared between the two.f Indicates the threshold for field differences; t_claim i Indicates assertion a i timestamp, t_evidence j Let be the timestamp of evidence j, ΔT_max be the maximum allowable time difference, and δ be the time stamp. t cred_level is the threshold for time variability. ij success_rate represents the credibility level of evidence j. ij As evidence j Historical success rate, freshness ij As evidence j The freshness of the time is higher the closer it is, and it usually decays over time. ρ1ρ2ρ3 are weighting coefficients, and ρ1+ρ2+ρ3=1.

[0029] Step 5: From A raw Select the set of assertions A that participate in gating. gate According to A gate Calculate the coverage score and the conflict score.

[0030] Specifically, this includes: When the number of original assertions |A raw When the threshold is 30, the top 30 assertions are truncated in descending order of their assertion weights to form the set A of participating gating assertions. gate Otherwise, let A gate =A raw , When|A gate |=0 or Σw i When the value is ≤0, a rollback is triggered directly. Coverage is calculated using the following formula: coverage_score=(Σw i *I i ) / max(Σw i ,ε), The conflict score is calculated using the following formula: conflict_score=(Σw i *c i ) / max(Σw i ,ε), where: ω ij =source_conf ij / max(Σ(j∈B i )source_conf ij ,ε), d_field i =Σ (j∈Bi) ω ij *d_field ijd_time i =Σ (j∈Bi) ω ij *d_time ij d_source i =Σ (j∈Bi) ω ij *(1-source_conf ij ), c i =min(1,α*d_field i +β*d_time i +γ*d_source i ), When B i When =∅, we can let c i =1 and H i =1; when B i When ≠∅, let H i =0, the fallback priority is defined as: priority i =c i +μ*H i , For evidence within the assertion, sort it according to the hit count correction term: Δpriority j =ν*log(1+hit_count_j), ε is a minimal constant to prevent division by zero, w i To assert the weight of i, I i To assert the hit indicator of i, c i For the conflict value of assertion i, the value is [0,1], source_conf ij To assert the confidence level of the source of evidence j in i, ω ij d_field is the normalized weight of evidence j within assertion i. ij d_time represents the distance between evidence j and assertion i on the field. ij d_field represents the temporal distance between evidence j and assertion i. i This represents the weighted mean of the field differences for assertion i, d_time i d_source represents the time-difference weighted mean of assertion i. i This represents the weighted mean of the uncertainty in the source of assertion i. α, β, and γ are the weighting coefficients for field differences, time differences, and source uncertainty, respectively; μ is the weighting coefficient for the rollback indication; and priority is... i The backoff priority of assertion i, ν is the weight coefficient of the hit count correction term, Δpriority jThis is the priority adjustment amount for evidence j. The more hits, the higher the priority. The logarithmic form prevents monopolization by head evidence.

[0031] Step 6: Calculate the stability score (stability_score_t) based on historical information, and calculate the gate score (gate_score) based on the risk level.

[0032] The formula is based on historical information: stability_score t =λ*stability_score (t-1) +(1-λ)*(1-conflict_score t ), Calculate stability_score t λ is the attenuation coefficient. λ ∈[0,1], Using the formula: gate_score=θ1*coverage_score-θ2*conflict_score+θ3*stability_score-θ4*risk_norm, The gate score is calculated as follows: gate_score, risk_norm is the normalized value of risk level, θ1 is the positive weight of coverage (the larger the value, the more coverage is valued), θ2 is the negative weight of conflict (the larger the value, the more conflict is penalized), θ3 is the positive weight of stability (the larger the value, the more historical consistency is valued), and θ4 is the negative weight of risk (the larger the value, the more high risk is penalized).

[0033] Step 7: Output the candidate answer r if and only if the following conditions are met simultaneously: gate_score≥τ(risk_level); coverage_score≥C_min(risk_level); conflict_score≤K_max(risk_level); τ ( risk _ level ) is the gate threshold. C _ my ( risk _ level ) is the minimum coverage threshold. K _ max ( risk _ level ) represents the maximum permissible conflict threshold. risk _ level Risk level.

[0034] Example

[0035] This invention also provides a large-scale question-answering control device, including a query access module, a generation module, an evidence construction module, an assertion parsing and binding module, a scoring calculation module, a dynamic gating module, and a rollback control module. The query access module receives user query q, as well as thread identifier thread_id, case identifier case_id, and token digest information, and initializes or restores session state state_t. The generation module is based on the user query q and the session state state. t Candidate answers r are generated by a large model. These candidate answers are not directly output before passing through the gating system. The evidence construction module collects at least one of the following: retrieval results and external tool call results, to construct the evidence unit set {E}. j Each unit of evidence shall include at least the following fields: evidence identifier, request fingerprint, response fingerprint, source identifier, timestamp, and assertion link field. The assertion parsing and binding module performs assertion parsing on the candidate answer r to obtain the original assertion set A. raw ={a i Each assertion contains a verifiable field. Evidence matching is performed on each assertion to establish a binding relationship M(a) between the assertion and the evidence unit. i ->{E j}), The scoring calculation module starts from A raw Select the set of assertions A that participate in gating. gate According to A gate Calculate the coverage score and the conflict score. The dynamic gating module calculates the stability score (stability_score_t) based on historical information and combines it with the risk level to calculate the gate score (gate_score). The backoff control module outputs a candidate answer r if and only if the following conditions are met simultaneously: gate_score ≥ τ(risk_level); coverage_score ≥ C_min(risk_level); conflict_score ≤ K_max(risk_level); τ ( risk _ level ) is the gate threshold. C _ my ( risk _ level ) is the minimum coverage threshold. K _ max ( risk _ level ) represents the maximum permissible conflict threshold. risk _ level Risk level, If any condition is not met, the rollback process will be triggered.

[0036] The information interaction and execution of readable programs between the modules in the above-mentioned device are based on the same concept as the method embodiments of the present invention, and the specific details can be found in the descriptions in the method embodiments of the present invention, and will not be repeated here.

[0037] Similarly, the advantages of the device of the present invention are: 1. By binding assertion-level evidence, a clear correspondence is established between the response and the evidence, thereby improving traceability.

[0038] 2. By linking coverage and conflict scores, it is possible to distinguish between two types of risks: “insufficient evidence” and “contradictory evidence”, and to adopt different rollback strategies.

[0039] 3. By introducing historical stability and risk levels, different release thresholds can be set for different tasks, improving the system's usability.

[0040] 4. By using external call parameter pattern validation and budget constraints, erroneous outbound calls, invalid retries, and resource waste can be reduced.

[0041] 5. By interrupting and resuming consistency checks, the risks of thread crosstalk, case crosstalk, and continued incorrect answers after recovery can be reduced.

[0042] 6. This solution is suitable for both single-machine deployment and multi-service collaborative deployment, and has good engineering feasibility.

[0043] It should be noted that not all steps and modules in the above processes and device structures are mandatory; some steps or modules may be omitted as needed. The execution order of the steps is not fixed and can be adjusted as required. The device structures described in the above embodiments can be physical or logical structures. That is, some modules may be implemented by the same physical entity, or some modules may be implemented by multiple physical entities, or they may be jointly implemented by certain components in multiple independent devices.

[0044] The above-described embodiments are merely preferred embodiments provided to fully illustrate the present invention, and the scope of protection of the present invention is not limited thereto. Equivalent substitutions or modifications made by those skilled in the art based on the present invention are all within the scope of protection of the present invention. The scope of protection of the present invention is defined by the claims.

Claims

1. A large-scale question-answering control method, characterized by: include: Step 1: Receive user query q and thread identifier thread id Case Identifier id With token digest information, initialize or restore session state. t ; Step 2: Based on user query q and session state t Candidate answers r are generated by a large model. These candidate answers are not directly output before passing through the gating system. Step 3: Collect at least one of the following: search results and results from external tool calls, to construct the evidence unit set {E} j Each unit of evidence shall include at least the following fields: evidence identifier, request fingerprint, response fingerprint, source identifier, timestamp, and assertion link field. Step 4: Perform assertion parsing on the candidate answer r to obtain the original assertion set A. raw ={a i Each assertion contains a verifiable field. Evidence matching is performed on each assertion to establish a binding relationship M(a) between the assertion and the evidence unit. i ->{E j }), Step 5: From A raw Select the set of assertions A that participate in gating. gate According to A gate Calculate the coverage score and the conflict score. Step 6: Calculate the stability score (stability_score_t) based on historical information, and combine this with the risk level to calculate the gate score (gate_score). Step 7: Output the candidate answer r if and only if the following conditions are met simultaneously: gate_score≥τ(risk_level); coverage_score≥C_min(risk_level); conflict_score≤K_max(risk_level); τ ( risk _ level ) is the gate threshold. C _ min ( risk _ level ) is the minimum coverage threshold. K _ max ( risk _ level ) represents the maximum permissible conflict threshold. risk _ level Risk level, If any condition is not met, the rollback process will be triggered.

2. The large model question answering control method according to claim 1, characterized in that in step 3, joint deduplication of evidence units is performed; for the same deduplication key, only the most recent timestamp record is retained, and the hit count hit_count_j is maintained.

3. The large-scale question-answering control method according to claim 1, characterized in that: Step 4 involves assertion parsing, including named entity extraction, relation extraction, and time attribute extraction. For assertion a... i Let B be the set of evidence units that are bound to it. i , A valid binding is determined based on the following conditions: When B i When =∅, assert that the indicator I is hit. i =0; When B i When ≠∅, if the field difference d_field i ≤δ f Time difference d_time i ≤δ t And max (j∈Bi) source_conf ij ≥δ s If so, the binding is considered valid and I is allowed. i =1; otherwise let I =1; i =0; in: d_field ij =mismatch_count ij / max(comparable_field_count ij ,1); d_time ij =min(1,|t_claim i -t_evidence j | / ΔT_max); source_conf ij =clip(ρ1*cred_level ij +ρ2*success_rate ij +ρ3*freshness ij ,0,1); mismatch_count ij Indicates assertion a i The number of fields that do not match evidence j, comparable_field_count ij δ represents the total number of fields that can be compared between the two. f Indicates the threshold for field differences; t_claim i Indicates assertion a i timestamp, t_evidence j Let be the timestamp of evidence j, ΔT_max be the maximum allowable time difference, and δ be the time stamp. t cred_level is the threshold for time variability. ij success_rate represents the credibility level of evidence j. ij As evidence j Historical success rate, freshness ij As evidence j The freshness of the time is higher the closer it is, and it usually decays over time. ρ1ρ2ρ3 are weighting coefficients, and ρ1+ρ2+ρ3=1.

4. The large-scale question-answering control method according to claim 1, characterized in that: Step 5 specifically includes: When the number of original assertions |A raw When the threshold is 30, the top 30 assertions are truncated in descending order of their assertion weights to form the set A of participating gating assertions. gate Otherwise, let A gate =A raw , When|A gate |=0 or Σw i When the value is ≤0, a rollback is triggered directly. Coverage is calculated using the following formula: coverage_score=(Σw i *I i ) / max(Σw i ,ε), The conflict score is calculated by the following formula: conflict_score=(Σw i *c i ) / max(Σw i ,ε), where: ω ij =source_conf ij / max(Σ(j∈B i )source_conf ij ,ε), d_field i =Σ (j∈Bi) ω ij *d_field ij , d_time i =Σ (j∈Bi) ω ij *d_time ij , d_source i =Σ (j∈Bi) ω ij *(1-source_conf ij ), c i =min(1,α*d_field i +β*d_time i +γ*d_source i ), When B i When =∅, we can let c i =1 and H i =1; when B i When ≠∅, let H i =0, the fallback priority is defined as: priority i =c i +μ*H i , For evidence within the assertion, sort it according to the hit count correction term: Δpriority j =ν*log(1+hit_count_j), ε is a minimal constant to prevent division by zero, w i To assert the weight of i, I i To assert the hit indicator of i, c i For the conflict value of assertion i, the value is [0,1], source_conf ij To assert the confidence level of the source of evidence j in i, ω ij d_field is the normalized weight of evidence j within assertion i. ij d_time represents the distance between evidence j and assertion i on the field. ij d_field represents the temporal distance between evidence j and assertion i. i This represents the weighted mean of the field differences for assertion i, d_time i d_source represents the time-difference weighted mean of assertion i. i This represents the weighted mean of the uncertainty in the source of assertion i. α, β, and γ are the weighting coefficients for field differences, time differences, and source uncertainty, respectively; μ is the weighting coefficient for the rollback indication; and priority is... i The backoff priority of assertion i, ν is the weight coefficient of the hit count correction term, Δpriority j This is the priority adjustment amount for evidence j. The more hits, the higher the priority. The logarithmic form prevents monopolization by head evidence.

5. The large-scale question-answering control method according to claim 1, characterized in that: Step 6: Use the formula based on historical information: stability_score t =λ*stability_score (t-1) +(1-λ)*(1-conflict_score t ), Calculate stability_score t λ is the attenuation coefficient. λ ∈[0,1], Using the formula: gate_score=θ1*coverage_score-θ2*conflict_score+θ3*stability_score-θ4*risk_norm, The gate score is calculated as follows: gate_score, risk_norm is the normalized value of risk level, θ1 is the positive weight of coverage (the larger the value, the more coverage is valued), θ2 is the negative weight of conflict (the larger the value, the more conflict is penalized), θ3 is the positive weight of stability (the larger the value, the more historical consistency is valued), and θ4 is the negative weight of risk (the larger the value, the more high risk is penalized).

6. The large-scale question-answering control method according to claim 1, characterized in that: The rollback process in step 7 includes at least the following: search enhancement; multi-source verification; external tool invocation; and regenerating the answer. Before making an external call, parameter pattern validation is performed. The validation items include at least the parameter name, parameter type, and required attributes. If the validation fails, the outbound call is prohibited and a structured error is returned.

7. The large-scale question-answering control method according to claim 1, characterized in that: This also includes step 8: In the scenario of resuming a session, perform at least three checks: Verify thread_id consistency; Verify the consistency of case_id; Verify the consistency of token_digest; If any verification fails, recovery is refused, and the context is required to be reconfirmed.

8. A large-scale question-and-answer control device, characterized in that: It includes a query access module, a generation module, an evidence construction module, an assertion parsing and binding module, a scoring calculation module, a dynamic gating module, and a rollback control module. The query access module receives user query q, as well as thread identifier thread_id, case identifier case_id, and token digest information, and initializes or restores session state state_t. The generation module is based on the user query q and the session state state. t Candidate answers r are generated by a large model. These candidate answers are not directly output before passing through the gating system. The evidence construction module collects at least one of the following: retrieval results and external tool call results, to construct the evidence unit set {E}. j Each unit of evidence shall include at least the following fields: evidence identifier, request fingerprint, response fingerprint, source identifier, timestamp, and assertion link field. The assertion parsing and binding module performs assertion parsing on the candidate answer r to obtain the original assertion set A. raw ={a i Each assertion contains a verifiable field. Evidence matching is performed on each assertion to establish a binding relationship M(a) between the assertion and the evidence unit. i ->{E j }), The scoring calculation module starts from A raw Select the set of assertions A that participate in gating. gate According to A gate Calculate the coverage score and the conflict score. The dynamic gating module calculates the stability score (stability_score_t) based on historical information and combines it with the risk level to calculate the gate score (gate_score). The backoff control module outputs a candidate answer r if and only if the following conditions are met simultaneously: gate_score ≥ τ(risk_level); coverage_score ≥ C_min(risk_level); conflict_score ≤ K_max(risk_level); τ ( risk _ level ) is the gate threshold. C _ min ( risk _ level ) is the minimum coverage threshold. K _ max ( risk _ level ) represents the maximum permissible conflict threshold. risk _ level Risk level, If any condition is not met, the rollback process will be triggered.

9. A large-scale question-and-answer control device according to claim 8, characterized in that: It also includes a consistency verification module, which performs at least three checks in the scenario of session recovery: Verify thread_id consistency; Verify the consistency of case_id; Verify the consistency of token_digest; If any verification fails, recovery is refused, and the context is required to be reconfirmed.

Images