Multi-agent cooperative output evidence traceability auditing method and system

Abstract

The present application provides a kind of multi-agent cooperation output evidence traceability audit method and system, and the process log is received by multi-agent cooperation output layer receiving multi-agent cooperation output;The output is divided into clause level, sentence segment level or field level audit unit by clause level segmentation layer, and the unique identification and hash signature are generated for audit unit;Evidence association chain building layer is associated with document segment, database record, tool result and model call information;Through the cooperation process trace layer, evidence traceability chain and cooperation process mapping are established;Through audit verification and output layer, it is judged whether there is evidence missing, expiration, chain break or multi-version inconsistency, if there is, the risk level is calculated and triggered to supplement the evidence, re-audit or manual review, if there is not, it is marked as audit passed;Finally, the audit view containing clause, evidence, agent, time, version and risk prompt is output.

Description

Technical Field

[0001] This invention relates to a source tracing audit method, and more particularly to a multi-agent collaborative output evidence source tracing audit method and system. Background Technology

[0002] With the development of large language models and multi-agent orchestration technologies, tasks such as complex report writing, document review, legal compliance analysis, patent analysis, financial risk control, and medical decision support are increasingly being completed collaboratively by multiple agents. Multiple agents can respectively undertake responsibilities such as retrieval, parsing, reasoning, computation, review, and report generation, thereby improving task coverage and processing efficiency.

[0003] However, multi-agent collaboration also brings new trust governance challenges. A particular sentence, conclusion, or field in the final output may have been generated by multiple agents and tools. If the system only saves the final text without saving the mappings between clauses and evidence, clauses and agents, clauses and tool calls, and clauses and prompt versions, it will be difficult to answer questions such as "Where did this conclusion come from?", "Which agent generated or modified it?", "Is the basis still valid?", and "Did it cite outdated evidence?" during subsequent reviews.

[0004] Therefore, a method for tracing and auditing evidence outputs from multi-agent collaborative processes is needed. This method breaks down the final output into auditable units and binds an evidence chain, process chain, and version chain to each unit. Existing solutions include in-depth document tracing, evidence verification and multi-agent collaborative review, document review based on multi-agent collaboration and enhanced retrieval, and carbon report review management.

[0005] (1) Deep Document Source Tracing Scheme. The publicly available document CN121071126A, "Deep Document Source Tracing System Based on Multi-Agent Collaboration," discloses a central control agent, an academic search agent, a document analysis agent, a correlation analysis agent, and a report generation agent. Document source tracing is achieved through reference search, correlation analysis, and report generation. This scheme focuses on citation relationships, similarity relationships, and source tracing report generation among documents. However, it does not disclose the segmentation of the final output of the multi-agent system into clause-level audit units, nor does it disclose the chain-like audit mechanism that binds model call parameters, prompts template versions, tool calculation results, participating agents, and revision history for each output clause. Source link: https: / / patents.google.com / patent / CN121071126A / zh

[0006] (2) Evidence Verification and Multi-Agent Collaborative Review Scheme. The publicly available document CN121504380A, "A Proposal Review Method and System Based on Evidence Verification and Multi-Agent Collaboration," discloses a cross-validation mechanism based on a review information pool, enabling multiple target agents to mutually verify, supplement, and correct each other to form a structured review report. This scheme emphasizes review accuracy, cross-validation, and convergence conditions, but it does not address the item-by-item binding between the final output clauses and various types of evidence, model call parameters, context pruning records, and version relationships. It also does not provide risk marking and supplementary verification procedures for cases of missing evidence, broken citations, or inconsistent conclusions across multiple versions. Source link: https: / / patents.google.com / patent / CN121504380A / zh

[0007] (3) A document review scheme based on multi-agent collaboration and retrieval enhancement. Published document CN121706760A, "A document review method based on multi-agent collaboration and retrieval enhancement," discloses a process for rule parsing, document segmentation, knowledge retrieval, matching and comparison, and review report generation, forming a review report containing a traceability link. This scheme mainly focuses on matching document review rules with document evidence, unlike this invention which targets arbitrary multi-agent collaborative output and audits the evidence, process, version, and responsibility nodes of the final output clauses. Source link: https: / / patents.google.com / patent / CN121706760A / zh

[0008] The existing technology has the following drawbacks:

[0009] (1) Insufficient granularity of audit objects. Existing solutions mostly use documents, reports, or review results as audit objects, lacking fine-grained audit unit division of the final output clauses, paragraphs, and fields.

[0010] (2) The evidence chain is incomplete. Existing tracing schemes focus on literature citations or review evidence, but do not simultaneously bind search fragments, database records, tool calculation results, model call parameters, prompt template versions, and participating agent identifiers.

[0011] (3) The collaboration process is disconnected from the final output. Task assignment, message passing, context trimming, tool calls and result revision among multiple agents usually only exist in the process log and are not mapped to the final terms.

[0012] (4) Insufficient risk identification. The existing system lacks automatic detection and risk classification for issues such as missing evidence, expired evidence, broken citations, inconsistent conclusions across multiple versions, and rolled-back nodes still being cited.

[0013] (5) Audit outputs are not subject to review. Simply outputting the final report text does not meet the review requirements of regulators, legal departments, or corporate internal controls regarding the source, responsibility, time, and version. Summary of the Invention

[0014] This invention provides a method and system for tracing and auditing evidence output by multi-agent collaborative systems. It addresses situations where, when multi-agent collaborative systems output reports, code descriptions, review conclusions, or Q&A results, the final content source is unclear, responsibility nodes are difficult to trace, evidence citation relationships are unstable, and process logs do not correspond to final clauses. These issues make it difficult to meet compliance audit, legal review, and internal control record-keeping requirements in high-risk business scenarios. The invention aims to solve the following problems:

[0015] (1) Multi-agent outputs are usually complete reports or long text results. Existing systems have difficulty accurately breaking down the final content into auditable clause-level, sentence-level, or field-level units.

[0016] (2) The output content may come from various sources such as search documents, database records, tool calculation results, model reasoning, prompt templates and manual revisions. Existing solutions lack an evidence association chain mechanism for each output unit.

[0017] (3) There are intermediate behaviors such as message passing, task assignment, context trimming, tool calling and result revision in the process of multi-agent collaboration. If a mapping is not established with the final output, it is difficult to track the responsible nodes and the generation path.

[0018] (4) There may be risks such as missing evidence, expired evidence, broken citations, inconsistent conclusions in multiple versions, or the rollback node still being cited in the output terms. The existing system lacks an automatic audit consistency verification mechanism.

[0019] (5) High-credibility scenarios require visual presentation of terms, source evidence, participating agents, generation time, version relationships and risk warnings. Existing report generation solutions usually only provide the final text, which is difficult to meet the requirements for audit archiving.

[0020] The technical solution is as follows:

[0021] A method for tracing and auditing the source of evidence output through multi-agent collaboration includes the following steps:

[0022] S1: Receive reports, code descriptions, review conclusions, or Q&A results generated by multi-agent collaboration, divide them into audit units at the clause level, sentence level, or field level, and generate unit_id, hash signature, and location information for each audit unit;

[0023] S2: For each audit unit, automatically associate the corresponding search document fragments, database records, tool calculation results, model call parameters, prompt template version, participating agent identifiers and manual revision records to generate an evidence tracing chain;

[0024] S3: Records message passing, task assignment, context trimming, tool invocation, result revision and version switching behaviors between agents, and establishes a mapping relationship with the final output audit unit;

[0025] S4: Perform checks on the completeness of evidence, timeliness of evidence, continuity of citation, version consistency, and chain of responsibility for each audit unit;

[0026] S5: When evidence is found to be missing, expired, or broken, or when multiple versions of conclusions are inconsistent or the responsible point is missing, the system will automatically mark the risk level and trigger the process of supplementing evidence, re-examination, or manual review.

[0027] S6: Generates audit views that include clause content, source evidence, participating agents, generation time, version relationship, risk level, and remediation status, enabling visualized audit output and supporting export and archiving.

[0028] Furthermore, in step S1, the process of generating audit units is as follows: The system divides the final output O into a set of audit units U={u1,u2,...,un}, and each audit unit u_i is represented as:

[0029] u_i=<unit_id, text, type, path, offset, hash, version> ;

[0030] Where unit_id is the audit unit identifier, text is the clause text, type indicates clause, paragraph or field, path indicates the structural path in the report or result, offset indicates the character offset, hash is the normalized text hash, and version is the output version.

[0031] Furthermore, in step S2, during the construction of the evidence chain, for each u_i, the system extracts evidence object e_j from process logs, retrieval results, database queries, tool calls, and model call records, and constructs the evidence chain:

[0032] L(u_i)={e_j | match(u_i,e_j) >= theta_e};

[0033] Where match(u_i,e_j) represents the matching degree between the audit unit and the evidence object, and theta_e is the evidence matching threshold. The evidence object includes at least the source type, source identifier, fragment location, generation time, participating agent, tool version, prompt template version, and hash signature;

[0034] Matching degree calculation:

[0035] ;

[0036] Where S_text represents text or semantic similarity, S_ref represents explicit reference matching, S_time represents temporal order consistency, and S_agent represents agent responsibility chain matching; c1 to c4 are weights, all ranging from 0 to 1, and c1+c2+c3+c4=1.

[0037] Furthermore, in step S3, the mapping relationship is established using a graph database, log index, or event stream to store the audit relationship, and the audit unit hash, evidence object hash, and process event hash are chained together:

[0038] H_i=Hash(H_(i-1)||hash(u_i)||hash(L(u_i))||hash(P(u_i)));

[0039] Where P(u_i) represents the set of process events related to the audit unit, and H_i represents the current chain summary. The chain summary can be used to verify whether the audit record has been tampered with.

[0040] Furthermore, in step S4, checks are performed on each audit unit for evidence completeness, evidence timeliness, citation continuity, version consistency, and chain of responsibility integrity; the audit consistency check involves calculating a risk score R(u) for each audit unit.

[0041] ;

[0042] Where M(u) represents the degree of missing evidence, E(u) represents the degree of expired evidence, B(u) represents the degree of broken citation chain, V(u) represents the degree of inconsistency among multiple versions, and A(u) represents the degree of missing chain of responsibility; d1 to d5 are weights, all ranging from 0 to 1, and d1+d2+d3+d4+d5=1.

[0043] Furthermore, when R(u) is greater than the risk threshold theta_r, the system marks the audit unit as a risk clause and triggers supplementary verification, re-examination, or manual review according to the risk type; if R(u) is less than or equal to theta_r, the audit unit is marked as audit passed.

[0044] A multi-agent collaborative output evidence tracing and auditing system includes a multi-agent collaborative output layer, a clause-level segmentation layer, an evidence association chain building layer, a collaborative process traceability layer, and an audit verification and output layer;

[0045] The multi-agent collaboration output layer is used to receive the output content generated by multi-agent collaboration. The output content includes report results, code descriptions, review conclusions, and question and answer results.

[0046] The clause-level segmentation divides the output content into clause-level, sentence-level, or field-level audit units, generating unique identifiers and hash signatures for each audit unit.

[0047] The evidence association chain building layer automatically associates the corresponding retrieval document fragments, database records, tool calculation results, model call parameters, prompt template versions, participating agent identifiers, and manual revision records for each audit unit to generate an evidence tracing chain;

[0048] The collaborative process traceability layer establishes an evidence tracing chain and a collaborative process mapping;

[0049] The audit verification and output layer determines whether there is missing, expired, broken, or inconsistent evidence. If so, it calculates the risk level and triggers supplementary evidence, re-examination, or manual review. If not, it marks the audit as passed and finally outputs an audit view containing terms, evidence, agent, time, version, and risk warnings.

[0050] Furthermore, the collaborative process traceability layer includes a chain storage module, which writes the evidence chain, process chain, and version chain into a graph database, log index, event stream, or tamper-proof storage, and maintains hash verification relationships.

[0051] Furthermore, the audit verification and output layer includes an audit consistency verification module, a remediation trigger module, and a visual audit output module. The audit consistency verification module checks whether evidence is missing, expired, broken, or inconsistent with the current version, and calculates the risk level. The remediation trigger module triggers supplementary evidence, re-examination, re-retrieval, regeneration, or manual review when the risk level reaches a threshold. The visual audit output module displays the source evidence, participating agents, generation time, version relationship, risk warning, and remediation status according to the clause.

[0052] The present invention has the following advantages:

[0053] (1) Finer audit granularity. This invention uses clauses, paragraphs or fields as audit units, rather than tracing the entire report or document as a whole.

[0054] (2) The evidence chain is more complete. This invention simultaneously binds retrieval fragments, database records, tool results, model call parameters, prompt template versions, participating agents, and manual revision records.

[0055] (3) Enhanced accountability. This invention maps task assignment, message passing, context trimming, tool invocation, and result revision in the collaboration process to the final output terms.

[0056] (4) Risk detection is more timely. This invention can automatically detect missing evidence, expired evidence, broken citations, inconsistencies between multiple versions, and missing chains of responsibility, and trigger supplementary evidence or retrial.

[0057] (5) Audit output is more suitable for high-credibility scenarios. The output of this invention includes an audit view containing clause content, source evidence, participating agents, generation time, version relationship and risk warning, which is convenient for use by regulators, legal departments and enterprise internal control.

[0058] (6) Enhanced tamper resistance. This invention can solidify audit records through hash signatures, version signatures, and chain digests, reducing the risk of subsequent tampering. Attached Figure Description

[0059] Figure 1 This is a block diagram of the multi-agent collaborative output evidence tracing and auditing system;

[0060] Figure 2 This is a flowchart of the multi-agent collaborative output evidence tracing and auditing method. Detailed Implementation

[0061] like Figure 1 As shown, the multi-agent collaborative output evidence tracing and auditing system includes a multi-agent collaborative output layer, a clause-level segmentation layer, an evidence association chain building layer, a collaborative process logging layer, and an audit verification and output layer. These layers sequentially complete output reception, audit unit generation, evidence chain binding, process chain binding, risk verification, and audit view output. Specifically, the multi-agent collaborative output layer receives multi-agent collaborative output and process logs; the clause-level segmentation layer divides the output into clause-level, sentence-level, or field-level audit units, generating unique identifiers and hash signatures for each audit unit; the evidence association chain building layer associates and retrieves document fragments, database records, tool results, and model call information; the collaborative process logging layer establishes an evidence tracing chain and a collaborative process mapping; the audit verification and output layer determines whether there is missing, expired, broken, or inconsistent evidence across versions. If so, it calculates the risk level and triggers supplementary evidence, re-examination, or manual review; if not, it marks the audit as passed; finally, it outputs an audit view containing clauses, evidence, agents, time, version, and risk warnings.

[0062] The multi-agent collaboration output layer is used to receive the output content generated by multi-agent collaboration. The output content includes report results, code descriptions, review conclusions, question and answer results, etc.

[0063] The clause-level segmentation layer segments the output content of reports, code descriptions, review conclusions, or Q&A results. This includes an output segmentation module, which segments the content into clause-level, sentence-level, or field-level audit units. Each audit unit generates a unique identifier (unit_id), a structure path (location information), a character offset, a hash signature, and a version number. The audit unit generation process is as follows: The system segments the final output O into an audit unit set U = {u1, u2, ..., un}. Each audit unit u_i is represented as:

[0064] u_i=<unit_id, text, type, path, offset, hash, version> ;

[0065] Where unit_id is the audit unit identifier, text is the clause text, type indicates clause, paragraph or field, path indicates the structural path in the report or result, offset indicates the character offset, hash is the normalized text hash, and version is the output version.

[0066] The evidence association chain building layer automatically associates with corresponding retrieval document fragments, database records, tool calculation results, model call parameters, prompt template versions, participating agent identifiers, and manual revision records for each audit unit, generating an evidence tracing chain.

[0067] When constructing the evidence chain, for each u_i, the system extracts evidence object e_j from process logs, retrieval results, database queries, tool calls, and model call records, and constructs the evidence chain:

[0068] L(u_i)={e_j | match(u_i,e_j) >= theta_e};

[0069] Where match(u_i,e_j) represents the matching degree between the audit unit and the evidence object, and theta_e is the evidence matching threshold. The evidence object includes at least the source type, source identifier, fragment location, generation time, participating agent, tool version, prompt template version, and hash signature.

[0070] The matching degree can be calculated using the following formula:

[0071] ;

[0072] Where S_text represents text or semantic similarity, S_ref represents explicit reference matching, S_time represents temporal order consistency, and S_agent represents agent responsibility chain matching; c1 to c4 are weights, all ranging from 0 to 1, and c1+c2+c3+c4=1.

[0073] The collaborative process traceability layer records message passing, task assignment, context pruning, tool invocation, result revision, and version switching behaviors between agents, and establishes a mapping relationship with the final output audit unit.

[0074] The collaborative process audit layer includes a chained storage module. This module writes the evidence chain, process chain, and version chain into a graph database, log index, event stream, or immutable storage, and maintains hash verification relationships. Graph databases, log indexes, or event streams can be used to store audit relationships. For high-trust scenarios, the audit unit hash, evidence object hash, and process event hash can be chained together.

[0075] H_i=Hash(H_(i-1)||hash(u_i)||hash(L(u_i))||hash(P(u_i)));

[0076] Where P(u_i) represents the set of process events related to the audit unit, and H_i represents the current chain summary. The chain summary can be used to verify whether the audit record has been tampered with.

[0077] The audit verification and output layer needs to perform operations such as evidence integrity verification, version consistency verification, risk level marking, and visual audit view.

[0078] The evidence integrity verification and version consistency verification are used to complete the audit consistency verification, which is implemented through the audit consistency verification module: For each audit unit, verifications are performed on evidence integrity, evidence timeliness, citation continuity, version consistency, and chain of responsibility integrity. The audit consistency verification calculates a risk score R(u) for each audit unit.

[0079] ;

[0080] Where M(u) represents the degree of missing evidence, E(u) represents the degree of expired evidence, B(u) represents the degree of broken citation chain, V(u) represents the degree of inconsistency among multiple versions, and A(u) represents the degree of missing chain of responsibility; d1 to d5 are weights, all ranging from 0 to 1, and d1+d2+d3+d4+d5=1.

[0081] When R(u) is greater than the risk threshold theta_r, the system marks the audit unit as a risk clause and triggers supplementary evidence, re-examination, or manual review based on the risk type. If R(u) is less than or equal to theta_r, the audit unit is marked as audit passed.

[0082] Risk level marking is an automatic risk level marking process triggered by the remedial trigger module when evidence is found to be missing, expired, broken, inconsistent conclusions in multiple versions, or missing responsibility points. This triggers the process of supplementing evidence, re-examination, or manual review.

[0083] The visual audit view is generated by the visual audit output module. It includes the content of the terms, source evidence, participating agents, generation time, version relationship, risk level and remediation status, and supports export and archiving.

[0084] Key technical parameters involved in the multi-agent collaborative output evidence tracing and auditing system:

[0085] The evidence matching threshold theta_e is used to determine whether an evidence object can be bound to an audit unit. It is recommended that the value be between 0.60 and 0.90.

[0086] Risk threshold theta_r: Used to determine whether an audit unit needs supplementary documentation, re-audit, or manual review. The recommended value range is 0.40 to 0.80.

[0087] Evidence validity period T_e: Used to determine whether evidence has expired. It can be set to days, months or version periods according to business scenarios.

[0088] Hash algorithm: SHA-256, SM3, or other hash algorithms that meet the deployment environment requirements can be used.

[0089] Audit unit granularity: can be set by clause, paragraph or field. Field level or paragraph level is preferred for high-risk scenarios.

[0090] Risk level: can be set to three levels: low, medium and high, or can be expanded to a continuous score from 0 to 100.

[0091] like Figure 2 As shown, the multi-agent collaborative output evidence tracing and auditing method includes the following steps:

[0092] S1: Output Clause-level Segmentation: Receive reports, code descriptions, review conclusions, or Q&A results generated by multi-agent collaboration, segment them into clause-level, sentence-level, or field-level audit units, and generate unit_id, hash signature, and location information for each audit unit.

[0093] The output segmentation module breaks down reports, code descriptions, review conclusions, or Q&A results into auditable units and generates unique identifiers, hash signatures, character offsets, paragraph paths, and display positions.

[0094] The process of generating audit units is as follows: The system divides the final output O into a set of audit units U = {u1, u2, ..., un}. Each audit unit u_i is represented as:

[0095] u_i=<unit_id, text, type, path, offset, hash, version> ;

[0096] Where unit_id is the audit unit identifier, text is the clause text, type indicates clause, paragraph or field, path indicates the structural path in the report or result, offset indicates the character offset, hash is the normalized text hash, and version is the output version.

[0097] S2: Evidence Association Chain Building: For each audit unit, automatically associate the corresponding search document fragments, database records, tool calculation results, model call parameters, prompt template versions, participating agent identifiers, and manual revision records to generate an evidence tracing chain.

[0098] When constructing the evidence chain, for each u_i, the system extracts evidence object e_j from process logs, retrieval results, database queries, tool calls, and model call records, and constructs the evidence chain:

[0099] L(u_i)={e_j | match(u_i,e_j) >= theta_e};

[0100] Where match(u_i,e_j) represents the matching degree between the audit unit and the evidence object, and theta_e is the evidence matching threshold. The evidence object includes at least the source type, source identifier, fragment location, generation time, participating agent, tool version, prompt template version, and hash signature.

[0101] The matching degree can be calculated using the following formula:

[0102] ;

[0103] Where S_text represents text or semantic similarity, S_ref represents explicit reference matching, S_time represents temporal order consistency, and S_agent represents agent responsibility chain matching; c1 to c4 are weights, all ranging from 0 to 1, and c1+c2+c3+c4=1.

[0104] S3: Collaboration Process Traceability: Records message passing, task assignment, context trimming, tool calls, result revision and version switching behaviors between agents, and establishes a mapping relationship with the final output audit unit.

[0105] The chained storage module writes the evidence chain, process chain, and version chain into a graph database, log index, event stream, or immutable storage, and maintains the hash verification relationship.

[0106] Audit relationships can be stored using graph databases, log indexes, or event streams. For high-trust scenarios, audit unit hashes, evidence object hashes, and process event hashes can be chained together.

[0107] H_i=Hash(H_(i-1)||hash(u_i)||hash(L(u_i))||hash(P(u_i)));

[0108] Where P(u_i) represents the set of process events related to the audit unit, and H_i represents the current chain summary. The chain summary can be used to verify whether the audit record has been tampered with.

[0109] S4: Audit Consistency Verification: Perform verification of evidence integrity, evidence timeliness, citation continuity, version consistency, and chain of responsibility integrity for each audit unit.

[0110] The audit consistency verification module checks whether evidence is missing, expired, broken, or inconsistent with the current version, and calculates the risk level.

[0111] Audit consistency verification involves calculating a risk score R(u) for each audit unit:

[0112] ;

[0113] Where M(u) represents the degree of missing evidence, E(u) represents the degree of expired evidence, B(u) represents the degree of broken citation chain, V(u) represents the degree of inconsistency among multiple versions, and A(u) represents the degree of missing chain of responsibility; d1 to d5 are weights, all ranging from 0 to 1, and d1+d2+d3+d4+d5=1.

[0114] S5: Risk Marking and Remedial Trigger: When evidence is found to be missing, expired, or broken, or when multiple versions of conclusions are inconsistent or responsibility points are missing, the system automatically marks the risk level and triggers the process of supplementing evidence, re-examination, or manual review.

[0115] The remedial trigger module can trigger supplementary certification, re-examination, re-retrieval, regeneration, or manual review when the risk level reaches the threshold.

[0116] When R(u) is greater than the risk threshold theta_r, the system marks the audit unit as a risk clause and triggers supplementary evidence, re-examination, or manual review based on the risk type. If R(u) is less than or equal to theta_r, the audit unit is marked as audit passed.

[0117] S6: Visualized Audit Output: Generates audit views that include clause content, source evidence, participating agents, generation time, version relationships, risk level, and remediation status, and supports export and archiving.

[0118] The visual audit output module displays source evidence, participating agents, generation time, version relationships, risk warnings, and remediation status by clause.

[0119] Example 1: Evidence tracing audit of patent search and analysis reports;

[0120] In a patent search and analysis scenario, multiple agents are responsible for tasks such as keyword expansion, patent search, prior art summary, inventive step assessment, and report generation. After receiving the final search and analysis report, the system divides it into audit units such as "search keywords," "prior art summary," "distinguishing points," and "inventive step conclusion."

[0121] For audit units identifying "distinction points," the system automatically associates original text fragments of the comparison documents, retrieves database records, generates intelligent agent output summaries, and performs creative judgment on agent messages and prompt template versions. If a missing comparison document fragment is found for a particular distinction, the system marks the unit as medium risk and triggers supplementary evidence. After supplementary evidence is obtained, the system updates the evidence chain and version chain, ultimately outputting an audit view that can be reviewed by the agent.

[0122] Example 2: Tracking the Responsible Nodes in Code Review Reports;

[0123] In a code review scenario, the planning agent assigns file inspection tasks, the code analysis agent generates risk conclusions, the testing agent generates test results, and the review agent generates a final report. The system divides each risk item in the final report into audit units and binds them to corresponding code snippets, static scan results, test logs, agent messages, and generation time.

[0124] When a risk item references a code version that has been subsequently modified, the system detects a version inconsistency, calculates a risk score R(u) greater than theta_r, and triggers a re-audit process. After the re-audit, the system generates a new version audit unit while retaining the old version's traceability to facilitate tracking the reasons for the report changes.

[0125] The multi-agent collaborative output evidence tracing and auditing method enables the system to establish verifiable evidence chains and process chains for each key clause in the final output, and has the following characteristics:

[0126] (1) Divide the multi-agent collaborative output into clause-level, sentence-level, or field-level audit units.

[0127] (2) For each audit unit, retrieve document fragments, database records, tool calculation results, model call parameters, prompt template version and participating agent identifiers.

[0128] (3) Record and map message passing, task assignment, context trimming, tool invocation and result revision behaviors between agents.

[0129] (4) Automatically detect missing evidence, expired evidence, broken citations and inconsistent conclusions in multiple versions, and trigger supplementary evidence, re-examination or manual review.

[0130] (5) Output a visual audit view that includes the terms, source evidence, participating agents, generation time, version relationship and risk warning.

[0131] This invention can be applied to scenarios with high credibility requirements, such as patents, legal affairs, finance, healthcare, auditing, compliance, government and enterprise reports, code review, and enterprise knowledge base Q&A. The system can adjust the granularity of audit units and risk thresholds according to the scenario. For example, field-level auditing and increased requirements for supplementary evidence can be adopted in legal or healthcare scenarios; clause-level auditing can be adopted for general office reports to reduce storage and computing costs.

[0132] The evidence matching threshold, risk threshold, hash algorithm, storage method, and risk level classification in this invention can be configured according to the deployment environment. The above embodiments are used to illustrate possible implementations of this invention and do not constitute a limitation on the scope of protection of the claims.

Claims

1. A method for tracing and auditing the source of evidence output through multi-agent collaboration, comprising the following steps: S1: Receive reports, code descriptions, review conclusions, or Q&A results generated by multi-agent collaboration, divide them into audit units at the clause level, sentence level, or field level, and generate unit_id, hash signature, and location information for each audit unit; S2: For each audit unit, automatically associate the corresponding search document fragments, database records, tool calculation results, model call parameters, prompt template version, participating agent identifiers and manual revision records to generate an evidence tracing chain; S3: Records message passing, task assignment, context trimming, tool invocation, result revision and version switching behaviors between agents, and establishes a mapping relationship with the final output audit unit; S4: Perform checks on the completeness of evidence, timeliness of evidence, continuity of citation, version consistency, and chain of responsibility for each audit unit; S5: When evidence is found to be missing, expired, or broken, or when multiple versions of conclusions are inconsistent or the responsible point is missing, the system will automatically mark the risk level and trigger the process of supplementing evidence, re-examination, or manual review. S6: Generates audit views that include clause content, source evidence, participating agents, generation time, version relationship, risk level, and remediation status, enabling visualized audit output and supporting export and archiving.

2. The multi-agent collaborative output evidence tracing and auditing method according to claim 1, characterized in that: In step S1, the process of generating audit units is as follows: The system divides the final output O into a set of audit units U={u1,u2,...,un}, and each audit unit u_i is represented as: u_i=<unit_id, text, type, path, offset, hash, version> ; Where unit_id is the audit unit identifier, text is the clause text, type indicates clause, paragraph or field, path indicates the structural path in the report or result, offset indicates the character offset, hash is the normalized text hash, and version is the output version.

3. The multi-agent collaborative output evidence tracing and auditing method according to claim 1, characterized in that: In step S2, during the construction of the evidence chain, for each u_i, the system extracts evidence object e_j from process logs, retrieval results, database queries, tool calls, and model call records, and constructs the evidence chain: L(u_i)={e_j | match(u_i,e_j) >= theta_e}; Where match(u_i,e_j) represents the matching degree between the audit unit and the evidence object, and theta_e is the evidence matching threshold; the evidence object includes at least the source type, source identifier, fragment location, generation time, participating agent, tool version, prompt template version, and hash signature.

4. The multi-agent collaborative output evidence tracing and auditing method according to claim 3, characterized in that: The matching degree is calculated as follows: ； Where S_text represents text or semantic similarity, S_ref represents explicit reference matching, S_time represents temporal order consistency, and S_agent represents agent responsibility chain matching; c1 to c4 are weights, all ranging from 0 to 1, and c1+c2+c3+c4=1.

5. The multi-agent collaborative output evidence tracing and auditing method according to claim 1, characterized in that: In step S3, the mapping relationship is established using a graph database, log index, or event stream to store the audit relationship, and the audit unit hash, evidence object hash, and process event hash are chained together: H_i=Hash(H_(i-1)||hash(u_i)||hash(L(u_i))||hash(P(u_i))); Where P(u_i) represents the set of process events related to the audit unit, and H_i represents the current chain summary. The chain summary can be used to verify whether the audit record has been tampered with.

6. The multi-agent collaborative output evidence tracing and auditing method according to claim 1, characterized in that: In step S4, checks are performed on the completeness of evidence, timeliness of evidence, continuity of citation, version consistency, and chain of responsibility for each audit unit; the audit consistency check involves calculating a risk score R(u) for each audit unit. ； Where M(u) represents the degree of missing evidence, E(u) represents the degree of expired evidence, B(u) represents the degree of broken citation chain, V(u) represents the degree of inconsistency among multiple versions, and A(u) represents the degree of missing chain of responsibility; d1 to d5 are weights, all ranging from 0 to 1, and d1+d2+d3+d4+d5=1.

7. The multi-agent collaborative output evidence tracing and auditing method according to claim 6, characterized in that: When R(u) is greater than the risk threshold theta_r, the system marks the audit unit as a risk clause and triggers supplementary verification, re-examination, or manual review according to the risk type; if R(u) is less than or equal to theta_r, the audit unit is marked as audit passed.

8. A multi-agent collaborative output evidence tracing and auditing system, characterized in that: This includes a multi-agent collaboration output layer, a clause-level segmentation layer, an evidence association and chain-building layer, a collaboration process auditing and verification layer, and an audit verification and output layer. The multi-agent collaboration output layer is used to receive the output content generated by multi-agent collaboration. The output content includes report results, code descriptions, review conclusions, and question and answer results. The clause-level segmentation divides the output content into clause-level, sentence-level, or field-level audit units, generating unique identifiers and hash signatures for each audit unit. The evidence association chain building layer automatically associates the corresponding retrieval document fragments, database records, tool calculation results, model call parameters, prompt template versions, participating agent identifiers, and manual revision records for each audit unit to generate an evidence tracing chain; The collaborative process traceability layer establishes an evidence tracing chain and a collaborative process mapping; The audit verification and output layer determines whether there is missing, expired, broken, or inconsistent evidence. If so, it calculates the risk level and triggers supplementary evidence, re-examination, or manual review. If not, it marks the audit as passed and finally outputs an audit view containing terms, evidence, agent, time, version, and risk warnings.

9. The multi-agent collaborative output evidence tracing and auditing system according to claim 8, characterized in that: The collaborative process traceability layer includes a chain storage module, which writes the evidence chain, process chain, and version chain into a graph database, log index, event stream, or tamper-proof storage, and maintains hash verification relationships.

10. The multi-agent collaborative output evidence tracing and auditing system according to claim 8, characterized in that: The audit verification and output layer includes an audit consistency verification module, a remediation trigger module, and a visual audit output module. The audit consistency verification module checks whether the evidence is missing, expired, broken, or inconsistent with the current version, and calculates the risk level. The remediation trigger module triggers supplementary evidence, re-examination, re-retrieval, regeneration, or manual review when the risk level reaches the threshold; the visual audit output module displays the source evidence, participating agents, generation time, version relationship, risk warning, and remediation status according to the clause.

Images