A financial intelligent agent output content evidence chain verification and responsibility boundary marking system

Abstract

The application discloses a financial intelligent agent output content evidence chain verification and responsibility boundary marking system, and relates to the field of financial content security.The system comprises a content type identification module, a sensitive field detection module, an evidence matching module, a boundary marking module, a rewriting and intercepting module and an artificial confirmation module.The content type identification module is used for receiving the content to be output by a financial intelligent agent and the generation context thereof, and identifying the business type of the output content.The evidence matching module is used for matching the output basis for the output content according to the business type.The sensitive field detection module is used for performing special detection on the financial sensitive fields in the output content.The boundary marking module is used for marking the responsibility boundary of the output content according to the matching result and the detection result.The rewriting and intercepting module is used for executing output, prompting, rewriting, transferring to an artificial or intercepting according to the responsibility boundary.The artificial confirmation module is used for providing an artificial confirmation channel.The application can realize the traceability of the source of the output content of the financial intelligent agent, the controllability of compliance, and has the self-adaptive and continuous optimization capability based on business feedback.

Description

Technical Field

[0001] This invention relates to the field of financial content security, and in particular to a system for verifying the evidence chain and marking the responsibility boundary of financial intelligent agent output content. Background Technology

[0002] Financial intelligent agents, as an advanced form of generative artificial intelligence in the financial field, are evolving from auxiliary efficiency tools to collaborative partners deeply involved in business processes and decision-making. Currently, financial intelligent agents based on large-scale language models have been deployed at scale in scenarios such as customer service and marketing outreach. However, financial businesses have almost stringent requirements for the accuracy, traceability, and compliance of output content. Any statements involving revenue, limits, approval conclusions, risk ratings, fee explanations, and customer rights must have clear sources and verifiable supporting materials. At the same time, the inherent probabilistic generation characteristics of large models naturally lead to problems such as "model illusion," unexplainable reasoning, and uncertainty of results in their output content. Against this backdrop, ensuring the credibility and compliance control of the output content of financial intelligent agents has become a key bottleneck restricting the large-scale implementation of this technology in the financial industry. The industry has begun to explore ways to build evidence chains for the content generated by large models through data hierarchical management, retrieval-enhanced generation, and output-side fact-checking. Some relevant group standards have also proposed a reference path for the credible governance of large models in the financial field in a systematic way for the first time.

[0003] Existing solutions for the security and compliance management of financial intelligent agents suffer from the following shortcomings: First, relying on existing models such as sensitive word filtering, rule interception, or manual review makes it difficult to perform structured matching and verification of the source evidence for each key conclusion in the intelligent agent's output content; there is a lack of clear typological distinctions and differentiated processing strategies among factual statements, business suggestions, risk warnings, and matters requiring manual confirmation in the output content. Second, existing compliance review mechanisms generally lack the ability to mark responsibility boundaries for financial business scenarios, and cannot define graded responsibility boundaries for output results based on the sufficiency of evidence, the hit rate of sensitive fields, and the degree of matching with compliance rules. Third, existing solutions lack the ability to audit and record the intelligent agent's output process; key information such as the generation basis of the output content and the hit rate of rules lacks a structured retention and associated storage mechanism, making it difficult to support post-event compliance review, complaint handling, and responsibility determination. Therefore, based on the above problems, this invention proposes a system for verifying the evidence chain and marking responsibility boundaries of financial intelligent agent output content. Summary of the Invention

[0004] To address the aforementioned issues, the present invention aims to provide a system for verifying the evidence chain and marking the responsibility boundary of the output content of a financial intelligent agent. This system enables each key conclusion in the output content of the financial intelligent agent to be traced back to its corresponding source of evidence, achieving differentiated compliance control over output content at different risk levels. It provides a complete, auditable, and reproducible record of the output process and allows the system to continuously optimize and adapt to changes in business scenarios, thus meeting the differentiated compliance requirements of different business scenarios.

[0005] To achieve the above objectives, this invention provides a system for verifying the evidence chain and marking the responsibility boundary of financial intelligent agent output content. The system includes a content type identification module, an evidence matching module, a sensitive field detection module, a boundary marking module, a rewriting and interception module, a manual confirmation module, and an audit record module. Upon receiving the content to be output and its context, the system first identifies the business type. Then, the evidence matching module matches the output basis from various data sources, while the sensitive field detection module detects financially sensitive fields. The boundary marking module, based on the matching and detection results, marks five levels of responsibility boundaries: direct output, requiring supplementary prompts, requiring rewriting, requiring manual confirmation, or prohibited output. The rewriting and interception module performs corresponding operations accordingly. The audit record module generates an audit record containing evidence source, rule hit, boundary marking, processing action, and manual confirmation status. The system uses a feedback optimization interface to feed back execution feedback, manual confirmation results, anomaly handling status, and changes in business indicators to the evidence matching and boundary marking module, continuously updating the basis and rule databases. This system can verify the basis source and compliance boundaries of each item of intelligent agent output content, achieving unified management of financial intelligent agent output content across three dimensions: evidence support, compliance control, and responsibility traceability.

[0006] In a first aspect, the present invention provides a system for verifying the chain of evidence and marking the boundary of responsibility of financial intelligent agents' output content, comprising: The content type recognition module is used to receive the content to be output by the financial intelligent agent and its generation context, and to identify the business type of the output content; An evidence matching module is used to match output evidence for the output content from at least one data source according to the business type; The sensitive field detection module is used to perform specific detection on financially sensitive fields in the output content; A boundary marking module is used to mark the responsibility boundary of the output content based on the matching result of the evidence matching module and the detection result of the sensitive field detection module; The rewrite and intercept module is used to perform direct output, output with supplementary prompts, rewrite, transfer to manual confirmation, or interception operations on the output content according to the responsibility boundary. The manual confirmation module is used to provide a manual confirmation channel, receive the content to be confirmed transferred by the rewrite and interception module, and generate manual confirmation status information; The audit log module is used to generate audit logs for the output content; The responsibility boundaries marked by the boundary marking module include five levels: direct output, requiring supplementary prompts, requiring rewriting, requiring manual confirmation, and prohibited output. The audit records include at least evidence source information, rule hit information, boundary marking information, processing action information, and manual confirmation status information.

[0007] Furthermore, the data source includes a knowledge base, a business rule base, a process rule base, a product rule base, and a manual confirmation record base; the evidence matching module compares the key information of the output content with the structured objects stored in the data source to match the corresponding output basis, and the structured objects form a unified financial business object model through preset association relationships.

[0008] Furthermore, the sensitive field detection module performs item-by-item detection on the financial sensitive fields in the output content based on a preset sensitive field library and semantic recognition rules, and marks the hit sensitive fields and their attributes; the boundary marking module determines the level of the responsibility boundary according to the degree of matching between the attributes of the sensitive fields and the preset compliance boundary rules.

[0009] Furthermore, the boundary marking module determines the responsibility boundary as one of the five levels according to the matching result and the detection result, based on preset compliance boundary rules; wherein, when the matching result or the detection result meets preset human intervention conditions, the boundary marking module marks the responsibility boundary as requiring human confirmation; the rewriting and interception module transfers the content to be confirmed to the human confirmation module, the human confirmation module generates human confirmation status information, and the audit record module records the human confirmation status information.

[0010] Furthermore, the rewriting and interception module includes a rewriting unit, an interception unit, and a manual confirmation and transfer unit. The rewriting unit is used to automatically correct the output content marked as needing rewriting according to a preset rewriting template or rules, and add references or risk warnings. The interception unit is used to block the output content marked as prohibited and trigger an alarm. The manual confirmation and transfer unit is used to transfer the output content marked as needing manual confirmation to the manual confirmation module. When the rewriting unit corrects the output content, it records the content before and after correction and the corresponding rewriting rules in the audit record.

[0011] Furthermore, the system also includes a feedback optimization interface for receiving execution feedback from external input, manual confirmation results from the manual confirmation module, and abnormal handling status or business indicator change data, and feeding the data back to the evidence matching module and the boundary marking module to update the rules or basis in the data source; wherein the rules are stored in at least one of the business rule base, process rule base, and product rule base, and the basis is stored in at least one of the knowledge base and manual confirmation record base; the system adjusts the matching strategy of the evidence matching module and the marking threshold of the boundary marking module according to the execution feedback.

[0012] Furthermore, the business types identified by the content type identification module include factual statements, business suggestions, customer tips, risk warnings, product descriptions, and operation guidelines; the evidence matching module matches corresponding evidence templates and verification rules from the data source according to different business types. The evidence templates define the evidence fields to be matched and their source requirements, and the verification rules define compliance check items.

[0013] Furthermore, the evidence matching module breaks down the output content into several atomic assertions. For each atomic assertion, it retrieves candidate evidence sources from the data source and evaluates the correlation strength between each candidate evidence source and the corresponding atomic assertion from multiple dimensions. The candidate evidence source whose correlation strength meets the preset conditions is selected as the main evidence for that atomic assertion. The evidence matching module performs an aggregate evaluation based on the evidence support of all atomic assertions to generate an evidence chain integrity evaluation result for the output content. Based on the comparison between the evaluation result and the preset integrity threshold, it determines the path for the output content to enter the subsequent processing flow.

[0014] Furthermore, the boundary marking module also includes an adaptive threshold adjustment unit, which is used to obtain historical compliance feedback data through the feedback optimization interface, and dynamically adjust the comprehensive judgment threshold used by the boundary marking module when determining the responsibility boundary based on the historical compliance feedback data; when adjusting the comprehensive judgment threshold, the adaptive threshold adjustment unit suppresses the adjustment range based on the dispersion of the historical compliance feedback data; the boundary marking module uses the adjusted comprehensive judgment threshold to mark the responsibility boundary of subsequent output content.

[0015] Secondly, the present invention also provides a method for verifying the evidence chain and marking the responsibility boundary of the output content of a financial intelligent agent, the method being based on the system described in the first aspect above, comprising: Receive the content to be output by the financial intelligent agent and its generation context; Identify the business type of the output content; Based on the business type, match the output criteria for the output content from at least one data source; Special checks are performed on the financially sensitive fields in the output content; Based on the matched output criteria and the detection results of the financial sensitive fields, the responsibility boundaries of the output content are marked. The responsibility boundaries include direct output, supplementary prompts required, rewriting required, manual confirmation required, and output prohibited. Based on the defined responsibility boundaries, the output content may be processed as follows: direct output, output with supplementary prompts, rewritten, transferred to manual confirmation, or intercepted. An audit log is generated for the output content. The audit log includes at least evidence source information, rule hit information, boundary marker information, processing action information, and manual confirmation status information.

[0016] This invention integrates evidence matching, sensitive field detection, responsibility boundary marking, and audit records into a unified framework, enabling every key conclusion in the output content to be traceable to its corresponding basis, thus systematically improving the verifiability of the output content. Based on the comprehensive judgment of evidence matching and sensitive fields, it achieves differentiated responsibility boundary marking, establishing clear typological distinctions and differentiated control strategies between factual statements, business suggestions, risk warnings, and matters requiring manual confirmation. Through structured audit records, it provides complete post-event traceability for the output process. By introducing an execution feedback and processing result feedback mechanism, the evidence matching strategy and boundary marking thresholds can continuously adapt and optimize according to business scenarios, maintaining the effectiveness and adaptability of compliance control in different financial business scenarios. Ultimately, it provides systematic technical guarantees for the output content in terms of the sufficiency of evidence support, the clarity of compliance boundaries, the proactive nature of risk control, and the traceability of responsibility attribution.

[0017] Beneficial effects By implementing the financial intelligent agent output content evidence chain verification and responsibility boundary marking system provided by the present invention, the following technical effects are achieved: (1) By establishing a structured object model for financial business scenarios, the atomic elements involved in financial business are abstracted into structured objects with pre-defined relationships. Under this unified model framework, the evidence matching module achieves unified addressing and association retrieval of multi-source heterogeneous data. This object model transforms the location of evidence sources from a text retrieval mode based on keyword matching to a structured query mode based on object relationship traversal, significantly improving the semantic accuracy of the evidence matching process and its scalability in complex business scenarios. At the same time, it provides a unified data foundation for the structured association storage of evidence source information and output content in the audit record module.

[0018] (2) By introducing a feedback optimization interface at the end of the execution chain and feeding back execution feedback, manual confirmation results, anomaly handling status, and changes in business metrics to the evidence matching module and boundary marking module, the system's evidence matching strategy and boundary marking thresholds can be iteratively corrected in a closed loop based on real business operation results. This feedback mechanism enables the system to extract new evidence sources from manual confirmation records, update compliance boundary rules from anomaly handling results, and perceive scenario drift from changes in business metrics. Thus, without relying on external system reconfiguration, the system can achieve continuous accumulation and adaptive evolution of its internal knowledge and rule assets.

[0019] (3) By constructing a quantitative evaluation mechanism for the strength of multidimensional evidence association, the matching relationship between the output content of the financial intelligent agent and the source of evidence is refined from the traditional binary judgment to a continuous multidimensional quantitative evaluation. This mechanism can differentiate the weighting according to the semantic importance of each atomic assertion in the output content, so that assertions involving key sensitive content such as profit commitments, approval conclusions and risk warnings can obtain higher contribution weights in the evidence chain integrity score. This effectively exposes weak links in evidence support at the quantitative level, and provides a calculable decision basis for the subsequent refined determination of liability boundaries.

[0020] (4) By introducing an adaptive threshold adjustment mechanism based on historical compliance feedback, the threshold for determining the boundary of responsibility is no longer fixed as a preset static value, but can continuously and dynamically evolve with historical data such as manual review results, compliance review conclusions, and customer feedback accumulated during system operation. The fluctuation suppression factor introduced in this mechanism can ensure that the threshold maintains sufficient adjustment inertia when the feedback data has a high degree of dispersion, avoiding threshold oscillation caused by short-term data disturbances, so that the system's compliance control strategy can achieve a balance between stability and adaptability, thereby maintaining the effectiveness and rationality of the boundary of responsibility marking in different financial business scenarios and regulatory environments. Attached Figure Description

[0021] To make the above-described financial intelligent agent output content evidence chain verification and responsibility boundary marking system of the present invention more obvious and understandable, the drawings used in the specific embodiments of the present invention will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0022] Figure 1 This is a flowchart illustrating the process of this invention. Figure 2 This is a flowchart illustrating the evaluation process for a multidimensional chain of evidence. Detailed Implementation

[0023] Example 1: In a specific application scenario of a financial account manager assistant, a financial intelligent agent responds to customer inquiries by generating introductory text for a particular financial product as output content. Figure 1 This demonstrates the overall processing flow of the system in this scenario. After receiving the content to be output and its generation context, the system first uses a content type recognition module to identify the business type to which the output content belongs. In this scenario, based on key semantic features in the content, the recognition module determines that the output content involves both product descriptions and risk warnings, and then passes the recognition result to downstream modules.

[0024] The evidence matching module retrieves matching output evidence from the knowledge base, business rule base, process rule base, product rule base, and manual confirmation record base, based on the identified business type. For key fields in the output content involving "expected annualized return," "risk level," "applicable customer scope," and "redemption rules," the evidence matching module matches them against the corresponding product description clauses, risk warning rule templates, suitability management rules, and product redemption process specifications. If the output content contains statements such as "stable returns," and the evidence matching module cannot find clear evidence supporting this statement in the product description or related announcements, then the statement is marked as lacking source.

[0025] The sensitive field detection module performs specific checks on financially sensitive fields in the output content. Based on a pre-set sensitive field library and semantic recognition rules, this module scans each field related to credit limits, approval results, return promises, fee descriptions, risk warnings, and customer rights. In this scenario, the detection module identified "return" and "risk level" as sensitive fields. Furthermore, based on their contextual semantics, it determined that the "return" field involves an expected description of future returns, thus constituting high-risk sensitive content.

[0026] The boundary marking module integrates the matching results from the evidence matching module and the detection results from the sensitive field detection module to mark the responsibility boundaries of the output content. Specifically, when all key information in the output content matches a valid source and no sensitive fields are hit, it is marked as directly output; when the output content hits a preset risk warning rule or suitability management rule, it is marked as requiring supplementary information; when at least one key piece of information in the output content lacks a source of evidence, it is marked as requiring rewriting; when the output content involves business advice, product recommendations, or result confirmation that requires manual judgment, it is marked as requiring manual confirmation; when the output content contains over-promises, misleading statements, or information that violates compliance red lines, it is marked as prohibiting output. In this scenario, because the statement "stable returns" lacks evidence, the boundary marking module marks the statement containing this statement as requiring rewriting and automatically associates it with a risk warning template to supplement the suitability warning. In addition, because the customer inquiry implicitly includes a request for purchase advice, the boundary marking module marks the response content involving purchase decisions as requiring manual confirmation and explicitly states that the final service is subject to the confirmation results of licensed personnel and the bank.

[0027] The rewriting and interception modules execute corresponding actions based on the boundary marking results. For content marked as requiring rewriting, the rewriting unit automatically corrects it according to a preset rewriting template or rules, inserting risk warning statements into the original description and adding the source of the reference, while retaining a copy of the original output content. For content marked as requiring manual confirmation, the rewriting unit does not perform automatic correction, but instead adds a manual confirmation placeholder, and the manual confirmation transfer unit transfers the content to be confirmed to the manual confirmation channel provided by the manual confirmation module. For content marked as prohibited from output, the interception unit directly blocks its output and triggers an alarm. All content before and after rewriting, along with their corresponding processing rules, are recorded.

[0028] The audit log module generates a complete audit log of the output content. This log includes at least evidence source information, rule matching information, boundary marker information, processing action information, and manual confirmation status information. The audit log, the original text of the output content, the rewritten text, and the evidence matching results form a closed-loop evidence chain record in an associated storage manner, which can be used for subsequent compliance audits, complaint handling, and post-event review.

[0029] The system also includes a feedback optimization interface. This interface receives execution feedback from external input, manual confirmation results from the manual confirmation module, anomaly handling status, and changes in business metrics. This data is then fed back to the evidence matching module and the boundary marking module to update the knowledge base and manual confirmation record database, and / or update the rules in the business rule base, process rule base, and product rule base, or adjust the marking strategy. For example, when the manual confirmation result differs from the system's initial marking of the responsibility boundary, the system feeds this discrepancy information back to the boundary marking module for reference in marking similar output content in the future. When changes in business metrics reflect a continuous decline in the compliance pass rate of a certain type of output content, the system adjusts the sensitive field detection threshold or evidence matching sufficiency requirements for the corresponding business type accordingly.

[0030] In actual deployment, the system can be deployed privately, via a dedicated cloud, or in a controlled zone, depending on the internal system requirements of the financial institution. The system interacts with banking business systems, customer management systems, process systems, knowledge base systems, intelligent agent platforms, or digital human platforms through controlled interfaces. These controlled interfaces are configured with identity authentication and permission verification mechanisms, and each interface call records the caller's identity, call time, and call parameters. When dealing with sensitive matters such as customer information, approval results, credit limits, risk pricing, and the suitability of financial products, the system enforces manual confirmation or result referencing boundaries, ensuring that the final business conclusion is based on the results of the financial institution's authorized system or manual confirmation, and that the system output is for reference only.

[0031] The system establishes a structured object model for financial business scenarios, unifying the management of elements such as roles, processes, rules, knowledge, tasks, interfaces, permissions, and evidence records. Rule objects are stored in at least one of the business rule base, process rule base, and product rule base, while evidence objects are stored in at least one of the knowledge base and manual confirmation record base. Rule objects are used to define compliance judgment conditions, responsibility boundary thresholds, or processing logic, while evidence objects provide the factual sources supporting atomic assertions in the output content. In this model, each business scenario is abstracted as a business object, the generation and processing of each output content is abstracted as a task object, various compliance requirements are abstracted as rule objects, various evidence sources are abstracted as evidence objects, and execution results are abstracted as execution result objects. These objects form a unified data relationship network through pre-defined associations, enabling the evidence matching module and boundary marking module to perform addressing and rule-based judgments within a unified semantic framework.

[0032] The system supports the reuse of the core technology framework in different financial business scenarios and provides differentiated adaptation through business templates, rule templates, and job templates. In banking wealth management scenarios, consumer finance approval assistance scenarios, and auto finance post-loan reminder scenarios, the system can load corresponding sensitive field configurations, product rule bases, and process rule templates, and adjust the judgment thresholds for each responsibility boundary level without changing the core processing flow. In different embodiments, the intelligent agent, rule engine, knowledge base, process engine, and data governance module can be implemented independently or in an integrated manner. The rule base can be composed of business rules, compliance rules, risk rules, product rules, and process rules.

[0033] Example 2: In the process of constructing the evidence chain for the output content of a financial intelligent agent, the relationship between each output assertion and its corresponding evidence source is not merely a binary "yes or no" relationship. Instead, it should exhibit varying degrees of association strength across multiple dimensions, including the type of evidence source, its authority, its temporal validity, and its semantic coverage. This embodiment proposes a quantitative evaluation method for multidimensional evidence association strength. By constructing a vectorized feature representation of the evidence source and calculating the multidimensional similarity and confidence levels between the output assertion and each evidence source, the methods are aggregated into an evidence chain integrity score. This score reflects the credibility of the output assertion at the evidentiary support level, providing a quantitative basis for subsequent differentiated labeling of liability boundaries. Figure 2 The process of evaluating this multidimensional chain of evidence is shown.

[0034] After the system receives the content to be output by the financial intelligent agent, the evidence matching module first performs assertion-level decomposition on the output content, breaking it down into several independent atomic assertions. Each atomic assertion corresponds to an indivisible factual statement or conclusive statement. For each atomic assertion, the evidence matching module retrieves candidate evidence sources from the knowledge base, business rule base, process rule base, product rule base, and manual confirmation record base to form a candidate evidence set.

[0035] Subsequently, the system extracts the following multi-dimensional feature vectors for each candidate evidence source: source type feature, used to distinguish whether the evidence originates from a structured knowledge base, unstructured document, rule base, or manually verified record; source authority feature, determined based on the pre-marked authority level of the evidence source within the financial institution's internal knowledge management system; time validity feature, calculated based on the attenuation coefficient between the generation or update time of the evidence source and the generation time of the current output content; and semantic coverage feature, determined by calculating the cosine similarity of the semantic vectors between the atomic assertions and the evidence source text. The system then normalizes these multi-dimensional features to form the feature vector of the evidence source.

[0036] For each atomic assertion and each source of evidence in its candidate evidence set, the system calculates a multidimensional association strength value. This association strength value comprehensively considers four dimensions: source type matching degree, source authority level, time decay factor, and semantic similarity, and obtains a normalized association strength score through weighted summation. The system selects the source of evidence with the highest association strength in the candidate evidence set for each atomic assertion as the main evidence for that assertion, and uses the highest association strength value as the evidence support level for that assertion.

[0037] Based on this, the system comprehensively aggregates the evidentiary support of all atomic assertions contained in the output content and calculates the overall evidence chain integrity score of the output content. This score adopts a weighted average method, where the weight of each atomic assertion is pre-set according to the semantic importance of the assertion in the output content, and assertions involving key and sensitive content such as benefit commitments, approval conclusions, and risk warnings are given higher weights.

[0038] When calculating the evidence chain integrity score, the system uses the following mathematical expression to quantify the overall evidentiary support of the output content:

[0039] In the formula, The score for the completeness of the chain of evidence in the output content is a normalized value between 0 and 1, used to quantitatively assess the sufficiency of the overall evidence supporting the output content. This represents the total number of atomic assertions obtained after the output content is split at the assertion level. Each atomic assertion corresponds to an indivisible factual statement or conclusive statement. For the first The semantic importance weight of each atomic assertion is pre-set based on the semantic importance of that atomic assertion in the output content. For the first Each atomic assertion corresponds to a set of candidate evidence sources, which includes all matching candidate evidence sources retrieved from the knowledge base, business rule base, process rule base, product rule base, and manual confirmation record base. The preset weight coefficients for the semantic similarity dimension reflect the relative importance of semantic coverage in the evaluation of association strength. The preset weighting coefficients for the source authority dimension reflect the relative importance of the authority level of the evidence source in the evaluation of the correlation strength. The preset weighting coefficients for the time validity dimension reflect the relative importance of the timeliness of evidence in the evaluation of the strength of association. The preset weighting coefficients for the type matching dimension reflect the relative importance of evidence type suitability in the association strength evaluation; preferably, The value range is 0.35~0.55. The value range is 0.20~0.35. The value range is 0.05 to 0.20. And the sum of the four is 1. For the first The atom assertion and the first The semantic vector cosine similarity between evidence sources is used to measure the degree of semantic correlation between the two sources, and the value ranges from 0 to 1. This is a quantitative value representing the authority of the evidence source, determined based on the pre-marked authority level of the evidence source within the financial institution's knowledge management system. The higher the level, the closer the quantitative value is to 1. The decay value of the time validity of the evidence source is calculated using the following formula: ,in, The preset time decay coefficient, For the source of evidence The time difference between the generation or update time of the current output content and the generation time of the current output content. The type matching value of the evidence source is used to characterize the degree of matching between the type attribute of the evidence source and the type of evidence expected by the current atomic assertion. The value ranges from 0 to 1.

[0040] The system compares the calculated evidence chain integrity score with a preset integrity threshold, which includes a first threshold and a second threshold, with the first threshold being greater than the second threshold. If the score is higher than the first threshold, the evidence chain of the output content is determined to be complete, and it can directly proceed to the subsequent sensitive field detection and boundary marking process. If the score is between the first threshold and the second threshold, the evidence chain is determined to be partially complete, and the system automatically triggers a supplementary retrieval and rematching process for assertions with low evidence support. If the score is lower than the second threshold, the evidence chain is determined to be severely incomplete, and the system directly marks the output content as needing to be rewritten or prohibiting output, blocking it from entering the subsequent output process.

[0041] The calculation process for the above-mentioned multidimensional correlation strength and evidence chain integrity scores is recorded in the audit record module as a traceable basis for subsequent compliance review and liability determination.

[0042] Using conventional binary evidence matching as a benchmark, this method employs the same retrieval tool. If a fragment matching the keywords of the output content exists in the search results, the evidence is deemed sufficient; otherwise, it is deemed insufficient. The test uses the same financial agent output sample set, consisting of 500 real historical outputs from a financial institution's wealth management client manager assistant scenario, after anonymization. These samples cover sensitive fields such as return promises, risk levels, and fee descriptions. The sufficiency of the samples was independently labeled by three financial compliance experts; in case of disagreement, the sufficiency was determined through arbitration, with the arbitration result serving as the gold standard. The test was conducted under the same knowledge base version, retrieval model, top-k parameters, and operating environment. The evaluation metric is the insufficient evidence identification rate, i.e., the proportion of correctly identified insufficient evidence samples out of the total number of actual insufficient evidence samples in the gold standard. Comparative experiments show that the conventional binary matching method has an identification rate of approximately 62% for insufficient evidence, while the multi-dimensional evidence association strength quantitative evaluation method proposed in this embodiment achieves an identification rate of over 89%. Regarding the discriminative power of evidence chain integrity scoring, conventional methods only output binary results and cannot quantify the sufficiency of evidence in a graded manner. This embodiment uses multi-dimensional weighted evaluation at the atomic assertion level to make the mean difference in evidence chain integrity scores between high-risk and low-risk output content about 0.4, providing a quantitative decision-making basis for the differentiated marking of subsequent responsibility boundaries.

[0043] Example 3: The determination of the responsibility boundary of the output content of a financial intelligent agent depends not only on the evidence matching results and sensitive field detection results at a single point in time, but also on multiple factors such as the dynamic changes in financial business scenarios, the continuous updates of regulatory rules, and compliance feedback from historical output behaviors. This embodiment proposes an adaptive responsibility boundary marking mechanism based on a dynamic compliance boundary threshold. By introducing historical compliance feedback data as an adjustment factor, the threshold for determining the responsibility boundary can be adaptively adjusted according to the compliance events accumulated during business operations, the results of manual confirmation, and changes in regulatory requirements. This mechanism overcomes the shortcomings of existing solutions where the responsibility boundary marking rules are static and fixed, and cannot dynamically evolve with business scenarios, enabling the system's compliance control capabilities to have continuous optimization and adaptive evolution characteristics.

[0044] During the initial deployment phase of the system, the boundary marking module sets initial responsibility boundary determination thresholds for different types of output content based on the compliance rule base preset by financial institutions. These initial thresholds include: evidence matching sufficiency threshold, used to determine whether the basis for the output content is sufficient; sensitive field risk threshold, used to determine the risk level of sensitive fields in the output content; and comprehensive determination threshold, used to comprehensively determine, based on the above two, which responsibility boundary level the output content should be classified into: directly output, requiring supplementary prompts, requiring rewriting, requiring manual confirmation, or prohibited from output.

[0045] During actual business operations, the system continuously collects the following historical compliance feedback data through the feedback optimization interface: confirmation or correction results of human reviewers regarding the marking of responsibility boundaries for output content; compliance review conclusions of regulatory agencies or internal compliance departments regarding historical output content; customer feedback records involving complaints or questions about the accuracy or appropriateness of output content; and fluctuations in output content quality reflected by changes in business indicators. The system maps the above feedback data into a compliance feedback score according to preset classification rules. This score reflects the accuracy and compliance of historical output content in terms of marking of responsibility boundaries.

[0046] The system uses a sliding time window to statistically aggregate historical compliance feedback scores, calculating the average compliance feedback score and score volatility variance within the current time window. Based on these statistics, the system dynamically adjusts the comprehensive threshold for determining responsibility boundaries: when historical compliance feedback scores consistently exceed a preset benchmark level, it indicates that the system's current responsibility boundary marking strategy has high accuracy and compliance, and the system appropriately relaxes the judgment threshold, allowing more low-risk output content to enter the direct output channel with lower review costs; when historical compliance feedback scores consistently fall below the benchmark level or the score volatility variance increases significantly, it indicates that the system's current responsibility boundary marking strategy may be biased or that the business environment has changed significantly, and the system automatically tightens the judgment threshold, classifying more output content into responsibility boundaries requiring manual confirmation or rewriting at a higher review level to reduce compliance risks.

[0047] When dynamically adjusting the responsibility boundary determination threshold, the system uses the following mathematical expression to calculate the comprehensive determination threshold within the current time window:

[0048] In the formula, This is the comprehensive judgment threshold within the current sliding time window, used for the final determination of the responsibility boundary. The baseline threshold, set based on the financial institution's pre-defined compliance rule base during the initial deployment of the system, serves as the starting reference point for dynamic adjustment of the threshold. This is a preset adaptive adjustment coefficient used to control the sensitivity of the comprehensive judgment threshold to changes in historical compliance feedback data. This coefficient is pre-configured during system initialization based on the compliance risk level of the business scenario. This represents the total number of compliance feedback data samples collected within the current sliding time window. The length of the sliding time window is preset based on the historical data accumulation period of the business scenario. For the first The compliance feedback score for each compliance feedback data sample is obtained by the system by mapping feedback data such as manual review confirmation results, regulatory review conclusions, customer feedback opinions, and changes in business indicators to a unified scoring scale. The preset benchmark compliance feedback level represents the average historical feedback score expected under a good compliance management state recognized by financial institutions, and serves as a reference benchmark for determining the degree of deviation of the current feedback level. This represents the statistical variance of all compliance feedback scores within the current sliding time window, used to quantify the dispersion and fluctuation of feedback data within the current time window. The preset benchmark variance reference value represents the normal fluctuation range allowed for feedback scores when compliance control is in a stable state, and serves as a reference benchmark for determining whether the current fluctuation is significant.

[0049] The aforementioned dynamic threshold adjustment process employs a smooth update mechanism, meaning that the magnitude of each adjustment is limited to a preset maximum step size, preventing drastic threshold fluctuations caused by a single abnormal feedback data. The adjusted threshold immediately takes effect in the responsibility boundary determination process of subsequent output content, and the adjustment record is written to the audit log module, forming a traceable threshold change history.

[0050] The system also supports maintaining independent dynamic threshold parameters for different financial business scenarios, enabling different business scenarios to independently and adaptively adjust based on their respective compliance risk characteristics and historical feedback data, thus avoiding misjudgments caused by threshold coupling across scenarios.

[0051] The aforementioned adaptive adjustment process of the dynamic compliance boundary threshold runs continuously over a preset time period, enabling the system's ability to mark the responsibility boundary to be continuously optimized as the business operation time extends, gradually approaching the optimal judgment strategy for specific financial business scenarios.

[0052] Using a static, fixed threshold for determining responsibility boundaries as a benchmark, the system was run continuously for six months under the same business scenario and testing conditions for a financial account manager assistant. The accuracy rate of responsibility boundary marking confirmed by manual review was used as the evaluation metric, calculated as the proportion of correctly marked samples in each month's manual review sample to the total number of manually reviewed samples. The comparative experiment showed that the system using static thresholds had an initial marking accuracy of approximately 84%, which decreased to approximately 76% by the sixth month as business rules were updated and regulatory requirements changed. The dynamic adaptive threshold adjustment mechanism proposed in this embodiment had an initial marking accuracy of approximately 83%, which increased to approximately 89% by the third month as historical compliance feedback data accumulated, and stabilized above 91% by the sixth month. Regarding threshold adjustment stability, static thresholds suffer from systematic deviations due to their inability to perceive changes in the business environment. The fluctuation suppression factor introduced in this embodiment narrows the threshold adjustment range by approximately 60% when feedback data fluctuates drastically, effectively avoiding threshold oscillations caused by short-term data disturbances and ensuring stable judgment performance throughout the entire operating cycle.

Claims

1. A system for verifying the chain of evidence and marking the boundary of responsibility in the output content of a financial intelligent agent, characterized in that, include: The content type recognition module is used to receive the content to be output by the financial intelligent agent and its generation context, and to identify the business type of the output content; An evidence matching module is used to match output evidence for the output content from at least one data source according to the business type; The sensitive field detection module is used to perform specific detection on financially sensitive fields in the output content; A boundary marking module is used to mark the responsibility boundary of the output content based on the matching result of the evidence matching module and the detection result of the sensitive field detection module; The rewrite and intercept module is used to perform direct output, output with supplementary prompts, rewrite, transfer to manual confirmation, or interception operations on the output content according to the responsibility boundary. The manual confirmation module is used to provide a manual confirmation channel, receive the content to be confirmed transferred by the rewrite and interception module, and generate manual confirmation status information; The audit log module is used to generate audit logs for the output content; The responsibility boundaries marked by the boundary marking module include five levels: direct output, requiring supplementary prompts, requiring rewriting, requiring manual confirmation, and prohibited output. The audit records include at least evidence source information, rule hit information, boundary marking information, processing action information, and manual confirmation status information.

2. The system according to claim 1, characterized in that: The data source includes a knowledge base, a business rule base, a process rule base, a product rule base, and a manual confirmation record base. The evidence matching module compares the key information of the output content with the structured objects stored in the data source to match the corresponding output basis. The structured objects form a unified financial business object model through preset association relationships.

3. The system according to claim 1, characterized in that: The sensitive field detection module detects each financial sensitive field in the output content based on a preset sensitive field library and semantic recognition rules, and marks the hit sensitive fields and their attributes. The boundary marking module determines the level of the responsibility boundary based on the degree of matching between the attributes of the sensitive field and the preset compliance boundary rules.

4. The system according to claim 1, characterized in that: The boundary marking module determines the responsibility boundary as one of the five levels according to the matching result and the detection result, in accordance with the preset compliance boundary rules. When the matching result or the detection result meets the preset conditions for manual intervention, the boundary marking module marks the responsibility boundary as requiring manual confirmation. The rewriting and interception module transfers the content to be confirmed to the manual confirmation module, the manual confirmation module generates manual confirmation status information, and the audit recording module records the manual confirmation status information.

5. The system according to claim 1, characterized in that: The rewriting and interception module includes a rewriting unit, an interception unit, and a manual confirmation and transfer unit. The rewriting unit is used to automatically correct output content marked as needing rewriting according to a preset rewriting template or rules, and add references or risk warnings. The interception unit is used to block output content marked as prohibited and trigger an alarm. The manual confirmation and transfer unit is used to transfer output content marked as needing manual confirmation to the manual confirmation module. When the rewriting unit corrects the output content, it records the content before and after correction and the corresponding rewriting rules in the audit log.

6. The system according to claim 1, characterized in that: The system also includes a feedback optimization interface, which is used to receive execution feedback from external input, manual confirmation results from the manual confirmation module, abnormal handling status or business indicator change data, and feed the data back to the evidence matching module and the boundary marking module to update the rules or basis in the data source; the system adjusts the matching strategy of the evidence matching module and the marking threshold of the boundary marking module according to the execution feedback.

7. The system according to claim 1, characterized in that: The content type identification module identifies business types including factual statements, business suggestions, customer tips, risk warnings, product descriptions, and operation guidelines. The evidence matching module matches corresponding evidence templates and verification rules from the data source according to different business types. The evidence templates define the evidence fields to be matched and their source requirements, and the verification rules define compliance check items.

8. The system according to claim 1, characterized in that: The evidence matching module breaks down the output content into several atomic assertions. For each atomic assertion, it retrieves candidate evidence sources from the data source and evaluates the correlation strength between each candidate evidence source and the corresponding atomic assertion from multiple dimensions. The candidate evidence source whose correlation strength meets the preset conditions is selected as the main evidence for that atomic assertion. The evidence matching module performs an aggregate evaluation based on the evidence support of all atomic assertions to generate an evidence chain integrity evaluation result for the output content. Based on the comparison between the evaluation result and the preset integrity threshold, it determines the path for the output content to enter the subsequent processing flow.

9. The system according to claim 6, characterized in that: The boundary marking module further includes an adaptive threshold adjustment unit, which is used to obtain historical compliance feedback data through the feedback optimization interface and dynamically adjust the comprehensive judgment threshold used by the boundary marking module when determining the responsibility boundary based on the historical compliance feedback data; when adjusting the comprehensive judgment threshold, the adaptive threshold adjustment unit suppresses the adjustment range based on the dispersion of the historical compliance feedback data; the boundary marking module uses the adjusted comprehensive judgment threshold to mark the responsibility boundary of subsequent output content.

10. A method for verifying the chain of evidence and marking the boundary of responsibility in the output content of a financial intelligent agent, characterized in that, The method is implemented based on the system described in any one of claims 1 to 9, and includes: Receive the content to be output by the financial intelligent agent and its generation context; Identify the business type of the output content; Based on the business type, match the output criteria for the output content from at least one data source; Special checks are performed on the financially sensitive fields in the output content; Based on the matched output criteria and the detection results of the financial sensitive fields, the responsibility boundaries of the output content are marked. The responsibility boundaries include direct output, supplementary prompts required, rewriting required, manual confirmation required, and output prohibited. Based on the defined responsibility boundaries, the output content may be processed as follows: direct output, output with supplementary prompts, rewritten, transferred to manual confirmation, or intercepted. An audit log is generated for the output content. The audit log includes at least evidence source information, rule hit information, boundary marker information, processing action information, and manual confirmation status information.

Images