Security auditing and forensics method and system for tool invocation type AI agent

Abstract

The application relates to the technical field of artificial intelligence security, and discloses a security auditing and evidence collection method and system for tool calling type AI Agent, wherein a security auditing node is arranged between a tool calling type AI Agent and an underlying execution environment, and high-risk actions such as file access, tool calling, command execution and the like are uniformly taken over; task identification, Skill identification, Tool identification, event identification and parent event identification are established, and task execution whole-process auditing information is associated; in combination with task intention normalization, semantic guardrail arrangement, multi-dimensional capability boundary modeling, request standardization and strategy evaluation, pre-constraint and risk judgment are performed on Agent tool calling behavior; structured runtime events are collected, an execution track graph is constructed, and risk paths such as out-of-bound access and unauthorized calling are identified; a structured evidence package is generated, integrity is solidified through chain abstraction, and checkable auditing and evidence collection results are output, so that the whole process of Agent tool calling is observable, associable, determinable, interpretable and reviewable.

Description

Technical Field

[0001] This invention relates to the field of artificial intelligence security technology, specifically to a security auditing and forensics method and system for tool-invoking AI agents. Background Technology

[0002] As large-scale models become more powerful, AI agents are evolving from question-and-answer systems into execution-oriented systems capable of autonomously planning tasks, invoking tools, and operating in real-world digital environments. These systems are widely used in scenarios such as office work, operations and maintenance, data processing, and business orchestration. Taking tool-invoking agents like OpenClaw as an example, because these agents can read files, access networks, invoke plugins, or execute commands, they are susceptible to vulnerabilities such as injection vulnerabilities, context pollution, malicious skill access, dependency poisoning, or permission bypassing. This can lead to unauthorized reads, sensitive configuration leaks, unauthorized tool invocations, high-risk command execution, and external data connections, extending security risks from model output risks to real-world system execution risks.

[0003] In tool-invoking agent scenarios, security auditing faces multiple challenges: On the one hand, agent execution is multi-stage, dynamic, and context-driven. The security of a single tool invocation depends not only on the tool itself but also on the current task intent, process state, invoking entity, resource type, and the relationship between preceding and subsequent events. On the other hand, entry points for behaviors such as file access, network requests, command execution, and tool invocation are scattered. Without a unified, controlled proxy layer, security systems struggle to perform stable pre-checks before actions are actually implemented. Furthermore, traditional logs often only record isolated events, failing to reconstruct the complete "task—Skill—Tool—resource" call chain, making post-event analysis difficult to explain the source and propagation path of risks. Many dangerous behaviors may superficially conform to basic permission rules, but are unnecessary or unreasonable under the specific task semantics. For example, in document summarization tasks, there may be environment variable reading, directory scanning, or external upload behaviors. Traditional blacklist-based or static permission-based solutions struggle to identify such semantic deviation risks in a timely manner.

[0004] Existing technologies for agent security or large-scale application security mainly include input filtering, prompt word security detection, static code scanning, dependency vulnerability detection, sandbox isolation, permission approval, and post-event log auditing. While these solutions can be effective in certain areas, they still have significant shortcomings:

[0005] Firstly, input filtering or output auditing mainly targets text interaction and is difficult to cover real actions such as file, command, network, and tool execution.

[0006] Secondly, static scanning can discover some known vulnerabilities or overtly dangerous functions, but it is difficult to identify runtime context changes, toolchain combination risks, and task semantic deviations.

[0007] Third, while traditional sandboxes can provide environment-level isolation, they lack fine-grained action-level interpretation capabilities in scenarios involving multiple skills, tools, and tasks.

[0008] Fourth, ordinary log systems typically lack unified identification and causal correlation mechanisms, making it difficult to encapsulate scattered events into a verifiable and verifiable chain of evidence;

[0009] Fifth, existing solutions often separate semantic guardrails from underlying runtime auditing, resulting in the system being able to determine "whether access to a resource is allowed" but not being able to fully determine "whether the current task should access the resource".

[0010] Therefore, a security auditing and forensics method for AI agents that invoke tools is needed. This method involves setting up security monitoring components in the agent's task execution chain, establishing task-level context identifiers and multi-dimensional capability boundaries, and combining task intent unification, semantic guardrail orchestration, unified request normalization, policy evaluation, runtime event collection, risk path association, and audit evidence solidification. This enables pre-constraints, dynamic monitoring, risk assessment, and verifiable forensics throughout the entire process of agent tool invocation, solving problems such as unobservable tool invocation process, difficulty in identifying semantic deviations, difficulty in associating combined risks, unreliable audit evidence, and complex deployment and access. Summary of the Invention

[0011] The purpose of this invention is to address the problems existing in tool-invoking AI Agent scenarios, such as insufficient supervision of the tool invoking process, difficulty in identifying semantic intent deviations, lack of unified control over high-risk actions, difficulty in correlating runtime events, and lack of integrity verification of audit evidence. It provides a security auditing and forensics method and system for tool-invoking AI Agents, achieving observability, correlability, determineability, explainability, and verifiability of the entire Agent task execution process. The technical solution is as follows:

[0012] A security auditing and forensics method for tool-invoking AI agents includes the following steps:

[0013] S1: Deploy Agent Security Audit Node: Deploy a security audit node between the tool-invoking AI Agent and the underlying execution environment, configure the task access endpoint, multiple proxy endpoints for proxying high-risk actions, and audit data storage endpoint, so that the security audit node has the conditions to receive task context, proxy high-risk actions, collect runtime events, and output audit evidence;

[0014] S2: Initialize the task context and establish a unified identification mechanism: When the Agent task enters, a unique task identifier is generated, the task context object is initialized, and an associated identifier is reserved for subsequent execution stages, so that the behavior of each stage under the same task can be continuously tracked through the unified context.

[0015] S3: Task Intent Unification and Semantic Guardrail Orchestration: Parse the user's natural language requests and dialogue context, unify them into stable task intent categories, load the corresponding state constraint rules according to the task intent, and determine whether the candidate call is consistent with the current task goal and state path before the tool is called.

[0016] S4: Establish multi-dimensional capability boundaries and issue policy rules: Establish an Agent security constraint model from three dimensions: resource boundary, capability boundary and trust boundary, and compile the above constraints into a computable set of policy rules;

[0017] S5: Standardize high-risk action requests and perform pre-policy evaluation: Unify the various high-risk actions initiated by the Agent into a standard request object, standardize the resource identifiers in the request, bind the task context, evaluate the policy, and obtain the disposal result.

[0018] S6: Controlled execution and collection of structured runtime events: For requests that pass the pre-check, the corresponding controlled execution adapter is invoked to complete the actual action; for rejected requests, a blocking event is generated; predefined audit fields are collected before and after execution to form structured audit events;

[0019] S7: Construct execution trajectory and identify risk paths: Based on structured audit events under the same task identifier, construct execution trajectory diagrams according to calling relationships and time sequence relationships, jointly determine single event risks and multi-event combination risks, and identify predefined risk path types;

[0020] S8: Generate audit evidence chain and evidence collection results: Gather key events, hit rules, risk paths and handling results in chronological order to generate a structured evidence package, and use hash chain or message authentication code to solidify the integrity of the key event sequence, and output audit evidence collection results.

[0021] A security auditing and forensics system for tool-invoking AI agents includes a security audit node deployed between the agent entity and the underlying resource environment. The security audit node includes:

[0022] The deployment and access module is used to deploy security audit nodes between the Agent entity and the underlying execution environment, and configure a unified takeover entry point for high-risk actions;

[0023] The context identifier management module is used to initialize the task context structure, assign a unique task identifier to each task execution, and establish the association between each execution stage;

[0024] The semantic guardrail orchestration module is used to perform task intent unification, state flow constraints, and semantic consistency judgment of tool calls;

[0025] The boundary modeling and policy distribution module is used to establish resource boundaries, capability boundaries, and trust boundaries, and generate a set of policy rules.

[0026] The request normalization and strategy evaluation module is used to convert high-risk actions into standard request objects and perform pre-policy evaluation based on the set of policy rules.

[0027] The runtime event acquisition module is used to collect structured audit events before and after the execution of an action, and to establish event relationships.

[0028] The risk correlation analysis module is used to construct execution trajectory diagrams and identify single-point risks and combined risk paths;

[0029] The evidence chain and evidence collection module is used to generate structured evidence packages and output the audit evidence collection results after integrity verification.

[0030] Compared with the prior art, the present invention has at least the following beneficial effects:

[0031] This invention deploys a security audit node between the Agent and the underlying execution environment to uniformly manage high-risk actions such as file access, tool invocation, command execution, network requests, and environment variable reading. This eliminates the need to reconstruct the underlying model or rewrite existing task scheduling logic, resulting in good compatibility and implementability. A context propagation mechanism is established using task identifiers, Skill identifiers, Tool identifiers, event identifiers, and parent event identifiers to achieve link tracing from "task—Skill—Tool—resource—action—result." Joint modeling of resource boundaries, capability boundaries, and trust boundaries forms executable, comparable, and auditable policy rules. Furthermore, task intent unification, state flow constraints, and invocation intent... Figure 1 Consistency assessment identifies actions that appear legitimate but exceed semantic authority; by using execution trajectory diagrams, time window joint judgments, and state machine joint judgments, it discovers combined risk paths such as "sensitive reading followed by external connection"; and by solidifying structured evidence packages and chained summaries, it enhances the authenticity, completeness, and verifiability of audit results. Attached Figure Description

[0032] Figure 1 This is a schematic diagram of the overall process of the method of the present invention, corresponding to S1-S8.

[0033] Figure 2The flowchart for multidimensional capability boundary modeling and policy evaluation corresponds to S5.

[0034] Figure 3 The flowchart for trajectory association and risk path determination corresponds to S7.

[0035] Figure 4 This is a flowchart for the solidification of audit evidence and the output of evidence collection results, corresponding to S8. Detailed Implementation

[0036] The technical method of the present invention will be further described below with reference to the embodiments and accompanying drawings.

[0037] An embodiment of the present invention: a security auditing and forensics method for tool-invoking AI agents, the flowchart of which is shown below. Figure 1 As shown, the specific steps include:

[0038] S1: Deploy the Agent security audit node. Deploy a security audit node between the tool-invoking AI Agent and the underlying execution environment, configure the task access endpoint, multiple proxy endpoints for proxying high-risk actions, and the audit data storage endpoint, so that the security audit node has the ability to receive task context, proxy high-risk actions, collect runtime events, and output audit evidence.

[0039] A security audit node is deployed between the tool-invoking AI Agent and the underlying execution environment, embedding it as a controlled proxy layer into the Agent's critical execution path. The security audit node can be deployed as a standalone service, pluggable middleware, communication gateway, edge security component, or local daemon, and interacts with the Agent through standard interfaces. The underlying execution environment includes at least one or more of the following: file system, command execution environment, network access environment, third-party tool interface, environment variables, and business resource interface.

[0040] The security audit node configuration should include at least the following endpoints: a task access endpoint for receiving Agent task requests and context; a tool call proxy endpoint for proxying Tool or Skill calls; a file access proxy endpoint for proxying file reading and writing; a network request proxy endpoint for proxying HTTP requests or external API access; a command execution proxy endpoint for proxying system command calls; an environment variable access proxy endpoint for proxying environment variable reading; and an audit data storage endpoint for storing runtime events, risk assessment results, and evidence packages.

[0041] S2: Initialize the task context and establish a unified identification mechanism. When the Agent task enters, a unique task identifier is generated, the task context object is initialized, and associated identifiers are reserved for subsequent execution stages, so that the behavior of each stage under the same task can be continuously tracked through the unified context.

[0042] Initialize the Agent task audit structure, define the task context object and audit event data structure, assign a unique task identifier to each task execution, and establish the association relationship between the task identifier, Skill identifier, Tool identifier, event identifier, and parent event identifier for unified association of subsequent audit information. Specifically, this includes the following steps:

[0043] S21: Initialize the task context data structure.

[0044] The security audit node initializes a task context object, which includes at least a task identifier, a session identifier, a user request summary, a task intent label, a current process status, a set of allowed actions, risk warning information, and a creation timestamp.

[0045] S22: Generate a unique task identifier.

[0046] When the Agent receives a user's natural language request and starts a task, the security audit node assigns a unique task identifier, run_id, to the task execution to distinguish audit data in different task sessions.

[0047] S23: Establish a multi-level execution identification mechanism.

[0048] During task execution, the security audit node assigns corresponding skill_id, tool_id, event_id, and parent_event_id to Skill activation, Tool invocation, and resource access actions, enabling various audit events to be traced back to their respective tasks, calling entities, and parent triggering behaviors.

[0049] S24: Binding context and subsequent execution flow.

[0050] The security audit node writes the task identifier, skill identifier, tool identifier, event identifier, and parent event identifier into a unified context object, and continuously propagates them in subsequent semantic guardrail judgment, request normalization, policy evaluation, event collection, risk analysis, and evidence generation processes.

[0051] S3: Task Intent Normalization and Semantic Guardrail Orchestration. The system parses user natural language requests and dialogue context, normalizes them into stable task intent categories, loads corresponding state constraint rules based on the task intent, and determines whether candidate calls are consistent with the current task goal and state path before tool invocation.

[0052] The security audit node receives task requests from the Agent, analyzes user input and task context, classifies natural language tasks into internally stable task intent types, and configures allowed state transition paths and action sets based on the task type. This is used to determine whether the call conforms to the current task semantics before the tool is invoked. Specifically, this includes the following steps:

[0053] S31: Obtain user request and task context.

[0054] The security audit node obtains the user's original request, recent dialogue history, system configuration, business constraints, and current task context information as input for task intent unification.

[0055] S32: Unify the execution of task intent.

[0056] Based on user requests and context information, tasks are categorized into preset task categories, which include at least one of the following: document summarization, retrieval and question answering, file processing, external call, and high-risk execution.

[0057] S33: Configure state transition rules.

[0058] Based on the unified task intent, configure the corresponding set of process states and the set of allowed actions. For example, for document summarization tasks, the allowed paths include reading workspace files, content analysis, and result generation, while defaulting to network external connections or system command execution paths is not allowed.

[0059] S34: Extract candidate tool invocation intent.

[0060] When the Agent generates a tool call request, the security audit node extracts the call intent of the tool call based on the tool name, parameter structure, target resource type, current status label, and neighboring event information.

[0061] S35: Execute the mission Figure 1 Consistency judgment.

[0062] The task intent is compared with the tool invocation intent to determine whether the tool invocation conforms to the current task objective and process status. If the tool invocation is not in the allowed action set, or the invocation intent is inconsistent with the task intent, the request is marked as a semantic mismatch request.

[0063] S36: Output the semantic guardrail determination result.

[0064] For tool call requests that conform to the task semantics and state constraints, allow them to proceed to the subsequent security gateway for processing; for requests that are semantically mismatched or exceed the process boundaries, trigger alarms, block, degrade execution, or require manual review, and write the judgment result into the audit event.

[0065] S4: Tool Invocation Request Standardization. An Agent security constraint model is established from three dimensions: resource boundary, capability boundary, and trust boundary, and these constraints are compiled into a computable set of policy rules.

[0066] The security audit node receives raw action requests initiated by the Agent, such as file access, tool invocation, command execution, network requests, and environment variable reading, and converts them into a unified standard request object, providing a consistent data foundation for subsequent policy evaluation and audit logging. Specifically, this includes the following steps:

[0067] S41: Receive the original action request.

[0068] The security audit node receives the original action request initiated by the Agent through a unified security takeover entry point. The original action request includes one or more of the following: file path, tool name, calling parameters, command content, network address, environment variable name, and task context information.

[0069] S42: Generate a standard request object.

[0070] The original action requests are uniformly abstracted into standard request objects, which at least include a request body, action type, target resource, parameter summary, task context, and timestamp. The request body corresponds to a Skill, Tool, or Agent execution unit, and the action type includes one of read, write, exec, network, tool_use, and env_read.

[0071] S43: Normalized document access request.

[0072] For file paths, perform absolute path resolution, relative path resolution, duplicate delimiter processing, and symbolic link processing to obtain normalized file resource objects, which are used for subsequent workspace boundary and sensitive resource determination.

[0073] S44: Normalized command execution request.

[0074] For command objects, the main command program, key parameters, and working directory are separated to determine whether the command itself is dangerous, whether the execution directory is out of bounds, and whether the parameters contain high-risk operations.

[0075] S45: Normalized network request.

[0076] For network requests, extract the protocol, host, port, path, request method, and data direction for subsequent determination of outbound target, port type, and request behavior.

[0077] S46: Normalization tool call request.

[0078] For tool requests, extract the tool identifier, operation type, parameter structure, and parameter summary to determine whether the calling tool is registered, whether the parameters conform to the declaration constraints, and whether the calling behavior conforms to the semantics of the current task.

[0079] S5: Multidimensional Capability Boundary Modeling and Policy Evaluation. Various high-risk actions initiated by the Agent are uniformly abstracted into standard request objects. Resource identifiers in the requests are standardized, bound to the task context, and then the policy is evaluated to obtain the disposal result.

[0080] The security audit node evaluates the attribute policies of standard request objects based on resource boundaries, capability boundaries, and trusted boundaries. It determines whether the current request is allowed to execute under the conditions of subject, action, resource, and context, and outputs a decision result: allow, alert, or deny. The flowchart is as follows. Figure 2 As shown, the specific steps include:

[0081] S51: Establish a resource boundary model.

[0082] The security audit node maintains the workspace path set, sensitive resource set, and prohibited access object set, which are used to limit the file directories, configuration files, environment variables, network addresses, and protected resources that the Agent can access.

[0083] S52: Establish a capability boundary model.

[0084] Security audit nodes establish a capability matrix based on Skill, Tool, task type, and action type to limit whether different subjects are allowed to perform actions such as file reading, file writing, network requests, command execution, or tool invocation under different task semantics.

[0085] S53: Establish a trustworthy boundary model.

[0086] The security audit node records the Skill source, Tool whitelist, version information, signature information, registration status, and digest verification value, which are used to determine whether the calling entity and its components are within a trusted range.

[0087] S54: Loading strategy rule set.

[0088] The resource boundary, capability boundary, and trust boundary are converted into executable policy rules, which include at least subject conditions, action conditions, resource conditions, context conditions, and disposal results.

[0089] S55: Execute policy matching.

[0090] The request subject, action type, target resource, and task context in the standard request object are matched with the policy rules to obtain candidate policy results.

[0091] S56: Merger strategy decision results.

[0092] When the same request hits multiple policies, they are merged according to the priority rule of rejection over alarm and alarm over permission to obtain the final policy decision result.

[0093] S57: Execute request processing.

[0094] If the final policy decision is to allow, the request will be forwarded to the controlled execution function; if the result is an alarm, the risk warning will be recorded and the system will choose to continue execution, downgrade execution, or enter the review process according to the configuration; if the result is to reject, the request will be blocked and a blocking audit event will be generated.

[0095] S6: Controlled execution and collection of structured runtime events. For requests that pass pre-checks, the corresponding controlled execution adapter is invoked to complete the actual action; for rejected requests, a blocking event is generated; predefined audit fields are collected before and after execution to form structured audit events.

[0096] The security audit node collects structured audit events before and after the execution of high-risk actions, and continuously propagates task identifiers, skill identifiers, tool identifiers, event identifiers, and parent event identifiers to each execution stage, achieving dynamic recording and chain tracing of the entire Agent tool invocation process. Specifically, this includes the following steps:

[0097] S61: Collect audit events before execution.

[0098] Before the action is actually executed, the security audit node records pre-check events, including at least the task identifier, subject identifier, action type, target resource, parameter summary, policy decision result, semantic guardrail result, risk label, and timestamp.

[0099] S62: Call the controlled execution function.

[0100] For requests that are evaluated using semantic guardrails and policies, the security audit node invokes the corresponding controlled execution function to complete the action. The controlled execution function includes one or more of the following: secure file access function, secure tool invocation function, secure command execution function, secure network request function, and secure environment variable reading function.

[0101] S63: Event for collecting execution results.

[0102] After the action is completed, the security audit node records the execution result event, which includes at least the execution status, return result summary, error message, risk warning and completion timestamp, and associates the execution result event with the corresponding pre-check event.

[0103] S64: Propagate the task context.

[0104] During Skill activation, Tool invocation, and resource access, the security audit node writes run_id, skill_id, tool_id, event_id, and parent_event_id into the context object, enabling child actions to trace back to their parent triggering events.

[0105] S65: Storage structure audit event.

[0106] The security audit node writes the collected structured audit events into the audit data storage module for subsequent execution trajectory construction, risk correlation analysis, and evidence chain generation.

[0107] S7: Construct execution trajectories and identify risk paths. Based on structured audit events under the same task identifier, construct execution trajectory diagrams according to calling relationships and temporal relationships, jointly determine single-event risks and multi-event combination risks, and identify predefined risk path types.

[0108] The security audit node constructs an execution trajectory graph based on a set of audit events under the same task identifier. Combining single-event rules, time window relationships, and state machine transition relationships, it determines single-point and combined risk paths during the Agent tool invocation process. The flowchart is as follows: Figure 3 As shown, the specific steps include:

[0109] S71: Construct the execution trajectory graph.

[0110] The security audit node constructs an execution trajectory diagram based on the set of audit events under the same task identifier, according to the parent-child call relationship, time sequence relationship and resource association relationship. The execution trajectory diagram is used to represent the complete behavioral chain of the agent from task triggering to resource access, tool call and result output.

[0111] S72: Perform single-event risk assessment.

[0112] For each audit event, risk rules are matched to identify single points of risk such as out-of-bounds access to workspaces, reading of sensitive resources, unauthorized tool calls, execution of high-risk commands, abnormal external network connections, reading of environment variables, and semantic mismatch.

[0113] S73: Execution time window joint judgment.

[0114] Under the same task identifier, determine whether multiple events form a risk association within a preset time window. For example, if a network outbound event occurs within a preset time after a sensitive resource read event, it is identified as a risky outbound path following a sensitive read.

[0115] S74: Execute state machine joint judgment.

[0116] Determine whether a risk path has formed based on the task state flow and the order of events. For example, if the task state sequentially passes through the sensitive read state, content processing state, and external connection state, it is identified as a potential data outbound risk path.

[0117] S75: Tracing back the path of key evidence.

[0118] For risk endpoint events, security audit nodes trace back their parent events, predecessor events, and resource-related events based on the execution trajectory map to extract key evidence paths that can explain the source and propagation process of the risk.

[0119] S76: Generate task-level risk conclusions.

[0120] Task-level risk levels are generated based on single-event risk, combined risk paths, subject credibility, semantic consistency, and the number of blocking events, and corresponding handling actions are output. These handling actions include one or more of the following: allowing passage, recording, alerting, blocking, downgrading execution, and manual review.

[0121] S8: Generate audit evidence chain and evidence collection results. Gather key events, hit rules, risk paths, and handling results in chronological order to generate a structured evidence package. Use a hash chain or message authentication code to solidify the integrity of the key event sequence and output the audit evidence collection results.

[0122] Security audit nodes aggregate audit information under the same task identifier according to time sequence and causal relationship, generate structured evidence packages, and solidify key events through chained digests. After integrity verification, the evidence collection results are output. The flowchart is as follows: Figure 4 As shown, the specific steps include:

[0123] S81: Aggregate task-level audit information.

[0124] The security audit node obtains multi-source audit information associated with the same task identifier from the audit data storage module. The multi-source audit information includes at least task context information, standard request objects, pre-check events, execution result events, hit rules, critical risk paths, and handling results.

[0125] S82: Generate a structured evidence package.

[0126] The task metadata, audit event set, hit rule set, key risk path, risk level, handling result, and evidence summary are encapsulated into a structured evidence package. This structured evidence package includes at least the task identifier, task summary, calling entity, tool identifier, resource object, event timeline, risk description, and handling record.

[0127] S83: Perform chained summary solidification.

[0128] Security audit nodes standardize and serialize key events according to the order of their occurrence, and recursively calculate chained summaries based on the summary of the previous event, the content of the current event, and the timestamp, in order to prevent audit events from being deleted, inserted, or tampered with.

[0129] S84: Evidence integrity verification.

[0130] Perform integrity checks on the structured evidence package, including at least digest consistency checks, temporal continuity checks, event association consistency checks, and task identifier consistency checks.

[0131] S85: Generate evidence results.

[0132] When the integrity verification passes, the security audit node generates forensic results. These forensic results are used to characterize at least whether the Agent task exhibits high-risk behavior, the time frame of the risk occurrence, the Skill or Tool involved, the resource objects accessed, the risk rules that were triggered, the actions taken, and evidence summary information.

[0133] S86: Outputs the evidence collection results and supports subsequent calls.

[0134] The security audit node outputs the evidence collection results to the audit report page, the evidence package export module, or an external security analysis system for use in anomaly review, security audit, liability determination, or security teaching scenarios.

[0135] Another embodiment of the present invention: a security auditing and forensics system for tool-invoking AI agents, comprising a security audit node deployed between the agent subject and the underlying execution environment, wherein the security audit node includes:

[0136] 1) Deployment and Access Module: This module is used to deploy a security audit node between the Agent and the underlying execution environment. It provides a unified security takeover entry point for accessing configuration files, calling tools, executing commands, making network requests, and reading environment variables. This enables the security audit node to perform pre-emptive checks, runtime recording, and audit evidence collection for high-risk Agent actions.

[0137] This module is specifically used to: deploy security audit nodes between the Agent body and the underlying execution environment. It can be deployed independently or integrated into the Agent tool invocation layer, runtime agent component, communication gateway or security audit device; configuration file access agent entry, tool invocation agent entry, command execution agent entry, network request agent entry, environment variable access agent entry and audit data storage endpoint.

[0138] 2) Context Identifier Management Module: Used to initialize the task context structure, assign a unique task identifier to each task execution, and establish the association between task identifier, Skill identifier, Tool identifier, event identifier and parent event identifier, so as to realize context propagation and link tracing throughout the entire task execution process.

[0139] This module is specifically used to: initialize a task context object, which includes at least a task identifier, a session identifier, a user request summary, a task intent label, the current process status, a set of allowed actions, and a timestamp; generate a unique task identifier when the task starts; and generate and propagate the corresponding identifier during Skill activation, Tool invocation, and resource access, thereby achieving the independence and traceability of audit information in multi-task, multi-Skill, and multi-tool scenarios.

[0140] 3) Semantic guardrail orchestration module: It is used to normalize the task intent of user requests, configure the state transition rules and allowed action sets corresponding to different task types, and determine whether the tool call intent is consistent with the current task intent and process state before the tool is called.

[0141] This module is specifically used to: map user requests to preset task categories, load the corresponding state stream and set of allowed actions, and perform semantic consistency auditing on the tool name, parameter structure, target resources and current state before the tool is called.

[0142] 4) Boundary modeling and policy distribution module, used to establish resource boundaries, capability boundaries and trust boundaries, and generate policy rule sets.

[0143] 5) Request normalization and policy evaluation module: It is used to receive raw action requests, convert actions such as file access, tool invocation, command execution, network request and environment variable reading into standard request objects, and perform policy evaluation based on resource boundaries, capability boundaries and trusted boundaries, and output the decision result of allow, alarm or deny.

[0144] 6) Runtime event collection module: Used to collect structured audit events before and after the execution of high-risk actions, record the request subject, action type, target resource, parameter summary, execution result, risk warning and timestamp, and establish event association through task identifier and parent event identifier.

[0145] 7) Risk Correlation Analysis Module: This module is used to construct an execution trajectory map based on the audit event set, and to identify risk paths such as out-of-bounds access to the work area, external connections after sensitive reading, and command execution after unauthorized tool calls through single event rule matching, time window joint judgment, and state machine joint judgment, and to generate task-level risk conclusions and handling actions.

[0146] 8) Evidence Chain and Evidence Collection Module: This module is used to associate audit information under the same task identifier according to time sequence and causal relationship, generate structured evidence packages, and output evidence collection results that can be used for security analysis, anomaly review or liability determination through chain-based summary solidification and integrity verification.

[0147] This module is specifically used to: aggregate task metadata, audit events, hit rules, risk paths and handling results under the same task identifier, generate a structured evidence package, solidify its integrity through chained summaries, and output audit evidence results containing task summary, risk level, key call chain, hit rules, handling actions and summary verification values.

[0148] One embodiment of the present invention:

[0149] The execution process of this invention will be explained using "Identification of Sensitive Information External Link Risks in Document Digestion Tasks" as an example.

[0150] After a user submits a task request to the Agent to "generate a summary report for project documents in the work area", the security audit node first receives the task request, generates a corresponding task identifier run_id, and initializes the task context object.

[0151] Subsequently, the semantic guardrail orchestration module normalizes the user request's task intent, identifying the task as a "document summarization task," and loads the corresponding state flow rules. According to these rules, the task is allowed to perform workspace file reading, text processing, and result generation actions, but is not allowed to default to environment variable reading, directory scanning, or external network upload paths.

[0152] When an agent attempts to access a workspace document using a file reading tool, the security audit node normalizes the file access request and generates a standard request object. Since the target file is within the allowed workspace and the call behavior conforms to the semantics of the current task, the policy evaluation result is to allow execution.

[0153] During the document summarization process, the agent further attempts to read system environment variables and initiates external network requests. The security audit node normalizes both the environment variable read request and the network request, and performs policy matching based on resource boundaries, capability boundaries, and trusted boundaries. The environment variable read behavior is not within the allowed action set of the document summarization task and is therefore marked as a semantic mismatch event; the external network request is marked as an abnormal outbound connection event because the target address is not within the trusted whitelist.

[0154] The risk association analysis module further uses time window relationships and state flow relationships to jointly determine the "sensitive information reading - external network request" event, identify the "sensitive reading followed by external connection" combined risk path, and generate corresponding risk levels and blocking actions.

[0155] Subsequently, the security audit node aggregates pre-inspection events, execution result events, hit rules, risk paths, and handling results during the task execution process to generate a structured evidence package, and solidifies the sequence of key events with integrity through chained summaries.

[0156] Finally, the system outputs the corresponding evidence collection results, which include at least the task identifier, the time of risk occurrence, the skill or tool involved, the accessed resource object, the hit rule, the risk path description, the blocking action, and the evidence summary information. These results can be used for anomaly review, security audit, or liability determination scenarios.

[0157] In summary, this invention addresses the challenges of unobservable tool execution processes, difficult semantic deviation identification, difficulty in associating combined risks, lack of integrity verification of audit evidence, and complex deployment and access in tool-invoking AI agent scenarios. It provides a security auditing and evidence collection method and system based on security audit node deployment, unified context identifier propagation, multi-dimensional capability boundary policy evaluation, task semantic guardrail orchestration, runtime event collection, risk path association, and chain-like evidence solidification. Through the above technical solutions, this invention achieves observability, determineability, traceability, explainability, and verifiability of the entire agent tool invocation process without reconstructing the underlying large-scale model or completely rewriting the original agent scheduling logic, effectively improving the security and traceability of tool-invoking agents in real-world deployment environments.

Claims

1. A security auditing and forensics method for tool-invoking AI agents, characterized in that, Includes the following steps: S1: Deploy Agent Security Audit Node: Deploy a security audit node between the tool-invoking AI Agent and the underlying execution environment, configure the task access endpoint, multiple proxy endpoints for proxying high-risk actions, and audit data storage endpoint, so that the security audit node has the conditions to receive task context, proxy high-risk actions, collect runtime events, and output audit evidence; S2: Initialize the task context and establish a unified identification mechanism: When the Agent task enters, a unique task identifier is generated, the task context object is initialized, and an associated identifier is reserved for subsequent execution stages, so that the behavior of each stage under the same task can be continuously tracked through the unified context. S3: Task Intent Unification and Semantic Guardrail Orchestration: Parse the user's natural language requests and dialogue context, unify them into stable task intent categories, load the corresponding state constraint rules according to the task intent, and determine whether the candidate call is consistent with the current task goal and state path before the tool is called. S4: Establish multi-dimensional capability boundaries and issue policy rules: Establish an Agent security constraint model from three dimensions: resource boundary, capability boundary and trust boundary, and compile the above constraints into a computable set of policy rules; S5: Standardize high-risk action requests and perform pre-policy evaluation: Unify the various high-risk actions initiated by the Agent into a standard request object, standardize the resource identifiers in the request, bind the task context, evaluate the policy, and obtain the disposal result. S6: Controlled execution and collection of structured runtime events: For requests that pass the pre-check, the corresponding controlled execution adapter is invoked to complete the actual action; for rejected requests, a blocking event is generated; predefined audit fields are collected before and after execution to form structured audit events; S7: Construct execution trajectory and identify risk paths: Based on structured audit events under the same task identifier, construct execution trajectory diagrams according to calling relationships and time sequence relationships, jointly determine single event risks and multi-event combination risks, and identify predefined risk path types; S8: Generate audit evidence chain and evidence collection results: Gather key events, hit rules, risk paths and handling results in chronological order to generate a structured evidence package, and use hash chain or message authentication code to solidify the integrity of the key event sequence, and output audit evidence collection results.

2. The security auditing and forensics method for tool-invoking AI agents according to claim 1, characterized in that, Step S1 is as follows: S11: Deploy a security audit node between the Agent and the underlying execution environment, embedding it as a controlled proxy layer into the Agent's critical execution path; the security audit node is deployed in the form of an independent service, plug-in middleware, communication gateway, runtime proxy component, edge security component, or local daemon process; S12: Configure task access endpoint, tool invocation proxy endpoint, file access proxy endpoint, network request proxy endpoint, command execution proxy endpoint, environment variable access proxy endpoint, and audit data storage endpoint; S13: The proxy endpoint provides unified control over Tool or Skill calls, file read / write, external API access, system command calls, and environment variable reading.

3. The security auditing and forensics method for tool-invoking AI agents according to claim 1, characterized in that, Step S2 is as follows: S21: Initialize the task context object, which includes at least the task identifier, session identifier, user request summary, task intent label, current process status, set of allowed actions, risk warning information, and creation timestamp; S22: When the Agent receives a user request and starts a task, it generates a unique task identifier for that task execution. S23: During Skill activation, Tool invocation, and resource access, generate or bind Skill identifiers, Tool identifiers, event identifiers, and parent event identifiers; S24: Write the task identifier, skill identifier, tool identifier, event identifier, and parent event identifier into a unified context object, and continuously propagate them during semantic guardrail judgment, request normalization, policy evaluation, event collection, risk analysis, and evidence generation.

4. The security auditing and forensics method for tool-invoking AI agents according to claim 1, characterized in that, Step S3 specifically includes: S31: Obtain the user's original request, recent dialogue history, system configuration, business constraints, and current task context information; S32: Based on the user's original request and task context information, classify the task into a preset task category, which includes one or more of the following: document summary, retrieval and question answering, file processing, external call, and high-risk execution. S33: Load the corresponding process state set, allowed action set, process constraint rules and state transition rules according to the normalized task intent; S34: When the Agent generates a candidate tool invocation request, it extracts the candidate tool invocation intent based on the tool name, parameter structure, target resource type, current status label, and neighboring event information; S35: Compare the task intent with the candidate tool invocation intent to determine whether the tool invocation conforms to the current task objective, process status, and set of allowed actions; S36: When a candidate tool call does not conform to the current task objective, process status, or set of allowed actions, mark it as a semantic mismatch request and trigger an alarm, block, downgrade execution, or manual review.

5. The security auditing and forensics method for tool-invoking AI agents according to claim 1, characterized in that, Step S4 specifically includes: S41: Establish a resource boundary model, maintain the work area path set, sensitive resource set, and prohibited access object set, to limit the file directories, configuration files, environment variables, network addresses, and protected resources that the Agent can access; S42: Establish a capability boundary model. Based on Skill, Tool, task type, and action type, establish a capability matrix to limit the file reading, file writing, network request, command execution, or tool invocation actions that different subjects are allowed to perform under different task semantics. S43: Establish a trusted boundary model, record the Skill source, Tool whitelist, version information, signature information, registration status, and digest verification value, which are used to determine whether the calling entity and its components are within a trusted range; S44: Convert the resource boundary model, capability boundary model, and trust boundary model into a set of policy rules, wherein the set of policy rules includes at least subject conditions, action conditions, resource conditions, context conditions, and disposal results; S45: The policy rule set is distributed to the semantic guardrail orchestration, security gateway and risk association analysis process for pre-invocation checks, runtime policy evaluation and risk determination.

6. The security auditing and forensics method for tool-invoking AI agents according to claim 1, characterized in that, Step S5 specifically includes: S51: Receive the original action request initiated by the Agent through the unified security takeover entry point. The original action request includes one or more of the following: file path, tool name, calling parameters, command content, network address, environment variable name, and task context information. S52: The original action requests are uniformly abstracted into standard request objects, which include at least the request subject, action type, target resource, parameter summary, task context and timestamp; S53: Perform absolute path resolution, relative path resolution, duplicate separator processing, and symbolic link processing on the file path to obtain a normalized file resource object; S54: The command object is split into the main command program, key parameters and working directory to obtain a standardized command resource object; S55: Extract the protocol, host, port, path, request method, and data direction from the network request to obtain a normalized network resource object; S56: Extract the tool identifier, operation type, parameter structure, and parameter summary from the tool request to obtain a normalized tool request object; S57: Match the request subject, action type, target resource, and task context in the standard request object with the policy rule set, and merge them according to the rule that denial takes precedence over alarms and alarms take precedence over permission to obtain the final processing result.

7. The security auditing and forensics method for tool-invoking AI agents according to claim 1, characterized in that, Step S6 specifically includes: S61: Before executing a high-risk action, record a pre-check event, which includes at least the task identifier, subject identifier, action type, target resource, parameter summary, strategy decision result, semantic guardrail result, risk label, and timestamp; S62: For requests that are judged by semantic guardrails and evaluated by policy, the corresponding controlled execution adapter is invoked to complete the action execution. The controlled execution adapter includes one or more of the following: secure file access function, secure tool invocation function, secure command execution function, secure network request function, and secure environment variable reading function. S63: For a rejected request, generate a blocking event and stop subsequent execution; S64: After the action is completed, record the execution result event. The execution result event includes at least the execution status, the returned result summary, the error message, the risk warning, and the completion timestamp. S65: Associate the execution result event with the corresponding pre-check event and write the structured audit event to the audit data storage endpoint.

8. The security auditing and forensics method for tool-invoking AI agents according to claim 1, characterized in that, Step S7 specifically includes: S71: Based on the structured audit event set under the same task identifier, construct an execution trajectory diagram according to parent-child call relationship, time sequence relationship and resource association relationship; S72: Perform single-event risk rule matching for each audit event to identify risks such as workspace out-of-bounds access, sensitive resource reading, unauthorized tool calls, high-risk command execution, abnormal network connections, environment variable reading, and semantic mismatch. S73: Under the same task identifier, determine whether multiple events form a risk association within a preset time window. When a network external connection event occurs within a preset time after a sensitive resource read event, determine it as a sensitive read external connection risk path. S74: Execute state machine joint judgment according to the task state flow and the order of event occurrence. When the task state passes through the sensitive reading state, content processing state and external connection state in sequence, it is judged as a data outgoing risk path. S75: For risk endpoint events, trace back their parent events, predecessor events, and resource-related events based on the execution trajectory diagram to extract key evidence paths; S76: Generate a task-level risk level based on single-event risk, combined risk path, subject credibility, semantic consistency, and number of blocking events, and output the corresponding handling actions.

9. The security auditing and forensics method for tool-invoking AI agents according to claim 1, characterized in that, Step S8 specifically includes: S81: Retrieve from the audit data storage endpoint the task context, standard request object, pre-check event, execution result event, hit rule, critical risk path and handling result associated with the same task identifier; S82: Encapsulate the task metadata, audit event set, hit rule set, key risk path, risk level, handling result, and evidence summary into a structured evidence package; S83: Standardize and serialize key events according to the order of their occurrence, and recursively calculate chained summaries based on the summary of the previous event, the content of the current event, and the timestamp; S84: Perform integrity checks on the structured evidence package, including digest consistency checks, temporal continuity checks, event association consistency checks, and task identifier consistency checks; S85: When the integrity verification passes, an audit and evidence collection result is generated. The audit and evidence collection result is used to characterize whether the Agent task has high-risk behavior, the time range of risk occurrence, the Skill or Tool involved, the resource objects accessed, the risk rules hit, the actions taken, and evidence summary information.

10. A security auditing and forensics system for tool-invoking AI agents, characterized in that, This includes a security audit node deployed between the Agent entity and the underlying execution environment, wherein the security audit node includes: The deployment and access module is used to deploy security audit nodes between the Agent entity and the underlying execution environment, and configure a unified takeover entry point for high-risk actions; The context identifier management module is used to initialize the task context structure, assign a unique task identifier to each task execution, and establish the association between each execution stage; The semantic guardrail orchestration module is used to perform task intent unification, state flow constraints, and semantic consistency judgment of tool calls; The boundary modeling and policy distribution module is used to establish resource boundaries, capability boundaries, and trust boundaries, and generate a set of policy rules. The request normalization and strategy evaluation module is used to convert high-risk actions into standard request objects and perform pre-policy evaluation based on the set of policy rules. The runtime event acquisition module is used to collect structured audit events before and after the execution of an action, and to establish event relationships. The risk correlation analysis module is used to construct execution trajectory diagrams and identify single-point risks and combined risk paths; The evidence chain and evidence collection module is used to generate structured evidence packages and output the audit evidence collection results after integrity verification.

Images