A security detection method, apparatus, storage medium, and electronic device

Abstract

This specification discloses a security detection method, apparatus, storage medium, and electronic device. The method includes: acquiring tool set information of tools invoked by an agent in a large model-driven agent system; identifying the side-effect attributes and functional characteristics of the tools invoked by the agent based on the tool set information; constructing a formal model of the tools describing their interactive behavior using a domain-specific language based on the side-effect attributes and functional characteristics; configuring a formal verification engine and a risk detection strategy; inputting the formal model and risk detection strategy into the formal verification engine to perform security risk verification on the interaction logic of the combined tool invocations to obtain a combined verification result; and generating a security risk report based on the potential tool invocation risk information in the combined verification result. Using this specification's embodiments can improve the security detection effect for agent tools.

Description

Technical Field

[0001] This specification relates to the field of network security technology, and in particular to a security detection method, device, storage medium, and electronic device. Background Technology

[0002] With the rapid development of Large Model (LLM) agent technology, LLM-driven agents are increasingly expanding their capabilities by integrating external tools (such as FunctionCall and API interfaces) to handle more complex tasks. However, this has led to deep-seated logical vulnerabilities arising from the combined use of these tools, which has become a core challenge for agent application security. These security risks include not only traditional software vulnerabilities but also emerging security issues such as target hijacking, tool misuse, and unintended consequences from interactions with the environment.

[0003] Security testing methods in related technologies typically employ traditional Static Application Security Testing (SAST) or Dynamic Application Security Testing (DAST). These methods often focus on internal code defects or simple input / output anomalies of a single tool, making it difficult to effectively address the complexity and dynamism of scenarios involving combined tool calls. For example, in multi-tool collaboration scenarios, even if each tool is secure when running independently, specific combined call sequences, data transfer paths, or state change logic can lead to compromised overall security (such as permission bypassing or unauthorized data transfer). Therefore, security testing methods in related technologies struggle to systematically investigate and predict such logical vulnerabilities arising from complex interactions. Summary of the Invention

[0004] This specification provides a security detection method, apparatus, storage medium, and electronic device, the technical solutions of which are as follows: Firstly, embodiments of this specification provide a security detection method, the method comprising: Obtain tool set information for agent invocation tools in a large model-driven agent system; Based on the tool set information, identify the tool side effect attributes and tool functional characteristics of the intelligent agent calling the tool, and construct a formal model of the tool describing the tool interaction behavior using a domain-specific language based on the tool side effect attributes and tool functional characteristics; Configure a formal verification engine and a risk detection strategy, input the formal model of the tool and the risk detection strategy into the formal verification engine, and perform security risk verification on the interaction logic of calling the tool combination to obtain the combined verification result; A security risk report is generated based on potential tool call risk information in the combined verification results.

[0005] Optionally, obtaining the tool set information of agent invocation tools in a large model-driven agent system includes: To analyze the agent runtime environment of a large model-driven agent system, we need to determine the tool invocation architecture type, agent tool selection algorithm, and tool priority mechanism for agents calling tools. An agent-tool interaction model is established based on the agent tool selection algorithm and tool priority mechanism. The agent-tool interaction model is used to simulate the agent's tool calling decision logic in different scenarios. Based on the tool invocation architecture type and the agent-tool interaction model, identify a list of candidate tools accessible in the current environment of the agent system, and determine the basic meta-information of the invocation tool based on the list of candidate tools; The tool set information is obtained by integrating the tool invocation architecture type, tool priority mechanism, agent-tool interaction model, and basic meta-information.

[0006] Optionally, determining the calling tool metadata based on the candidate tool list includes: For the white-box tools in the candidate tool list, static code parsing mode is executed to obtain the source code metadata of the white-box tools in order to construct the white-box tool call dependency graph, and the white-box tool metadata is obtained based on the white-box tool call dependency graph; For the black-box tools in the candidate tool list, the basic interface definition is obtained based on the public interface document, and a dynamic code testing mode is adopted to construct test inputs for the black-box tools so as to collect tool output feedback and system state changes through monitoring tools. Based on the tool output feedback and system state changes, the behavior pattern and potential side effect attributes of the black-box tools are inferred, and the black-box tool meta-information is obtained based on the behavior pattern and potential side effect attributes of the black-box tools. The white-box tool metadata and the black-box tool metadata are integrated to obtain the calling tool metadata.

[0007] Optionally, the step of constructing a formal model of the tool describing its interactive behavior using a domain-specific language based on the tool's side-effect attributes and functional characteristics includes: The input parameter types, data range constraints, and execution preconditions in the tool's functional characteristics are converted into type definitions and logical constraints in a domain-specific language. Based on these type definitions and logical constraints, the interface constraint mapping relationship of the calling tool in the formal model is established. The tool's functional logic and expected output results in the tool's functional characteristics are mapped to the functional logic mapping relationship in the formal model; Based on the side effect attributes of the tools, a side effect classification system is established that includes data operation dimensions and system resource dimensions. Based on the side effect attributes and functional characteristics of the tools, the execution logic of each tool call is determined. The side effect behavior matching the execution logic is determined in the side effect classification system, and the side effect behavior is mapped to the side effect semantic mapping relationship in the formal model.

[0008] Optionally, before converting the input parameter types, data range constraints, and execution preconditions in the tool's functional characteristics into type definitions and logical constraints in a domain-specific language, the method further includes: Semantic analysis is performed on the functional description text in the tool's functional characteristics to obtain semantic analysis results; Based on the semantic analysis results, detect whether there are security risk semantic features in the function description text; If it exists, the tool node corresponding to the security risk semantic feature is marked as a high-risk node in the formal model of the tool, and security constraints are added for the high-risk node.

[0009] Optionally, the configuration of the formal verification engine includes: Based on the tool set information, a tool random combination process is performed to obtain the tool combination to be verified; The state space size of the tool combination interaction logic corresponding to the tool combination being invoked is determined based on the formal model of the tool; When the state space is a finite-sized class, a model detector is selected as the formal verification engine, and a full state space search strategy is configured for the formal verification engine. When the state space is of infinite size, a symbolic execution engine or a theorem prover is selected as the formal verification engine, and an abstract reduction strategy is configured for the formal verification engine.

[0010] Optionally, the security risk verification of the interaction logic for calling the tool combination includes: Based on the tool function descriptions in the toolset information, the side effect definitions in the tool formal model, and the vulnerability patterns in the risk detection strategy, scenario analysis prompts are constructed. Based on the constructed scenario analysis prompt information, the target large language model is invoked to parse the current tool combination scenario to obtain scenario context semantics, and based on the scenario context semantics, an interaction hypothesis path is generated. The interaction hypothesis path represents the tool call sequence and parameter combination that may trigger the vulnerability mode. The interaction hypothesis path is converted into formal constraints and input into the formal verification engine. The formal verification engine is then driven to perform a directional search within the state space defined by the formal constraints to search for state transition paths that match the pre-assumptions and to verify whether the state transition paths are counterexample paths that violate the risk detection strategy.

[0011] Optionally, generating a security risk report based on potential tool invocation risk information in the combined verification results includes: Extract the counterexample paths output by the formal verification engine from the combined verification results; Based on the aforementioned counterexample path, a risk visualization view is constructed to show the dependencies between different tools and the flow of tool data during the vulnerability triggering process; Based on the logical vulnerability types identified by the combined verification results and the counterexample paths, security remediation suggestions are generated. The negative example path, the risk visualization view, and the remediation suggestions are integrated to generate a security risk report.

[0012] Secondly, embodiments of this specification provide a security detection device, the device comprising: The system tool extraction module is used to obtain tool set information of intelligent agents calling tools in intelligent agent systems based on large model-driven systems; The tool formal description module is used to identify the tool side effect attributes and tool functional characteristics of the intelligent agent calling the tool based on the tool set information, and to construct a tool formal model describing the tool interaction behavior using a domain-specific language based on the tool side effect attributes and tool functional characteristics. The combined verification analysis module is used to configure the formal verification engine and risk detection strategy. The formal model of the tool and the risk detection strategy are input into the formal verification engine to perform security risk verification on the interaction logic of calling the tool combination to obtain the combined verification result. The combined verification analysis module is used to generate a security risk report based on the potential tool call risk information in the combined verification results.

[0013] Optionally, obtaining the tool set information of agent invocation tools in a large model-driven agent system includes: To analyze the agent runtime environment of a large model-driven agent system, we need to determine the tool invocation architecture type, agent tool selection algorithm, and tool priority mechanism for agents calling tools. An agent-tool interaction model is established based on the agent tool selection algorithm and tool priority mechanism. The agent-tool interaction model is used to simulate the agent's tool calling decision logic in different scenarios. Based on the tool invocation architecture type and the agent-tool interaction model, identify a list of candidate tools accessible in the current environment of the agent system, and determine the basic meta-information of the invocation tool based on the list of candidate tools; The tool set information is obtained by integrating the tool invocation architecture type, tool priority mechanism, agent-tool interaction model, and basic meta-information.

[0014] Optionally, determining the calling tool metadata based on the candidate tool list includes: For the white-box tools in the candidate tool list, static code parsing mode is executed to obtain the source code metadata of the white-box tools in order to construct the white-box tool call dependency graph, and the white-box tool metadata is obtained based on the white-box tool call dependency graph; For the black-box tools in the candidate tool list, the basic interface definition is obtained based on the public interface document, and a dynamic code testing mode is adopted to construct test inputs for the black-box tools so as to collect tool output feedback and system state changes through monitoring tools. Based on the tool output feedback and system state changes, the behavior pattern and potential side effect attributes of the black-box tools are inferred, and the black-box tool meta-information is obtained based on the behavior pattern and potential side effect attributes of the black-box tools. The white-box tool metadata and the black-box tool metadata are integrated to obtain the calling tool metadata.

[0015] Optionally, the step of constructing a formal model of the tool describing its interactive behavior using a domain-specific language based on the tool's side-effect attributes and functional characteristics includes: The input parameter types, data range constraints, and execution preconditions in the tool's functional characteristics are converted into type definitions and logical constraints in a domain-specific language. Based on these type definitions and logical constraints, the interface constraint mapping relationship of the calling tool in the formal model is established. The tool's functional logic and expected output results in the tool's functional characteristics are mapped to the functional logic mapping relationship in the formal model; Based on the side effect attributes of the tools, a side effect classification system is established that includes data operation dimensions and system resource dimensions. Based on the side effect attributes and functional characteristics of the tools, the execution logic of each tool call is determined. The side effect behavior matching the execution logic is determined in the side effect classification system, and the side effect behavior is mapped to the side effect semantic mapping relationship in the formal model.

[0016] Optionally, before converting the input parameter types, data range constraints, and execution preconditions in the tool's functional characteristics into type definitions and logical constraints in a domain-specific language, the method further includes: Semantic analysis is performed on the functional description text in the tool's functional characteristics to obtain semantic analysis results; Based on the semantic analysis results, detect whether there are security risk semantic features in the function description text; If it exists, the tool node corresponding to the security risk semantic feature is marked as a high-risk node in the formal model of the tool, and security constraints are added for the high-risk node.

[0017] Optionally, the configuration of the formal verification engine includes: Based on the tool set information, a tool random combination process is performed to obtain the tool combination to be verified; The state space size of the tool combination interaction logic corresponding to the tool combination being invoked is determined based on the formal model of the tool; When the state space is a finite-sized class, a model detector is selected as the formal verification engine, and a full state space search strategy is configured for the formal verification engine. When the state space is of infinite size, a symbolic execution engine or a theorem prover is selected as the formal verification engine, and an abstract reduction strategy is configured for the formal verification engine.

[0018] Optionally, the security risk verification of the interaction logic for calling the tool combination includes: Based on the tool function descriptions in the toolset information, the side effect definitions in the tool formal model, and the vulnerability patterns in the risk detection strategy, scenario analysis prompts are constructed. Based on the constructed scenario analysis prompt information, the target large language model is invoked to parse the current tool combination scenario to obtain scenario context semantics, and based on the scenario context semantics, an interaction hypothesis path is generated. The interaction hypothesis path represents the tool call sequence and parameter combination that may trigger the vulnerability mode. The interaction hypothesis path is converted into formal constraints and input into the formal verification engine. The formal verification engine is then driven to perform a directional search within the state space defined by the formal constraints to search for state transition paths that match the pre-assumptions and to verify whether the state transition paths are counterexample paths that violate the risk detection strategy.

[0019] Optionally, generating a security risk report based on potential tool invocation risk information in the combined verification results includes: Extract the counterexample paths output by the formal verification engine from the combined verification results; Based on the aforementioned counterexample path, a risk visualization view is constructed to show the dependencies between different tools and the flow of tool data during the vulnerability triggering process; Based on the logical vulnerability types identified by the combined verification results and the counterexample paths, security remediation suggestions are generated. The negative example path, the risk visualization view, and the remediation suggestions are integrated to generate a security risk report.

[0020] Thirdly, embodiments of this specification provide a computer storage medium storing a plurality of instructions adapted for loading by a processor and executing the above-described method steps.

[0021] Fourthly, this specification provides a computer program product storing at least one instruction adapted to be loaded by a processor and to execute the method steps of one or more embodiments of this specification.

[0022] Fifthly, this specification provides a computer program product storing at least one instruction adapted to be loaded by a processor and to execute the method steps of one or more embodiments of this specification.

[0023] Fifthly, embodiments of this specification provide an electronic device that may include: a processor and a memory; wherein the memory stores a computer program adapted to be loaded by the processor and to execute the above-described method steps.

[0024] The beneficial effects of the technical solutions provided in some embodiments of this specification include at least the following: In one or more embodiments of this specification, the architecture of the intelligent agent system is analyzed and tool set information, including white-box and black-box tools, is extracted. Then, the side-effect attributes and functional characteristics of each tool are identified. This information is translated into a formal model describing the tool interaction behavior using a domain-specific language. Subsequently, an adapted formal verification engine and risk detection strategy are configured. The model and strategy are input into the engine, and model detection or symbolic execution techniques are used to perform mathematical verification analysis on the interaction logic of the tool combination. Finally, a security risk report containing a visual view and remediation suggestions is generated based on the counterexample paths obtained from the verification. This solves the problem of effectively detecting deep logic vulnerabilities generated by large-model intelligent agents in complex multi-tool interaction scenarios, and addresses the technical challenge of traditional testing methods being unable to cope with the massive state space generated by intelligent agent tool combinations, resulting in low verification coverage and high false positive rates. It can accurately verify the tool invocation behavior of intelligent agents. By transforming fuzzy natural language specifications or unstructured code logic into a rigorous formal model, it eliminates the randomness and uncertainty in traditional fuzz testing. It can effectively cover the full state space of tool combination invocation, thereby accurately identifying long chain complex logic vulnerabilities across multiple tools that are difficult to find by conventional testing methods. This reduces the difficulty of security investigation and operation and maintenance costs of intelligent agent systems, and significantly improves the robustness and credibility of intelligent agent applications in complex task environments. Attached Figure Description

[0025] To more clearly illustrate the technical solutions in the embodiments or prior art of this specification, the drawings used in the description of the embodiments or prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this specification. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0026] Figure 1 This is a schematic flowchart of a security detection method provided in the embodiments of this specification; Figure 2 This is a schematic diagram of a security detection scenario provided in the embodiments of this specification; Figure 3 This is a flowchart illustrating a tool set information determination process provided in the embodiments of this specification; Figure 4 This is a schematic diagram of the process of constructing a formal model using a tool provided in the embodiments of this specification; Figure 5 This is a schematic diagram of a verification analysis process provided in the embodiments of this specification; Figure 6 This is a schematic diagram of the structure of a safety detection device provided in the embodiments of this specification; Figure 7This is a schematic diagram of the structure of an electronic device provided in the embodiments of this specification. Detailed Implementation

[0027] The technical solutions in the embodiments of this specification will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this specification, and not all embodiments. Based on the embodiments in this specification, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this specification.

[0028] In this specification, it should be understood that the terms "first," "second," etc., are used for descriptive purposes only and should not be construed as indicating or implying relative importance. In this specification, it should be noted that, unless otherwise expressly specified and limited, "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion. For example, a process, method, system, product, or device that includes a series of steps or units is not limited to the listed steps or units, but may optionally include steps or units not listed, or may optionally include other steps or units inherent to these processes, methods, products, or devices. Those skilled in the art can understand the specific meaning of the above terms in this specification based on the specific circumstances. Furthermore, in this specification, unless otherwise stated, "multiple" means two or more. "And / or" describes the relationship between related objects, indicating that three relationships can exist; for example, A and / or B can represent: A alone, A and B simultaneously, or B alone. The character " / " generally indicates that the preceding and following related objects are in an "or" relationship.

[0029] The present specification will now be described in detail with reference to specific embodiments.

[0030] In one embodiment, such as Figure 1 As shown, a security detection method is proposed, which can be implemented using a computer program and run on a security detection device based on the von Neumann architecture. This computer program can be integrated into an application or run as a standalone utility application. The security detection device can be an electronic device, including but not limited to: personal computers, tablets, handheld devices, vehicle-mounted devices, server devices, computing devices, or other processing devices connected to a wireless modem. In different networks, terminal devices can be called by different names, such as: user equipment, access terminal, user unit, user station, mobile station, mobile station, remote station, remote terminal, mobile device, user terminal, terminal, wireless communication device, user agent or user equipment, cellular phone, cordless phone, device in a 5G network or future evolved network, etc.

[0031] Specifically, the safety testing method includes: S102: Obtain tool set information for agent invocation tools in a large model-driven agent system; An intelligent agent system (or agent for short) refers to an automatic or semi-automatic system built on a large model (LLM) that has the ability to perceive the environment, make inference decisions, and invoke external tools to perform tasks.

[0032] Tool set information refers to the total metadata set of all external tools currently accessible to the intelligent agent system. Optionally, it includes, but is not limited to: the tool's unique identifier, interface specifications (such as input parameter names, data types, and value ranges), function description text, tool invocation priority configuration, and tool architecture type (such as local functions or remote APIs).

[0033] Indicative, such as Figure 2 As shown, Figure 2 This is a schematic diagram of a security detection scenario architecture, such as... Figure 2 The system can comprehensively analyze the agent's operating environment through its included system tool extraction module (mainly involving agent type analysis, white-box tool source code analysis, and black-box tool list acquisition). Specifically, by scanning the agent's configuration file and initialization code, it identifies the agent's basic architecture type, determining whether it is based on a "Model-Controller-Program" (MCP) architecture, a FunctionCall mechanism, and a custom API interface architecture. Based on this basic architecture type, it further analyzes the tool selection algorithm logic inside the agent, extracts the tool call priority rules or weight parameters configured in the tool selection algorithm logic, identifies whether there is a preemption mechanism that makes certain tools easy to be called first, and simulates the real tool selection and calling process by establishing an interaction model between the agent and the calling tools.

[0034] After clarifying the infrastructure type and tool selection algorithm logic, a differentiated extraction strategy is adopted based on the visibility characteristics of the calling tools to construct a complete list of calling tools.

[0035] For white-box tools whose source code is visible in the tool call list, the system performs white-box tool source code analysis. Static code analysis technology is used to directly parse the tool source code and construct an Abstract Syntax Tree (AST). From this, the system accurately extracts function signature information, including function name, parameter list, specific parameter type declarations, and preset parameter constraints, such as data range or format requirements. During static code analysis, the electronic device system deeply traces the control flow of the source code, identifying embedded permission checking logic, sensitive data processing logic, and API calls to external services. Based on this identification, a tool call dependency graph is constructed. Using the function signature information and the tool call dependency graph, the system further generates tool behavior characteristics for white-box class tool calls, explicitly revealing the data and control flows between different tools, thereby identifying cross-tool interaction points.

[0036] Meanwhile, for black-box tools (such as third-party or external services) whose source code is not visible in the tool list, the black-box tool list is retrieved by accessing the publicly available documentation and interface description files of the black-box tools. This allows for the extraction of basic functional descriptions, interface endpoints, request methods, and parameter definitions, which are then compiled into basic tool information for the black-box tools. In practical applications, to compensate for potential gaps or delays in documentation, electronic devices further employ dynamic testing techniques for probing analysis of the tools. By constructing and sending a series of specific test inputs, the output of the tool calls by a large model is monitored, allowing for real-time monitoring of the tool's output feedback and state changes. For example, observing whether inputting a specific path triggers file read errors or network requests allows for reverse inference of the actual behavior patterns and potential side effects of the black-box tools, thus forming the tool behavior characteristics of black-box tool calls.

[0037] Finally, all architectural attributes, priority logic, interface definitions, and behavioral characteristics obtained through architecture parsing, static analysis, and dynamic detection are cleaned, standardized, and integrated to generate a structured set of tools information.

[0038] S104: Based on the tool set information, identify the tool side effect attributes and tool functional characteristics of the intelligent agent calling the tool, and construct a formal model of the tool describing the tool interaction behavior using a domain-specific language based on the tool side effect attributes and tool functional characteristics; The functional characteristics of a tool refer to the core business logic and interface constraints that the tool completes during normal execution. Optionally, this may include the data type of the input parameters, value range constraints, preconditions, and the expected output of the tool after execution.

[0039] Tool side effects refer to implicit effects on the system environment or data assets during tool execution, other than the intended tool function. These implicit effects are typically not reflected in the function return value of the tool. In some embodiments, they can be categorized into data operation dimensions (such as reading, modifying, deleting, and providing) and system resource dimensions (such as network calls, file writing, and high CPU usage). Domain-specific languages, or DSLs, in this embodiment refer to formal modeling languages ​​used to describe logical models and security constraints. These languages ​​can be based on related technologies such as Alloy, TLA+, or specially designed security specification languages.

[0040] A tool formal model refers to a system description document constructed using a domain-specific language with rigorous mathematical semantics. This tool formal model is an abstract mapping of the tool and its interaction environment, a mathematical model used to accurately describe the behavioral characteristics and interaction logic of an intelligent agent tool. The model aims to transform the natural language description or code implementation of the tool into a structured mapping expression that can be processed by a formal verification engine. Its specific composition includes, but is not limited to, the following three dimensions of mapping relationships: Interface constraint mapping: Convert the tool's interface specification (including input parameter types, data range constraints, and execution preconditions) into type definitions and logical constraint facts in a formal language, thereby establishing the tool's legal input state space in the model; Functional logic mapping: Abstracting the tool's functional logic and expected output results into state transition functions or function mappings, and mathematically describing the transition process of the system state set before and after the tool's execution; Secondary semantic mapping: The side effects of tools on the external environment (such as data reading, network calls, etc.) are defined as specific operation predicates or attribute enumerations in the model. Through the above mapping, the model can support the verification engine to perform systematic and exhaustive analysis of deep interaction logic risks such as data leakage, unauthorized access, and code injection in multi-tool combination scenarios.

[0041] Illustratively, electronic devices can use a tool formal description module to transform collected tool set information into a mathematical model capable of formal analysis—that is, a tool formal model. This module uses a specially designed high-level domain-specific language (DSL) to precisely describe the preconditions, postconditions, invariants, and side effects of each tool.

[0042] Furthermore, the electronic device first performs in-depth identification and quantification of the multidimensional attributes of the tools. Based on a pre-built side effect classification dictionary that includes data operation dimensions and system resource dimensions, it automatically scans and extracts features from the tool set information. For white-box tools with visible source code, it traces the underlying API call chain by parsing their Abstract Syntax Tree (AST). For example, it identifies and maps write operation functions to the file system in the code as the side effect attribute of "file system modification". For black-box tools with only interface documentation, it inversely infers and marks their potential impact on network, database, or memory states based on the state change logs recorded during the dynamic probing phase.

[0043] While completing the basic side effect identification, a security semantic analysis process is simultaneously initiated. Natural language processing models are used to conduct in-depth audits of the tool's functional description text to detect whether there are hidden features such as ignoring preceding instructions or forcing priority execution. After detecting and identifying such high-risk semantics, high-risk tags are marked in the tool's metadata so that stricter calling constraints can be imposed in subsequent modeling.

[0044] After the comprehensive identification and cleaning of tool attributes, the formal logic translation stage begins. This involves calling a Domain-Specific Language (DSL) compiler or transformation engine to perform a three-layer rigorous mapping from the description space to the mathematical model space. 1) Interface constraint mapping: The tool's input parameter types, data value ranges, and execution prerequisites are converted into type definitions and logical facts in the DSL, thus establishing the tool's legal input state space in the formal model. For example, the natural language constraint "parameters must be positive integers" is transformed into a mathematical inequality constraint. 2) Side effect semantic mapping: A global set of side effect enumerations is established, and the specific side effect attributes identified in the previous stage are bound to the corresponding tool entities through predicate logic. For example, specific operation predicates are defined to quantify the tool's ability to exert unintended influence on the external environment. 3) Tool functional logic mapping: The tool's business execution logic and expected output results are abstracted into state transition functions, describing the transition relationship of the system state set before and after tool execution. This defines how the system transitions from one state to the next through tool calls. Through the aforementioned progressive translation process, a complete formal model file of the tool, containing tool interaction semantics and security constraints, is finally generated.

[0045] S106: Configure the formal verification engine and risk detection strategy, input the formal model of the tool and the risk detection strategy into the formal verification engine to perform security risk verification on the interaction logic of calling the tool combination and obtain the combined verification result; A formal verification engine is a core computational component that automatically deduces and verifies a system model (i.e., a formal model of the tool) based on rigorous mathematical logic (such as first-order predicate logic, linear temporal logic (LTL), and computational tree logic (CTL)). In this embodiment, the formal verification engine does not rely on random testing or fuzzy input. Instead, it exhaustively or purposefully searches the state space of the formal model of the tool at the mathematical level using algorithms (such as SAT / SMT solving algorithms and BDD binary decision graph algorithms) to confirm whether the system meets the preset security attributes through mathematical proof. Common formal verification engines include, but are not limited to, model detectors (such as AlloyAnalyzer and Spin) and symbolic execution engines (such as KLEE).

[0046] A risk detection strategy refers to transforming abstract security compliance requirements or known logical vulnerability patterns into a set of logical assertions or constraint formulas that can be recognized and computed by a formal verification engine. This strategy defines the security attributes that the intelligent agent system must satisfy (e.g., sensitive data never flows to public networks) and the prohibited dangerous states (e.g., unauthorized users eventually being able to execute high-risk commands). In this technical solution, the risk detection strategy can typically be represented as a set of logical formulas, used as the basis for the verification engine's judgment.

[0047] Tool invocation refers to the coordinated execution sequence of multiple tools in a large model-driven intelligent agent system, arranged in a specific order, with conditional branches or nested relationships, to achieve a final goal when performing complex tasks. Unlike the independent invocation of a single tool, tool invocation emphasizes the sequential or parallel relationships between multiple tools (such as search tools, file reading tools, and network sending tools) in both temporal and logical dimensions. This specification focuses on identifying emergent behavioral risks that may arise from logical coupling when individual tools are safe when combined.

[0048] Interaction logic refers to the rules and state transition mechanisms of data flow between different tool entities and between tools and the system environment in a tool-invoking combination. This includes: data flow logic: how the output of one tool becomes the input of the next, and whether data type conversion or permission changes occur during transmission; control flow logic: whether the execution of a tool depends on the execution state of the preceding tool, and whether it changes the global system state. The core of formal verification is to mathematically deduce the legality of this interaction logic.

[0049] The combined verification result refers to the deterministic conclusion and related evidence information output by the formal verification engine after matching and solving the tool's formal model with the risk detection strategy. This result typically includes two states: Verification passed: proving that the system satisfies the risk detection strategy under all possible interaction paths and that no known vulnerabilities exist; Verification failed: proving that at least one execution path violates the risk detection strategy. In this case, the result will include specific counterexample paths, detailing the complete tool call sequence that led to the vulnerability, parameter passing details, and a snapshot of the system state at each step.

[0050] As an illustration, electronic devices can use combined verification analysis modules based on the constructed tool formal models, and employ formal verification engines such as efficient model detectors, symbolic execution engines, or automated theorem provers to systematically and exhaustively explore and verify the possible combined call sequences and complex interaction logic between different tools, thereby promptly identifying potential security risks.

[0051] Specifically: The electronic device first initializes the verification environment, evaluates and selects an appropriate formal verification engine type based on the system size and complexity of the intelligent agent system to be verified, and loads the corresponding risk detection strategy library. During the strategy loading phase, the pre-set general security rules are converted into formal logic formulas recognizable by the verification engine. For example, natural language rules prohibiting the leakage of sensitive data are converted into linear time-series logic formulas, stipulating that data objects marked as sensitive must not appear in the input domain of public network interfaces in all possible time paths of system evolution, thereby completing the mathematical definition of the verification rules.

[0052] After configuring the formal verification engine and risk detection strategy, import the tool's formal model and start the verification process. To balance verification completeness and efficiency, different scenario characteristics can be determined based on the tool's formal model to adopt differentiated verification processing paths corresponding to these characteristics. For scenario characteristic types with a relatively small number of tools and closed interaction logic, a full-state-space exhaustive verification path based on model detection is adopted. Under this path, a model detector is configured as the core engine, converting the tool's formal model and risk strategy formula into a Boolean satisfiability problem (SAT). The formal verification engine is controlled to exhaustively traverse all possible tool call sequences, parameter combinations, and state transitions within a set search depth, without relying on any prior assumptions. Each reachable state is checked one by one to see if it satisfies the safety policy formula. When a state sequence is found to violate the policy, the formal verification engine is controlled to stop searching and the complete execution sequence that violates the policy is recorded as a counterexample path.

[0053] Meanwhile, for scenarios with numerous tools and a huge or even infinite parameter space, a symbolic execution and targeted verification path guided by a Large Language Model (LLM) is adopted. Under this path, the LLM is first invoked to analyze the current tool combination scenario, generating several high-risk interaction hypothesis paths based on semantic understanding. For example, it is speculated that an attacker might use the output of a file search tool as a parameter to invoke a code execution tool. Subsequently, the electronic device converts the natural language hypotheses output by the LLM into path constraints and injects them into the symbolic execution engine as a formal verification engine. During execution, the engine uses these constraints to prune the massive state tree, ignoring branches irrelevant to the hypotheses and performing deep probing only within the local state space defined by the hypothetical path. By solving the constraint equations on the path, it determines whether there are states that violate security policies. If the equations have solutions, it proves that the vulnerable path is logically reachable. Finally, regardless of the path used, the formal verification engine outputs a combined verification result containing the judgment conclusion and potential counterexample evidence.

[0054] S108: Generate a security risk report based on potential tool call risk information in the combined verification results.

[0055] Potential tool call risk information refers to key vulnerability evidence data extracted from the raw results output by the formal verification engine. Its core component is the counterexample path, which records the complete logical trajectory from the initial state of the system, through a series of specific tool call steps (including the input parameter values ​​and function return results of each step), to finally reaching the violation state (such as a data leakage point or an unauthorized privilege state).

[0056] A security risk report refers to the security detection output generated based on risk information. Optionally, the report can be a user-readable document containing graphical representations and textual descriptions of vulnerabilities; or it can be a machine-readable structured file containing vulnerability code line numbers, severity ratings, and automated blocking instructions.

[0057] As an illustration, the system determines whether there are counterexamples in the verification status based on the combined verification results. If so, it extracts the original counterexample path data, filters out intermediate state variables unrelated to vulnerability triggering, and retains only key tool nodes, parameter passing edges, and system state change snapshots. Based on this, a concise information on potential tool call risks is constructed. To meet the needs of different application scenarios, the following two differentiated report generation and processing paths can be adopted.

[0058] Optionally, one approach focuses on the visual reproduction and manual auditing of vulnerabilities. It constructs a directed acyclic graph or sequence graph using extracted counterexample path data. Different shaped primitives are used in the view to distinguish intelligent agents, tool entities, and external environment objects. Lines are used to represent the flow of tool data and control, and key links that trigger risks are highlighted with special colors (such as red). For example, the complete path of sensitive data flowing from database tools to external API tools is marked. At the same time, a natural language generation module is called to translate the mathematical counterexample steps into readable attack narrative text, describing how attackers use tools to call in a specific order to achieve their attack targets. Finally, the visual view and narrative text are integrated to generate an interactive security report in PDF or HTML format.

[0059] Optionally, one approach focuses on integrating automated remediation with attribution information. This involves analyzing risk information to identify the root causes of risks, such as determining whether the issue stems from a lack of data cleansing mechanisms between tools or overly permissive interface permissions. Based on these attribution results, corresponding remediation strategies are matched from a pre-built knowledge base, automatically generating code-level patch suggestions or configuration file modification snippets. For example, this could generate code that mandates the insertion of filters between two tools. The vulnerability type, severity level, reproduction steps, and remediation code are then encapsulated in a standard structured data format. This security risk report can be directly pushed to a code hosting platform or CI / CD pipeline for automated risk mitigation and remediation.

[0060] In the embodiments of this specification, the architecture of the intelligent agent system is analyzed and tool set information, including white-box and black-box tools, is extracted. Then, the side-effect attributes and functional characteristics of each tool are identified. This information is translated into a formal model describing the tool interaction behavior using a domain-specific language. Subsequently, an adapted formal verification engine and risk detection strategy are configured. The model and strategy are input into the engine, and model detection or symbolic execution techniques are used to perform mathematical verification analysis on the interaction logic of the tool combination. Finally, a security risk report containing a visual view and remediation suggestions is generated based on the counterexample paths obtained from the verification. This solves the problem of effectively detecting deep logic vulnerabilities generated by large-model intelligent agents in complex multi-tool interaction scenarios, and addresses the technical challenge that traditional testing methods cannot cope with the massive state space generated by intelligent agent tool combinations, resulting in low verification coverage and high false positive rates. It can accurately verify the tool invocation behavior of intelligent agents. By transforming fuzzy natural language specifications or unstructured code logic into a rigorous formal model, it eliminates the randomness and uncertainty in traditional fuzz testing. It can effectively cover the full state space of tool combination invocation, thereby accurately identifying long chain complex logic vulnerabilities across multiple tools that are difficult to find by conventional testing methods. This reduces the difficulty of security investigation and operation and maintenance costs of intelligent agent systems, and significantly improves the robustness and credibility of intelligent agent applications in complex task environments.

[0061] Optional, please see Figure 3 , Figure 3 This is a flowchart illustrating a process for determining tool set information as proposed in this specification. Specifically, the process of obtaining tool set information for agent-invoking tools in a large model-driven agent system includes: S202: Analyze the agent runtime environment of a large model-driven agent system to determine the tool invocation architecture type, agent tool selection algorithm, and tool priority mechanism for agents calling tools. Tool call architecture type refers to the technical framework standard for intelligent agent systems to integrate and schedule external tools. Common types include: MCP type, function call type, ReAct type, and Plugins / API type; Intelligent agent tool selection algorithms refer to the decision-making logic by which an intelligent agent, when faced with user commands, decides which (or several) of available tools to invoke from among many available tools. Common algorithms include semantic matching based on vector similarity, intent recognition based on LLM inference, or conditional triggering based on hard-coded rules.

[0062] To illustrate, electronic devices perform deep analysis of the agent's operating environment. For white-box tool calling systems, they read configuration files or initialization code, identify the defined architecture fields to determine the tool calling architecture type, and parse prompt word templates and retrieval configurations in the code to extract weight parameters for tool priority, thus determining the agent's tool selection algorithm and tool priority mechanism. For black-box systems, they send probing meta-instructions (such as "Please describe your thought process" or "List your capability boundaries"), analyze the log structure and thought chain format returned by the agent, and infer its architecture type from this. They also construct competing instructions with overlapping functions (such as simultaneously requesting to find and read files) to observe the agent's selection tendency, thereby inferring its implicit agent tool selection algorithm and tool priority mechanism.

[0063] S204: Based on the agent tool selection algorithm and tool priority mechanism, establish an agent-tool interaction model. The agent-tool interaction model is used to simulate the agent's tool invocation decision logic in different scenarios. The agent-tool interaction model refers to a logical model used to simulate the decision-making behavior of an agent (different from the formal model of tool behavior). This agent-tool interaction model describes the decision distribution of an agent choosing tool T with probability P under input scenario X, and is mainly used to analyze tool preemption risk.

[0064] Tool invocation decision logic refers to the internal processing by which an agent, upon receiving natural language instructions, transforms unstructured intent into specific tool invocations. This tool invocation decision logic is typically influenced by prompt word engineering, the model's own preferences, and hard-coded priority rules.

[0065] Tool preemption can be understood as a security risk, which refers to malicious tools inducing an agent to mistakenly call a malicious tool when it should call a normal tool by constructing descriptions similar to high-frequency words or by using high-priority rules.

[0066] This is illustrative of a processing strategy that combines rule-based deterministic modeling with vector-based probabilistic modeling for different types of agent decision-making mechanisms.

[0067] For agent systems based on rules or hard-coded priorities (such as systems based on the ReAct framework and containing explicit if-then logic), a deterministic decision tree model is constructed. Priority rules are transformed into node splitting conditions of the tree, and the functional description keywords of the calling tools are parsed and mapped to leaf nodes. This deterministic decision tree model can accurately simulate the single execution path of the agent triggered by specific keywords, and is used to detect whether there are logical conflicts or dead code regions, i.e., situations where some legitimate tools can never be selected due to improper priority rule settings.

[0068] For agent systems based on large-model semantic reasoning (such as systems based on embedding similarity matching), a probabilistic vector space model is constructed. Using the same embedding model as the target agent, the functional description texts of all candidate tools are converted into high-dimensional vectors, building a tool vector library. Then, different user input scenarios are simulated to generate query vectors, and the cosine similarity score between the query vector and each tool vector is calculated. A probability distribution function is established based on the selection threshold parsed from S202 (e.g., only calling tools if similarity > 0.8). This probabilistic vector space model is used to simulate the decision boundary of the agent when facing ambiguous instructions or malicious adversarial examples, and is particularly useful for identifying the potential risk of malicious tools squeezing out the normal tool selection space by fine-tuning description vectors.

[0069] S206: Based on the tool invocation architecture type and the agent-tool interaction model, identify a list of candidate tools accessible in the current environment of the agent system, and determine the basic meta-information of the invocation tool based on the list of candidate tools; The candidate tool list refers to the set of tools that can actually be accessed and scheduled by the agent in the current specific agent operating environment and permission context.

[0070] Basic metadata refers to standard data fields that describe the static characteristics of a tool. This includes, but is not limited to, the tool's unique identifier, functional description text, interface parameter definitions, and calling protocols.

[0071] Optionally, a static identification mode based on architecture traversal and interaction model filtering can be adopted, specifically locating the storage location of tool definitions based on the tool's architecture type. If the architecture is MCP, the system directly scans the controller's configuration file; if the architecture is Plugin mode, it traverses the plugin registry. After obtaining a preliminary "full toolset," the constructed agent-tool interaction model is introduced for reachability filtering: all tools are substituted into the interaction model for logical deduction, eliminating tools that are on unreachable branches in the interaction model's decision tree, as well as redundant tools whose selection probability in the probability model is below a preset threshold. The filtered set is the candidate tool list. Finally, for each tool in the list, its interface definition file is parsed, its name, parameters, and description are extracted, and then the parsed information is standardized into a basic metadata data structure.

[0072] Optionally, an extraction and recognition model based on scene simulation and dynamic reflection can be adopted, and a communication protocol adapter can be constructed according to the determined architecture type. The constructed agent-tool interaction model generates a set of simulation scene contexts covering high-frequency decision paths. For example, if the interaction model shows that the agent will activate a specific subset of tools when processing "financial intentions," a test instruction containing financial keywords is constructed, and the simulation scene context is sent to the agent system to trigger its tool selection mechanism. After the agent responds (e.g., returning a FunctionCall request or API call log), the identifier of the tool it attempted to invoke is captured. A candidate tool list is constructed by collecting all successfully activated tools through multiple rounds of simulation. For the tools in the list, the system further sends a metadata query instruction, guiding the agent to return the tool's usage instructions, and parses the parameter structure and functional description from the tool's usage instructions to determine the basic metadata.

[0073] This specification establishes the architecture type and internal priority mechanism of tool invocation by deeply analyzing the operating environment of the intelligent agent system. Using the analyzed selection algorithm, an intelligent agent-tool interaction model is constructed to simulate the decision-making logic of the intelligent agent in different contexts. Furthermore, based on this interaction model, accessibility analysis and filtering are performed on all tools to accurately identify the list of actually accessible candidate tools and their basic metadata in the current environment. This approach establishes a precise mapping between the intelligent agent's decision-making logic and the tool set. Introducing the interaction model for dynamic simulation not only eliminates unreachable redundant tools in the early stages of detection, reducing the computational overhead of subsequent formal verification, but also exposes potential logical vulnerabilities in the intelligent agent's tool selection phase that could be maliciously guided or misused without actually executing code. This ensures the completeness and effectiveness of the subsequently generated candidate tool list, significantly improving the adaptability and accuracy of security testing for heterogeneous intelligent agent architectures (including white-box and black-box architectures).

[0074] In one feasible implementation, the specific execution of determining the calling tool metadata based on the candidate tool list includes: Step A2: For the white-box tools in the candidate tool list, execute static code parsing mode to obtain the source code metadata of the white-box tools to construct the white-box tool call dependency graph, and obtain the white-box tool metadata based on the white-box tool call dependency graph; Static code analysis refers to a technique for examining software source code in a non-runtime environment. In this embodiment, this technique mainly extracts the semantic features of the program by constructing an Abstract Syntax Tree (AST) and a Control Flow Graph (CFG) to perform syntactic structure analysis, data flow tracing, and control logic deduction.

[0075] White-box tool source code metadata refers to atomic feature data extracted directly from the source code. This includes: function signatures, variable definitions, class inheritance relationships, third-party library declarations, and key API call statements.

[0076] A white-box tool dependency graph is a directed graph structure used to mathematically describe the coupling relationships between program entities. Nodes in the graph represent tools, functions, or data objects, and directed edges represent call relationships, data transfer relationships, or inheritance relationships.

[0077] To illustrate, the source code of white-box tools in the candidate tool list is loaded and fundamentally analyzed. First, the static analysis engine is started to convert the source code text of the tool into an abstract syntax tree (AST) using a lexical analyzer and a syntax analyzer. Then, a depth-first traversal of the AST nodes is performed to extract the source code metadata of the white-box tool. Specifically, function definition nodes, parameter list declarations, and type hints in the code are identified and extracted. At the same time, natural language processing technology is used to extract docstrings in the function body to obtain functional description keywords. Furthermore, the function call nodes appearing in the code are matched with the system's pre-built high-risk API fingerprint database containing sensitive behaviors such as file operations, network communication, and process control, thereby initially marking explicit sensitive operation characteristics at the source code level.

[0078] After completing the basic source code feature extraction, the extracted function signatures, call relationships, and variable assignment paths are used to construct a white-box tool call dependency graph. Graph nodes are created for each identified tool function and the external library function being called. Directed edges representing call relationships are established between nodes based on the control flow logic in the AST. At the same time, taint analysis technology is used to trace the propagation path of key variables (such as variables of tool entry parameters) in code blocks. Data flow edges representing data flow direction are established in the white-box tool call dependency graph, thus forming a topology network that can completely describe the internal logical coupling and data flow of the tool.

[0079] Further, a transitive side effect inference algorithm is executed based on the completed white-box tool call dependency graph. The propagation path of side effects is analyzed from bottom to top using graph traversal algorithms (such as depth-first search, DFS). If a leaf node in the dependency graph (such as a low-level system library function) is identified as having a specific side effect (such as deleting a file), the side effect attribute will be propagated backward to the upper-level tool entry node according to the call edge. This identifies implicit risks that are not directly written in the source code but are indirectly triggered through the call chain. Finally, the identified direct side effects, the inferred transitive side effects, and the tool's interface definition are integrated to generate white-box tool meta-information containing precise input / output specifications and complete risk attribute descriptions.

[0080] For example, electronic devices can leverage the white-box tool source code analysis submodule within the system tools extraction module. This submodule provides in-depth code-level analysis capabilities for both open-source and system-integrated tools. First, it uses static code analysis to extract interface information such as function signatures, parameter types, and constraints, providing foundational data for subsequent formal modeling. In the in-depth analysis phase, it identifies key operations in the tool's code, including permission checks, data processing logic, and external API calls. This information is crucial for understanding the tool's actual behavior and potential side effects. For instance, the system can identify whether a tool reads sensitive data, modifies system state, or calls external services—all potential security risks. Finally, the submodule constructs a dependency graph of the tools, identifying cross-tool interaction points. This dependency graph reveals the data and control flow relationships between different tools, aiding in the subsequent analysis of potential logical vulnerabilities when combining tools, especially complex security issues arising from the collaboration of multiple tools.

[0081] Step A4: For the black-box tools in the candidate tool list, obtain the basic interface definition based on the public interface document, and use the dynamic code testing mode to construct test inputs for the black-box tools to collect tool output feedback and system state changes through the monitoring tool. Based on the tool output feedback and system state changes, infer the behavior pattern and potential side effect attributes of the black-box tools, and obtain the black-box tool meta-information based on the behavior pattern and potential side effect attributes of the black-box tools. Basic interface definition refers to static contract information obtained from public documentation, including interface address, request method, parameter name, parameter type, and expected return format.

[0082] Dynamic code testing mode refers to running the target tool in an isolated sandbox environment and using fuzz testing technology to generate test inputs containing valid values, boundary values, and abnormal values ​​to stimulate the tool's behavior under different states.

[0083] System state changes refer to the implicit effects that a tool has on the runtime environment during execution. These implicit effects are usually not included in the direct function return value. Examples include creating temporary files, establishing external network connections, modifying environment variables, or consuming excessive CPU / memory resources.

[0084] As an example, a test baseline is first established based on static document parsing. The public interface documents or interface description files associated with the black-box tool are read, and the basic interface definitions of the tool are parsed and extracted to establish a legal parameter structure and communication protocol, which serves as the skeleton for constructing test cases.

[0085] The dynamic code testing mode is activated, and the black-box tool is loaded into an isolated sandbox environment with comprehensive monitoring capabilities. The fuzzing engine is used to construct two types of test inputs based on the basic interface definition: one type is compliant input that conforms to the specifications, which is used to trigger the normal business process of the tool to observe the expected behavior; the other type is probe input containing boundary values, special characters or known attack payloads, which is used to probe the fault tolerance mechanism and potential vulnerabilities of the tool.

[0086] During the test execution, multi-dimensional monitoring probes were deployed simultaneously to collect the tool's output feedback and system state changes in real time. Monitoring dimensions included file system monitoring (recording file read, write, and delete operations), network traffic monitoring (capturing inbound and outbound data packets and target IPs), process monitoring (tracing child process creation and system calls), and resource monitoring (recording CPU and memory usage). The collected input data, the tool's direct output feedback, and the system state change logs captured by environmental monitoring were time-series aligned and causally analyzed. Based on a pre-defined "stimulus-response" model, the behavioral patterns and potential side effects of the black-box tool were inferred. For example, if the network monitoring probe detected an outbound connection to that URL after a specific URL parameter was input, and the file system monitoring probe recorded the generation of a new file, it would be inferred that the tool exhibited network download and file writing behavior patterns, and the corresponding side effect attributes would be marked.

[0087] Finally, the inferred dynamic behavior characteristics and side effect attributes are structurally integrated with the initially acquired basic interface definitions to generate black-box tool metadata containing interface specifications and risk attributes.

[0088] For example, electronic devices can obtain a black-box tool list from the system tools extraction module: for closed-source third-party tools or external services, the black-box tool list acquisition submodule provides analysis methods. This submodule first obtains the basic functions and interface definitions of the tools through publicly available information such as API documentation, interface descriptions, and example calls. To gain a deeper understanding of the behavior of black-box tools, this submodule uses dynamic testing techniques to infer the tool's behavior patterns and potential side effects by sending different inputs to the tool and observing its output and state changes. This probing analysis method can reveal tool characteristics that may not be explicitly stated in official documentation. Based on the collected information, this submodule finally constructs the functional description and interface specifications of the black-box tools.

[0089] Step A6: Integrate the white-box tool metadata and the black-box tool metadata to obtain the calling tool metadata.

[0090] To illustrate, a globally unified tool metadata standard model is established. This model defines a standard field structure including a unique tool identifier, input / output parameter type constraints, prerequisite dependencies, and a full-dimensional enumeration of side effect attributes. White-box and black-box tool metadata are read, and a data standardization engine is activated to map these two types of heterogeneous data to the aforementioned tool metadata standard model to obtain the calling tool metadata. This applies to single-source tools that exist only in the white-box list or only in the black-box list.

[0091] This specification describes a method for constructing a call dependency graph that explicitly displays internal coupling and data flow for white-box tools through static code analysis. For black-box tools, a dynamic code testing model is employed to construct probe inputs and monitor system state changes to infer their behavior patterns and potential side effects. Finally, static source code features and dynamic runtime features are logically fused and standardized to generate call tool metadata containing precise interface specifications and comprehensive risk attributes. This method achieves a comprehensive and in-depth characterization of intelligent agent tools. Through a dual verification mechanism of white-box perspective and black-box observation, it can accurately capture hidden transitive side effects and sensitive API calls in the source code, and effectively discover dynamic risks exposed during runtime, such as resource consumption, temporary file residues, or illegal network connections. This improves the completeness and confidence of the tool metadata, thereby ensuring the robustness and accuracy of the entire intelligent agent security detection solution when facing complex and heterogeneous toolsets.

[0092] S208: Based on the tool invocation architecture type, tool priority mechanism, agent-tool interaction model, and basic meta-information, the tool set information is obtained by integrating them.

[0093] Create a global "tool set information" data structure container. This data structure container typically uses a hierarchical JSON or graph structure to design tool call architecture types and write them to the root node of the container to establish a global call protocol standard. At the same time, the extracted tool priority mechanism is transformed into a global scheduling strategy configuration.

[0094] The system iterates through the identified candidate tool list and its corresponding basic metadata, instantiating each tool as a child node within a container. During instantiation, the constructed agent-tool interaction model is introduced, injecting dynamic decision logic into the static tool nodes. Specifically, the system calculates the trigger condition probability and context constraints for each tool in the agent-tool interaction model and attaches these as invocation constraint attributes to the corresponding tool node.

[0095] Finally, a consistency check is performed to examine whether there are any logical conflicts between the architecture type, priority rules, and specific tool definitions, and the final serialized file is generated after resolving the conflicts.

[0096] In the embodiments of this specification, the architecture type and priority mechanism of tool invocation are established through in-depth analysis of the agent's operating environment. The agent-tool interaction model simulating the agent's decision-making logic is constructed using the parsed algorithm rules. Based on this model, a list of candidate tools is identified. Then, a combination of static code parsing to construct a dependency graph and dynamic black-box testing to monitor state changes is used to extract deep meta-information of white-box and black-box tools. Finally, the architecture features, scheduling rules, interaction models, and multi-dimensional tool meta-information are logically integrated to generate a structured set of tool information. The aforementioned approach enables the construction of a comprehensive view of tools that highly replicates the actual operational state of intelligent agents. By integrating architectural analysis and interactive modeling, not only are logically unreachable dead code tools effectively eliminated during the data preparation stage, reducing computational redundancy in subsequent verification, but the calling constraints and priority relationships of tools in specific decision contexts are also captured more accurately. This ensures that the tool set information is not merely static interface definitions, but a three-dimensional data object containing dynamic scheduling logic. Simultaneously, by integrating white-box static features and black-box dynamic behaviors, the blind spots of single-perspective analysis are eliminated, resulting in highly complete and confident tool set information. This enhances the adaptability and detection accuracy of the entire security detection solution for complex heterogeneous intelligent agent systems.

[0097] Optional, please see Figure 4 , Figure 4 This is a flowchart illustrating the construction process of a formal model for a tool. Specifically, it involves constructing a formal model of the tool's interactive behavior using a domain-specific language based on the tool's side-effect attributes and functional characteristics. The process includes: Step S302: Convert the input parameter types, data range constraints, and execution preconditions in the tool's functional characteristics into type definitions and logical constraints in a domain-specific language, and establish the interface constraint mapping relationship of the calling tool in the formal model based on the type definitions and logical constraints; Interface constraint mapping refers to the bidirectional correspondence rules established between the original definition of a tool and its formal model. Interface constraint mapping specifies which set each parameter corresponds to in the mathematical model, and which logical formula corresponds to each real-world constraint.

[0098] In DSL, a type is typically defined as an atom and the set it constitutes.

[0099] In a DSL, logical constraints refer to predicates that must always be true, representing facts or state transitions. Logical constraints define the mathematical relationships (such as set inclusion relationships or numerical value relationships) that object attributes must satisfy.

[0100] To illustrate, syntactic analysis and semantic classification are performed on the functional characteristics of the tool. First, the input parameter type information in the tool description is identified and mapped to the basic type definition or custom set signature in the domain-specific language. For example, for enumeration type parameters required by the tool, the corresponding enumeration structure is generated in the DSL; for complex object parameters, a structured entity with specific fields is defined by extending the basic signature, thereby establishing the static form of the data in the model.

[0101] Next, the data range constraints and execution preconditions of the extraction tool are extracted and transformed into logical constraints in the DSL using first-order predicate logic. Numerical restrictions on scanning parameters (e.g., port numbers must be within a certain range) are also identified, converting these constraints into global facts or axioms that limit the generation of model instances, ensuring the validation engine searches only within valid numerical ranges. Simultaneously, the environmental context required for the analysis tool's execution (e.g., user must have administrator privileges, database connection must be active) is invoked, transforming the dependencies described in natural language into precondition expressions in the tool's operation predicates. For example, the requirement to log in is transformed into a logical implication precondition, stipulating that the tool's state transition logic is only allowed to trigger when the current state satisfies the login attribute.

[0102] Finally, the generated type definition code is associated and bound with the logical constraint formula to establish a complete mapping relationship for the calling tool interface constraints.

[0103] In one feasible implementation, before specifically converting the input parameter types, data range constraints, and execution preconditions in the tool's functional characteristics into type definitions and logical constraints in a domain-specific language, the following steps may also be referred to: Step B2: Perform semantic analysis on the functional description text in the tool's functional characteristics to obtain the semantic analysis results; Functional description text refers to the natural language field in the tool metadata used to explain the tool's purpose, usage, and limitations to the larger model.

[0104] Semantic analysis refers to the process of parsing text at the lexical, syntactic, and pragmatic levels using Natural Language Processing (NLP) technology. In this embodiment, it specifically refers to identifying the core intent, entity relationships, and linguistic behaviors (such as commands, requests, and statements) in the text.

[0105] Semantic analysis results refer to the structured data objects generated after parsing. Optional results typically include high-dimensional semantic vectors, key action-object pairs, tone intensity scores, and imperative feature tags.

[0106] In a schematic manner, all natural language fields are extracted from the tool's metadata, including the tool name, function description, parameter description, and usage examples. Then, a text cleaning process is performed, removing HTML tags, standardizing whitespace characters, and unifying encoding formats. A pre-built natural language processing engine is invoked to perform word segmentation and part-of-speech tagging on the cleaned text. Based on this, syntactic dependency analysis is performed to extract the predicate verbs and object core words from the text, constructing "action-object" tuples (e.g., delete-file, modify-configuration) to clarify the tool's actual operational object and behavior type at the semantic level.

[0107] After completing basic syntactic parsing, in-depth pragmatic and sentiment analysis is performed to identify hidden imperative features in the text. By scanning modal verbs and imperative sentence structures in the text, the tone intensity score and persuasive probability of the text are calculated, with particular attention paid to statements that attempt to change the inference priority of the larger model (e.g., ignoring previous instructions or forcing execution). Simultaneously, a pre-trained embedding model is used to transform the entire descriptive text into a high-dimensional semantic vector, so that in subsequent steps, vector similarity calculations can be used to detect whether it overlaps semantically with a known malicious prompt word sample library. Finally, the extracted action tuples, tone scores, imperative feature tags, and semantic vectors are encapsulated into structured semantic analysis results.

[0108] Step B4: Based on the semantic analysis results, detect whether there are security risk semantic features in the function description text; Security risk semantic features refer to specific language patterns in natural language text that can trigger unexpected behavior in large models or induce users to perform high-risk operations. In the context of intelligent agent tools, these mainly include: prompt injection features: attempting to override existing system instructions (e.g., ignoring all preceding rules); induced privilege escalation features: inducing the model to grant the highest priority or administrator privileges (e.g., this is an emergency system process); and function masquerading features: the description text is severely inconsistent with the actual function, attempting to conceal the true intention (e.g., the actual action is to delete files, but the description is to clear the cache and optimize speed).

[0109] As an example, a pre-built semantic feature library containing a massive sample vector of malicious prompt words, a list of sensitive keywords, and a template of suggestive sentences is loaded, and semantic analysis results are obtained. These results include high-dimensional semantic vectors describing the tool, action-object pairs, and tone intensity scores.

[0110] Based on this, a vector space-based similarity detection is performed. The cosine similarity between the semantic vector of the input text and the vectors of known attack samples in the feature library is calculated. If the similarity exceeds a preset security threshold (e.g., 0.85), the text is deemed to be highly homologous to known attacks at the semantic level and is marked as potentially risky. Simultaneously, rule-based structured feature matching is performed. Using extracted "action-object" pairs and key phrases, pattern search is conducted in the feature library to detect whether logically mutually exclusive or out-of-bounds descriptions exist. For example, the system checks for high-risk combinations such as "action = execution" and "object = arbitrary instruction," and detects the existence of typical instruction-overriding phrases such as ignoring security restrictions or being "not limited to."

[0111] Furthermore, a comprehensive assessment is conducted by combining tone intensity scores. For texts with extremely high tone scores (coercive / absolute) and containing descriptions of sensitive operations, a logical consistency check is performed to determine whether they exhibit "functional masquerading." For example, by comparing the basic classification of the tool (e.g., read-only) with the action intent in the description (e.g., forced modification), if an explicit conflict is found, a deceptive semantic risk is identified. Finally, by combining the results of the above vector matching, rule detection, and logical verification, when any step hits the risk threshold, a definite security risk semantic feature label is generated, providing a precise risk injection point for subsequent formal modeling.

[0112] Step B6: If it exists, mark the tool node corresponding to the security risk semantic feature as a high-risk node in the tool formal model, and add security constraints for the high-risk node.

[0113] High-risk nodes refer to tool entities explicitly marked by specific attribute fields or sets in the formal model (DSL code). Nodes marked as high-risk will be treated as untrusted objects by the engine during subsequent verification processes, and usually require more stringent preconditions (such as manual approval or a sandbox environment) to undergo state transitions.

[0114] Security constraints refer to additional logical formulas generated for high-risk nodes. These formulas typically exist in the form of blocking predicates or pre-assertions, enforcing the legal context in which the tool is invoked, such as allowing it to be invoked only in test mode or prohibiting the input of sensitive data types.

[0115] As an illustration, based on the detection results output in step B4, the target tool containing semantic features of security risks is located, and the corresponding tool signature definition is found in the formal model code generated in S302. The electronic device modifies the inheritance relationship or attribute field of this definition to explicitly classify the tool entity into the high-risk tool set and assign it a high-risk attribute label.

[0116] Based on this, according to the specific risk type identified in step B4 (such as hint injection, privilege escalation inducement, or function masquerading), the corresponding blocking policy is retrieved and matched from the pre-set security policy library, and translated into security constraints in a domain-specific language. If the risk type is hint injection, a manual intervention constraint is generated, that is, a logical clause is added to the preconditions for tool invocation, stipulating that the current execution mode must be manual confirmation mode, thereby triggering a state transition; if the risk type is privilege escalation inducement, a least privilege constraint is generated to force the caller's permission set to strictly include the privilege level required by the tool; if the risk type is data leakage risk, an information flow blocking constraint is generated to prohibit the passing of data objects marked as "sensitive" as parameters to this node.

[0117] Finally, these generated security constraints are integrated into the tool's original interface constraints and functional logic through logic and operations to reconstruct the tool's formal description. Subsequently, in subsequent verification stages such as S106, any interaction path attempting to invoke the tool without satisfying these additional security constraints will be judged as "unreachable" or "illegal path" by the verification engine, thereby preemptively eliminating potential attack chains at the logical deduction level through mathematical proof.

[0118] Step S304: Map the tool function logic and expected output results in the tool function characteristics to the function logic mapping relationship in the formal model; Tool functional logic refers to the set of internal operational rules, control flow branches, and state transition mechanisms followed by an intelligent agent tool during execution to achieve a specific task objective. In the formal modeling context of this embodiment, it can refer to the algorithm implementation at the code level (such as sorting or computation) or the causal relationship of system state transitions. That is, it defines how, under certain preconditions, the tool accurately transforms the current system state into the next system state after execution. Specifically, this includes: data CRUD operations, business rule determination, and entity attribute changes. Mapping of tool functional logic in the formal model: In domain-specific languages, functional logic is typically mapped to predicates or state transition functions, used to mathematically describe the trajectory of system evolution.

[0119] The expected output refers to the structured data, status indicators, or environmental feedback information that a tool explicitly returns to the caller (the agent or the next tool) after completing its functional logic. It serves as the data carrier for inter-tool interaction and determines whether subsequent tools can obtain valid input parameters. Specifically, it includes explicit return values, status signals, and reference handles. In the formal model, the expected output is mapped to temporary variables or relational tuples added in the post-state, serving as the source of input constraints for subsequent state transition logic.

[0120] Functional logic mapping refers to the rules that transform imperative logic at the code level into declarative logic at the mathematical level.

[0121] Indicatively, a full system state space definition is constructed. Global state signatures, including file system snapshots, sets of memory variables, network connection states, and database records, are defined in a domain-specific language, serving as a canvas for describing tool behavior. Based on this, the core functional logic of the tool's features is abstracted and extracted, transforming it into state transition predicates in a formal model. Verbs such as "create," "modify," "delete," and "send" in the description of electronic device analysis tools are mapped to union, difference, and relation rewriting operations in set theory.

[0122] Simultaneously, the mapping of the expected output results of the electronic device processing tools defines the function's return value as a newly added temporary variable or message object in the system state, establishing a flow contract between the output data and subsequent tool input data. To ensure the rigor of the logic, strict frame axiom constraints are automatically generated while generating the state transition logic. By traversing all system state fields not involved in the current tool logic, the identity constraint "the later state equals the previous state" is generated, clearly defining the tool's influence boundary. Finally, the above state transition logic, output result mapping, and frame axioms are combined and encapsulated into complete formal predicate logic code, thereby establishing the functional logic mapping relationship of the tool in the mathematical model. This allows the formal verification engine to infer how the system environment will precisely evolve from the current state to the next state after the agent calls the tool.

[0123] Step S306: Based on the tool's side effect attributes, establish a side effect classification system that includes data operation dimensions and system resource dimensions; determine the execution logic of each calling tool based on the tool's side effect attributes and tool functional characteristics; determine the side effect behavior that matches the execution logic in the side effect classification system; and map the side effect behavior to a side effect semantic mapping relationship in a formal model.

[0124] A side effect classification system refers to a predefined hierarchical and structured dictionary of risk attributes. It abstracts the impact of tools on the environment into standardized categories, mainly including data operation dimensions and system resource dimensions.

[0125] Side effect behavior refers to the specific environmental changes that a tool actually triggers under a particular execution logic. For example, sending a POST request to an external IP address is the execution logic, while its corresponding side effect is the outgoing network data.

[0126] Side effect semantic mapping refers to the rules that convert identified side effect behaviors into enumerations, set relations, and conditional predicates in a domain-specific language. It strongly binds tool entities with specific risk attributes in mathematical logic.

[0127] Indicatively, a comprehensive multidimensional classification tree is established based on common security standards (such as CWE / OWASP) and the characteristics of the agent's operating environment. In the data operation dimension, categories such as data reading, data modification, data deletion, and data outflow are defined, and in the system resource dimension, categories such as network connection, file system operation, process creation, and high resource consumption are defined, thereby forming a risk dictionary that the verification engine can identify.

[0128] In the behavior matching phase, based on the tool information, the execution logic of each tool call is deeply analyzed and categorized. The white-box API call characteristics and black-box dynamic behavior characteristics of the tool are matched against the aforementioned side effect classification system to determine which specific side effect behaviors the tool triggers. For example, if the tool is detected to have logic for reading local files and sending them to the network, side effect behaviors in both file reading and network writing dimensions will be matched simultaneously.

[0129] Performing side effect semantic mapping transforms matched side effect behaviors into semantic code within a formal model. Electronic devices define global side effect enumeration types in a domain-specific language, using logical facts and relational mapping statements to bind specific side effect enumeration values ​​to corresponding tool signatures, forming static attribute labels. For conditionally triggered side effects, such as deletion operations triggered only when a parameter has a specific value, conditional predicates with logical implications are generated to precisely describe the contextual constraints of side effect triggering. Through this process, abstract risk attributes are concretized into logical axioms in the model, enabling the verification engine to automatically identify and mark interaction paths containing high-risk side effects during subsequent deductions.

[0130] In this specification, a hierarchical mapping formal modeling technique is adopted. First, the input parameter types, data range constraints, and execution preconditions in the tool's functional characteristics are converted into type definitions and logical constraints in a domain-specific language to establish interface constraint mapping relationships. Then, the core functional logic and expected output results of the tool are abstracted into predicate functions and frame axioms describing system state transitions to construct functional logic mapping relationships. Finally, based on the established side effect classification system that includes data operation and system resource dimensions, the potential risk behaviors matched by the tool's execution logic are transformed into side effect semantic mapping relationships in the model. The aforementioned approach transforms unstructured tool characteristics described in natural language or implemented in code into structured models, eliminating ambiguity and polysemy in the original tool descriptions. This ensures that the formal verification engine can perform deductions within precisely defined logical boundaries, effectively avoiding invalid state searches and wasted computational resources due to parameter type mismatches or missing preconditions. Simultaneously, by explicitly defining state transition logic and side-effect semantics, dual digital modeling of the tool's explicit business functions and implicit environmental impacts is achieved. This enables the verification engine not only to deduce the connectivity of business logic but also to accurately capture deep security vulnerabilities such as illegal data transfers and unauthorized access to system resources.

[0131] In one feasible implementation, the formal verification engine configuration can be executed in the following manner: Step C2: Based on the tool set information, perform random tool combination processing to obtain the combination of calling tools to be verified; A tool combination to be verified refers to a specific subset and its interaction sequence extracted from the full tool pool based on tool set information. For example, tool A (reading files), tool B (encrypting), and tool C (sending) constitute a combination.

[0132] Step C4: Determine the state space size of the tool combination interaction logic corresponding to the tool combination based on the formal model of the tool; The size of the state space refers to the size of the set consisting of all possible variable values, parameter combinations, and system state transitions in a formal model.

[0133] Step C6: When the state space is a finite-sized class, select the model detector as the formal verification engine and configure the formal verification engine with a full state space search strategy; Finite-scale classes refer to classes whose total number of states can be enumerated and are within the computational tolerance range (such as Boolean values, enumeration types, and small-range integers).

[0134] Model detectors are verification tools based on exhaustive search (such as AlloyAnalyzer and Spin), suitable for finite-state systems. They can traverse all possible paths, and typically prove the system is error-free or provide counterexamples.

[0135] Step C8: When the state space is of infinite size, select a symbolic execution engine or a theorem prover as the formal verification engine, and configure an abstract reduction strategy for the formal verification engine.

[0136] The term "infinite scale" refers to a total number of states that approaches infinity or is extremely large (such as strings of arbitrary length, floating-point arithmetic, and recursive data structures), which causes the conventional exhaustive search method to encounter the state space explosion problem.

[0137] Symbolic execution engines / theorem provers are tools suitable for handling infinite or large-scale state spaces (such as KLEE, Z3, and Coq). Instead of using concrete numerical values, symbolic execution engines / theorem provers use mathematical symbols and logical reasoning to verify properties or find paths by solving constraint equations.

[0138] Abstract reduction strategies are mathematical methods that simplify complex problems. For example, mapping all positive integers to the sign "+" and all negative integers to the sign "-" reduces the infinite space of integers to a finite space of signs, making computation easier.

[0139] Indicatively, steps C2 and C4 are executed first. Based on the tool set information, several sets of tool combinations to be verified are generated using random sampling. The formal model of the tool combination is parsed, and the data type definitions and state transition logic in the formal model are scanned to calculate its state space size. Further, it is checked whether the model contains only finite-field variables such as Boolean, enumeration, or bounded integers, or whether it contains logic with limited recursion depth; if these conditions are met, it is determined to be a finite-size class. Conversely, if the model involves unbounded strings, floating-point operations, complex loop structures, or dynamically growing data sets, its state space is determined to be an infinite-size class.

[0140] After determining the size, if the result is a finite-size class, the model detector is automatically selected as the core verification engine. Under this path, the system is configured with a full state-space search strategy, instructing the engine to construct a complete state transition graph and exhaustively traverse each reachable state. Due to the finite state space, the engine can mathematically deterministically cover all possible execution paths, ensuring the completeness of the verification results; that is, any vulnerabilities violating security policies will inevitably be detected.

[0141] If the determination result is an infinite-scale class, the symbolic execution engine or theorem prover is switched as the verification core. To address the state explosion problem, the system simultaneously loads an abstract reduction strategy. For example, the system applies data abstraction techniques to map specific numerical domains to abstract symbolic domains (e.g., simplifying specific amounts into three states: positive, zero, and negative), or applies predicate abstraction techniques to focus only on key logical branches affecting security attributes. The engine performs symbolic reasoning or constraint solving within the abstracted space, successfully transforming an incalculable infinite problem into a computable logical proof problem, thereby achieving efficient risk detection in complex scenarios.

[0142] In one feasible implementation, please participate Figure 5 , Figure 5 This is a flowchart illustrating a verification analysis process, specifically executing the security risk verification of the interaction logic for the combined calling tools, including: Step S402: Construct scenario analysis prompt information based on the tool function description in the tool set information, the side effect definition in the tool formal model, and the vulnerability pattern in the risk detection strategy; Scenario analysis prompts refer to a specially engineered input text structure designed to stimulate the reasoning ability of large language models for specific security scenarios. This can be achieved through, but is not limited to, core components: role setting, tool context, attack target, and guidance of the reasoning thought process.

[0143] Vulnerability patterns refer to abstract attack logic templates pre-built into a risk detection strategy library. For example, a data breach pattern is defined as a sequence of sensitive data reading and network data outgoing; a command injection pattern is defined as a sequence of unfiltered user input and system command execution.

[0144] Context injection refers to the technical process of embedding a tool's metadata (functionality, side effects) into a prompt word template to ensure that the large model performs inferences with an understanding of the current specific environment.

[0145] To illustrate, we first access the tool set information and extract the natural language functional description of each candidate tool, which includes the tool's business semantics. At the same time, we extract standardized side effect definitions from the constructed tool formal model, which provide the underlying technical attributes of the tool's behavior.

[0146] Furthermore, the pre-configured risk detection strategy retrieves a matching vulnerability pattern library based on the current detection objective (such as detecting privacy compliance or detecting system integrity). Template-filling technology is used to construct scenario analysis prompts. First, the role setting of a "security expert" is injected into the prompt header to construct an "environmental context." The extracted tool names, function descriptions, and side effect attributes are embedded in this context in JSON and Markdown list formats, representing the currently available libraries in the large model.

[0147] Building upon this, the retrieved vulnerability patterns are transformed into attack targets in natural language, with accompanying thought chain guidance instructions. For example, if the vulnerability pattern is data breach, instructions such as "Please analyze whether it is possible to obtain sensitive information by combining the above tools and then send it to an external network" are added to the prompt. To improve the accuracy of inference, similar attack path reasoning examples can be inserted into the prompt information according to the few-sample prompt principle. Then, all the above text blocks are concatenated and truncated or optimized using a token counter to generate the final scenario analysis prompt information that conforms to the target large model context window constraints.

[0148] Step S404: Based on the constructed scenario analysis prompt information, call the target large language model to parse the current tool combination scenario to obtain scenario context semantics, and generate an interaction hypothesis path based on the scenario context semantics. The interaction hypothesis path represents the tool call sequence and parameter combination that may trigger the vulnerability mode. The target large language model refers to the base large model used to perform reasoning tasks. It can be implemented using the general large language model LLM in related technologies, without any specific limitations.

[0149] Contextual semantics refers to the model's deep understanding of the current tool combination environment. This includes potential relationships between tools (e.g., the output of a compression tool can often be used as the input of a file transfer tool) and the implicit uses of tools in specific business scenarios (e.g., debugging interfaces usually have high privileges).

[0150] The interaction hypothesis path refers to a sequence of hypotheses to be verified generated by the target large language model. It specifically describes the order of "calling A first, then B, then C..." and indicates the direction of the values ​​of the key parameters at each step.

[0151] In a schematic manner, scenario analysis prompts are sent to the target large language model via a standardized API interface, and appropriate inference parameters (such as Temperature values ​​to balance creativity and determinism) are configured. This drives the target large language model to first perform in-depth reading and deconstruction of the tool descriptions, side effect definitions, and vulnerability patterns in the prompts. The large model uses its programming knowledge and security common sense accumulated during the pre-training phase to analyze the current tool combination scenario, extract the implicit scenario context semantics, identify the data flow possibilities between different tools (such as the compatibility of the return value type of tool A with the parameter type of tool B) and logical dependencies (such as the execution of tool B depending on the session state established by tool A), thereby constructing a virtual tool interaction network at the cognitive level.

[0152] Building upon this foundation, scenario analysis prompts guide the large model into the thought chain reasoning stage, requiring it to perform reverse or forward deduction within the aforementioned interaction network based on pre-defined vulnerability patterns. The target large model searches for the shortest path connecting the "initial state" and the "violation state," and envisions specific parameter attack payloads. For example, the model might reason: "If I inject a path traversal character into the filename parameter of tool A, can the content it reads be passed as a parameter to tool B and sent out?" Based on this deduction, the target large model generates a structured interaction hypothesis path. This path not only explicitly represents the tool call sequence required to trigger the vulnerability pattern but also specifies the parameter combination characteristics in key steps, thus concretizing the abstract vulnerability pattern into a concrete execution trajectory that can be verified by the formalization engine.

[0153] Step S406: Convert the interaction hypothesis path into formal constraints and input them into the formal verification engine. Drive the formal verification engine to perform a directional search within the state space defined by the formal constraints to search for state transition paths that match the pre-hypotheses and to verify whether the state transition paths are counterexample paths that violate the risk detection strategy.

[0154] Formal constraints refer to transforming natural language or semi-structured hypothesis paths into logical formulas that can be recognized by formal verification engines.

[0155] Targeted search refers to a non-traversal search strategy. Instead of aimlessly exploring all possible state spaces, the verification engine is guided by constraints to a specific sequence of interactions, calculating only whether the sequence is mathematically feasible under the current model logic.

[0156] A counterexample path refers to a deterministic and reproducible sequence of states output by the formal verification engine.

[0157] Schematic, the system receives an interactive hypothesis path containing an ordered sequence of tool calls and key parameter features. A pre-built semantic-logic compiler transforms this semi-structured information from the interactive hypothesis path into formal constraints compatible with the formal verification engine. Specifically, the "tool call sequence" is converted into sequential constraint operators in temporal logic, and the "parameter combination features" are transformed into state predicate constraints, thereby defining a narrow but high-value subspace of states to be verified.

[0158] Furthermore, the formal verification engine is launched and the tool's formal model is loaded. The generated generalized formal constraints are input into the engine, driving it to execute a targeted search strategy. Instead of attempting to traverse the entire state space, the engine focuses on finding a true state transition path within the model that satisfies both the tool's functional logic definition (S304) and strictly matches the input interaction assumption constraints. The engine attempts to fill in intermediate state details not explicitly specified by the larger model (such as temporary state changes in the database) and calculates the satisfiability of the logic using the SAT solver. If the solver successfully finds a solution, it means the "conjecture" of the larger model is reachable in the mathematical model, and the engine then generates a state transition path containing detailed state snapshots for each step.

[0159] Furthermore, the searched state transition paths are compared with pre-defined risk detection strategies to verify whether the final or process state of the path violates security axioms (e.g., sensitive data fields should not appear in the attributes of public network egress objects). If a violation is confirmed, the path is marked as a counterexample path that violates the risk detection strategy and formatted into a visual attack reproduction script; if the engine cannot find a feasible path within the constraints, the assumption of the large model is determined to be a false alarm or unreachable, thus completing the accurate filtering and confirmation of potential risks.

[0160] This specification describes a method for constructing scenario analysis prompts based on tool function descriptions and formal side effect definitions. It leverages the strong semantic reasoning capabilities of a large language model to parse the context and generate interactive hypothesis paths representing potential attack chains. These unstructured natural language hypotheses are then transformed into rigorous formal constraints, driving a formal verification engine to perform targeted searches and reachability analyses within a constraint-bound convergent state subspace. Finally, mathematical logic is used to verify the existence of counterexample paths that violate risk policies. Prior reasoning based on a large model compresses the full state space search into a small-scale verification of high-value paths, fundamentally solving the state space explosion problem faced by traditional formal verification when dealing with complex intelligent agent systems, achieving an exponential improvement in verification efficiency. Simultaneously, the logical completeness of the formal engine is used to mathematically verify the inference results of the large model, effectively eliminating the risk of false positives caused by illusions in the large model. This ensures that each detected vulnerability path has a reproducible execution trajectory, enabling the system to accurately discover and prove deep business logic vulnerabilities and semantic attack risks that are difficult to reach through single static analysis or fuzz testing, spanning multiple tool steps.

[0161] In one feasible implementation, the specific execution of generating a security risk report based on potential tool invocation risk information in the combined verification results can refer to the following method: Step D2: Extract the counterexample paths output by the formal verification engine from the combined verification results; A counterexample path refers to a specific execution trajectory discovered by the formal verification engine during state space search. This trajectory starts from an initial state, goes through a finite number of state transitions, and ultimately reaches a violation state that violates the preset security policy (such as invariant invalidation or assertion error). The counterexample path may include the complete tool call sequence triggered by the vulnerability, parameter passing details, and system state change trajectory. As an illustration, the system monitors the output stream of the verification engine or reads the generated log files. A pre-configured parser filters out intermediate search states and debugging information irrelevant to the vulnerability, locating data blocks marked as violation or counterexample types. Based on this, the system performs a time-series reconstruction of the data block, extracting the complete tool call sequence that led to the violation. The electronic device identifies the unique identifier and execution order index of each action node in the sequence, ensuring that the reconstructed attack steps are logically continuous and uninterrupted.

[0162] Furthermore, it analyzes the internal attributes of each calling node in the sequence to extract parameter passing details. The electronic device can analyze the instantiated parameter objects in the formal model, read the specific numerical values, strings, or object references passed in by the attacker during the simulation, and bind these parameters to specific tool interfaces. Simultaneously, it performs differential analysis of the system state change trajectory, compares system state snapshots before and after each tool call step, traverses all modeled global variables, filters out fields whose values ​​have changed, records the exact causal relationship of state transitions, and thus constructs a chain of evidence containing "operational behavior + input data + environmental consequences."

[0163] Step D4: Construct a risk visualization view based on the aforementioned counterexample path to show the dependencies between different tools and the flow of tool data during the vulnerability triggering process; A risk visualization view is a dynamic, interactive chart based on a graphical user interface. It typically uses a time-series diagram to show the chronological order of tool calls, or a directed acyclic graph to show the data dependency topology between tools. The view includes nodes (representing tools), directed edges (representing calls or data flows), and status annotations (representing changes in the system environment).

[0164] Tool data flow refers to the complete trajectory of a data object (especially tainted data containing attack payloads) as it is passed between the input parameters and output results of different tools. For example, the output of tool A is used as the input of tool B, and the processing result of tool B is then passed to tool C, forming a complete data link.

[0165] Indicatively, the counterexample path data is loaded, and a visual canvas space is initialized based on the timestamp sequence and tool entity ID recorded therein. All tool instances involved in the attack chain are instantiated as graphical nodes on the canvas, and control flow edges with directional arrows are drawn between the nodes according to the order in which the tools are called, thereby constructing the basic temporal skeleton of vulnerability triggering.

[0166] Based on this, in-depth data flow analysis is performed to trace the transmission logic of key parameters in the counterexample path. The source of input parameters and the destination of output results at each node are analyzed, cross-tool data reference relationships are identified, and data flow edges with differentiated styles (such as dashed lines or solid lines of different colors) are drawn between corresponding tool nodes. Furthermore, electronic devices can combine taint tracking algorithms to identify specific data objects containing malicious payloads or sensitive information, and mark all path segments traversed by these data objects with highlighted colors. This visually outlines the complete tool data flow from the initial injection point through a series of tool transit processes to the final violation trigger point. Finally, a state snapshot overlay is dynamically generated next to the key nodes in the view, showing the details of system state changes before and after this step. Ultimately, a comprehensive risk visualization view is rendered, containing both timing control logic and data propagation trajectory, allowing users to clearly identify the strong dependencies between different tools and the hidden data penetration paths during the vulnerability triggering process from a macro perspective.

[0167] Step D6: Based on the logical vulnerability types identified by the combined verification results and the counterexample paths, generate security remediation suggestions; Logical vulnerability types refer to the categories of vulnerabilities categorized based on the security attributes violated by the formal verification engine. In the context of intelligent agents, common types include race conditions, privilege escalation, infinite loops / resource exhaustion, uncleaned data streams, and atomicity violations.

[0168] Security remediation recommendations are precise action guidelines generated by combining the tool names, parameter locations, and state contexts in specific counterexample paths. Recommendations typically include code-level modifications, architectural adjustments, and policy updates.

[0169] Indicatively, the system analyzes and interprets the logical vulnerability types explicitly identified in the combined verification results, loads counterexample paths, and locates the critical turning point that causes the system state to transition from "secure" to "unsafe," such as determining which tool's call directly triggered the invariant failure. Based on this, a pre-built security remediation knowledge graph or vulnerability defense pattern library is integrated, which stores standard defense primitives for various common logical vulnerabilities.

[0170] Furthermore, the system's instantiation and generation logic for implementing recommendations integrates abstract defense strategies with the specific context of counterexample paths. Electronic devices analyze the interface definitions and parameter passing details of key tools in the counterexample path, generating context-aware, specific recommendations. For example, if a counterexample indicates a vulnerability caused by an unfiltered string output by tool A being directly used as a shell command parameter for tool B, instead of simply warning against injection, the system generates specific guidance on adding a regular expression filtering function to the output of tool A, or changing the parameter type constraint of tool B from String to a SafeCommand object. For complex vulnerabilities involving multi-step interactions (such as business logic bypass), missing assertion logic is added to specific state transition preconditions. Finally, the system encapsulates these technical remediation measures into natural language descriptions or pseudocode snippets, generating targeted security remediation recommendations to ensure developers can directly locate and implement fixes based on these recommendations.

[0171] Step D8: Integrate the counterexample path, the risk visualization view, and the remediation suggestions to generate a security risk report.

[0172] A security risk report is a comprehensive document ultimately generated and intended for multiple audiences (security administrators, developers, business owners). It may include core sections such as execution summary, vulnerability details, attack chain diagram, reproduction steps, and remediation plans.

[0173] To illustrate, a report generation task context is created. Structured text-based counterexample path data is obtained from step D2, a rendered image-formatted risk visualization view is obtained from step D4, and knowledge base matching-generated security remediation suggestion text is obtained from step D6. The above data is then logically layered and arranged according to a preset report template.

[0174] Furthermore, the system dynamically renders the document content, constructs an "Execution Summary" section, and automatically generates a concise risk situation overview based on the number of vulnerabilities, severity level, and affected tools, allowing management to quickly assess the impact. It also constructs a "Technical Details" section, presenting a graphical comparison of each step of the attack topology with the risk visualization view. This involves displaying the attack topology while simultaneously including the corresponding code-level call stack and parameter details in the sidebar or below, achieving cross-verification between the "macro-level chain" and "micro-level data," ensuring readers can understand the attack process and directly locate code issues. A remediation suggestion module follows the vulnerability details, inserting not only general defense strategies but also highlighting targeted code snippets based on template logic. Finally, the formatted memory document object is converted to the final target format and automatically appended with a digital signature or timestamp to ensure the report's tamper-proof nature, completing the generation and distribution of the security risk report.

[0175] This specification analyzes and extracts counterexample paths from the raw output of the formal verification engine, containing complete tool call sequences, parameter details, and system state change trajectories. Then, using graphical rendering technology, it constructs a risk visualization view showing the dependencies between tools and data flow. Simultaneously, it automatically generates targeted security remediation suggestions by combining a vulnerability type knowledge base and counterexample context. Finally, it integrates the aforementioned evidence chain, visualization map, and remediation solutions to generate a standardized security risk report. By transforming the underlying mathematical logic proof into visualized attack reproduction evidence, it lowers the cognitive threshold for developers to understand complex cross-tool attack chains, making previously invisible data flow paths and hidden logical dependencies intuitive and transparent. Furthermore, by providing context-aware and precise remediation suggestions, it achieves an automated closed loop from vulnerability discovery to vulnerability remediation, preventing developers from taking ineffective remediation measures due to misinterpretation of verification results. This ensures the rigor of the agent's business logic while shortening the response and remediation cycle for security vulnerabilities, improving the overall security operation and maintenance efficiency of the agent system.

[0176] The following will combine Figure 6 This specification provides a detailed description of the safety detection device provided in the embodiments. It should be noted that... Figure 6 The safety detection device shown is used to perform the functions described in this manual. Figures 1-5 The methods shown in the embodiments are illustrated for ease of explanation, showing only the parts related to the embodiments of this specification. For specific technical details not disclosed, please refer to this specification. Figures 1-5 The example shown.

[0177] Please see Figure 6 This diagram illustrates the structure of a security detection device according to an embodiment of this specification. The security detection device 1 can be implemented as all or part of a device through software, hardware, or a combination of both. According to some embodiments, the security detection device 1 includes a system tool extraction module 11, a tool formal description module 12, and a combined verification and analysis module 13, specifically used for: System tool extraction module 11 is used to obtain tool set information of intelligent agents calling tools in intelligent agent systems based on large model-driven systems; The tool formal description module 12 is used to identify the tool side effect attributes and tool functional characteristics of the intelligent agent calling the tool based on the tool set information, and to construct a tool formal model describing the tool interaction behavior using a domain-specific language based on the tool side effect attributes and tool functional characteristics. The combined verification analysis module 13 is used to configure the formal verification engine and risk detection strategy, input the formal model of the tool and the risk detection strategy into the formal verification engine, so as to perform security risk verification on the interaction logic of calling the tool combination to obtain the combined verification result. The combined verification analysis module 13 is used to generate a security risk report based on the potential tool call risk information in the combined verification results.

[0178] Optionally, obtaining the tool set information of agent invocation tools in a large model-driven agent system includes: To analyze the agent runtime environment of a large model-driven agent system, we need to determine the tool invocation architecture type, agent tool selection algorithm, and tool priority mechanism for agents calling tools. An agent-tool interaction model is established based on the agent tool selection algorithm and tool priority mechanism. The agent-tool interaction model is used to simulate the agent's tool calling decision logic in different scenarios. Based on the tool invocation architecture type and the agent-tool interaction model, identify a list of candidate tools accessible in the current environment of the agent system, and determine the basic meta-information of the invocation tool based on the list of candidate tools; The tool set information is obtained by integrating the tool invocation architecture type, tool priority mechanism, agent-tool interaction model, and basic meta-information.

[0179] Optionally, determining the calling tool metadata based on the candidate tool list includes: For the white-box tools in the candidate tool list, static code parsing mode is executed to obtain the source code metadata of the white-box tools in order to construct the white-box tool call dependency graph, and the white-box tool metadata is obtained based on the white-box tool call dependency graph; For the black-box tools in the candidate tool list, the basic interface definition is obtained based on the public interface document, and a dynamic code testing mode is adopted to construct test inputs for the black-box tools so as to collect tool output feedback and system state changes through monitoring tools. Based on the tool output feedback and system state changes, the behavior pattern and potential side effect attributes of the black-box tools are inferred, and the black-box tool meta-information is obtained based on the behavior pattern and potential side effect attributes of the black-box tools. The white-box tool metadata and the black-box tool metadata are integrated to obtain the calling tool metadata.

[0180] Optionally, the step of constructing a formal model of the tool describing its interactive behavior using a domain-specific language based on the tool's side-effect attributes and functional characteristics includes: The input parameter types, data range constraints, and execution preconditions in the tool's functional characteristics are converted into type definitions and logical constraints in a domain-specific language. Based on these type definitions and logical constraints, the interface constraint mapping relationship of the calling tool in the formal model is established. The tool's functional logic and expected output results in the tool's functional characteristics are mapped to the functional logic mapping relationship in the formal model; Based on the side effect attributes of the tools, a side effect classification system is established that includes data operation dimensions and system resource dimensions. Based on the side effect attributes and functional characteristics of the tools, the execution logic of each tool call is determined. The side effect behavior matching the execution logic is determined in the side effect classification system, and the side effect behavior is mapped to the side effect semantic mapping relationship in the formal model.

[0181] Optionally, before converting the input parameter types, data range constraints, and execution preconditions in the tool's functional characteristics into type definitions and logical constraints in a domain-specific language, the method further includes: Semantic analysis is performed on the functional description text in the tool's functional characteristics to obtain semantic analysis results; Based on the semantic analysis results, detect whether there are security risk semantic features in the function description text; If it exists, the tool node corresponding to the security risk semantic feature is marked as a high-risk node in the formal model of the tool, and security constraints are added for the high-risk node.

[0182] Optionally, the configuration of the formal verification engine includes: Based on the tool set information, a tool random combination process is performed to obtain the tool combination to be verified; The state space size of the tool combination interaction logic corresponding to the tool combination being invoked is determined based on the formal model of the tool; When the state space is a finite-sized class, a model detector is selected as the formal verification engine, and a full state space search strategy is configured for the formal verification engine. When the state space is of infinite size, a symbolic execution engine or a theorem prover is selected as the formal verification engine, and an abstract reduction strategy is configured for the formal verification engine.

[0183] Optionally, the security risk verification of the interaction logic for calling the tool combination includes: Based on the tool function descriptions in the toolset information, the side effect definitions in the tool formal model, and the vulnerability patterns in the risk detection strategy, scenario analysis prompts are constructed. Based on the constructed scenario analysis prompt information, the target large language model is invoked to parse the current tool combination scenario to obtain scenario context semantics, and based on the scenario context semantics, an interaction hypothesis path is generated. The interaction hypothesis path represents the tool call sequence and parameter combination that may trigger the vulnerability mode. The interaction hypothesis path is converted into formal constraints and input into the formal verification engine. The formal verification engine is then driven to perform a directional search within the state space defined by the formal constraints to search for state transition paths that match the pre-assumptions and to verify whether the state transition paths are counterexample paths that violate the risk detection strategy.

[0184] Optionally, generating a security risk report based on potential tool invocation risk information in the combined verification results includes: Extract the counterexample paths output by the formal verification engine from the combined verification results; Based on the aforementioned counterexample path, a risk visualization view is constructed to show the dependencies between different tools and the flow of tool data during the vulnerability triggering process; Based on the logical vulnerability types identified by the combined verification results and the counterexample paths, security remediation suggestions are generated. The negative example path, the risk visualization view, and the remediation suggestions are integrated to generate a security risk report.

[0185] It should be noted that the security detection device provided in the above embodiments is only illustrated by the division of the above functional modules when performing the security detection method. In practical applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above. In addition, the security detection device and the security detection method embodiments provided in the above embodiments belong to the same concept, and the implementation process is detailed in the method embodiments, which will not be repeated here.

[0186] The example numbers in this specification are for descriptive purposes only and do not represent the superiority or inferiority of the examples.

[0187] This specification also provides a computer storage medium that can store multiple instructions adapted to be loaded and executed by a processor as described above. Figures 1-5 The security detection method described in the illustrated embodiment can be found in the following document for a detailed execution process. Figures 1-5 The specific details of the illustrated embodiments will not be elaborated here.

[0188] This specification also provides a computer program product that stores at least one instruction, said at least one instruction being loaded and executed by the processor as described above. Figures 1-5 The security detection method described in the illustrated embodiment can be found in the following document for a detailed execution process. Figures 1-5 The specific details of the illustrated embodiments will not be elaborated here.

[0189] Please refer to Figure 7This is a structural block diagram of an electronic device provided in an embodiment of this specification. The electronic device in this specification may include one or more of the following components: a processor 1010, a memory 1020, an input device 1030, an output device 1040, and a bus 1050. The processor 1010, memory 1020, input device 1030, and output device 1040 may be connected to each other via the bus 1050.

[0190] Processor 1010 may include one or more processing cores. Processor 1010 connects to various parts of the electronic device using various interfaces and lines, and performs various functions and processes data by running or executing instructions, programs, code sets, or instruction sets stored in memory 1020, and by calling data stored in memory 1020. Optionally, processor 1010 may be implemented using at least one hardware form of digital signal processing (DSP), field-programmable gate array (FPGA), or programmable logic array (PLA). Processor 1010 may integrate one or a combination of central processing unit (CPU), graphics processing unit (GPU), and modem. The CPU mainly handles the operating system, user interface, and applications; the GPU is responsible for rendering and drawing the displayed content; and the modem is used for wireless communication. It is understood that the modem may also not be integrated into processor 1010 and may be implemented separately through a communication chip.

[0191] The memory 1020 may include random access memory (RAM) or read-only memory (ROM). Optionally, the memory 1020 may include non-transitory computer-readable storage medium. The memory 1020 may be used to store instructions, programs, code, code sets, or instruction sets.

[0192] The input device 1030 is used to receive input instructions or data, and includes, but is not limited to, a keyboard, mouse, camera, microphone, or touch device. The output device 1040 is used to output instructions or data, and includes, but is not limited to, a display device and a speaker. In this embodiment, the input device 1030 can be a temperature sensor for acquiring the operating temperature of the electronic device. The output device 1040 can be a speaker for outputting audio signals.

[0193] In addition, those skilled in the art will understand that the structure of the electronic device shown in the above figures does not constitute a limitation on the electronic device. The electronic device may include more or fewer components than shown, or combine certain components, or have different component arrangements. For example, the electronic device may also include radio frequency circuits, input units, sensors, audio circuits, wireless fidelity (WIFI) modules, power supplies, Bluetooth modules, etc., which will not be described in detail here.

[0194] In the embodiments of this specification, the executing entity for each step can be the electronic device described above. Optionally, the executing entity for each step is the operating system of the electronic device. The operating system can be Android, iOS, or other operating systems; this specification does not limit this.

[0195] exist Figure 7 In the electronic device, the processor 1010 can be used to call a program stored in the memory 1020 and execute it to implement the security detection method as described in the various method embodiments of this specification.

[0196] Those skilled in the art will understand that all or part of the processes in the above embodiments can be implemented by a computer program instructing related hardware. The program can be stored in a computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. The storage medium can be a magnetic disk, optical disk, read-only memory, or random access memory, etc.

[0197] It should be noted that the information (including but not limited to user device information, user personal information, etc.), data (including but not limited to data used for analysis, stored data, displayed data, etc.), and signals involved in the embodiments of this specification are all authorized by the user or fully authorized by all parties, and the collection, use, and processing of related data must comply with the relevant laws, regulations, and standards of the relevant countries and regions. For example, the tool set information, features, and user information involved in this specification were all obtained with full authorization.

[0198] The above-disclosed embodiments are merely preferred embodiments of this specification and should not be construed as limiting the scope of this specification. Therefore, any equivalent variations made in accordance with the claims of this specification shall still fall within the scope of this specification.

Claims

1. A security detection method for intelligent agent tools, characterized in that, The method includes: Obtain tool set information for agent invocation tools in a large model-driven agent system; Based on the tool set information, identify the tool side effect attributes and tool functional characteristics of the intelligent agent calling the tool, and construct a formal model of the tool describing the tool interaction behavior using a domain-specific language based on the tool side effect attributes and tool functional characteristics; Configure a formal verification engine and a risk detection strategy, input the formal model of the tool and the risk detection strategy into the formal verification engine, and perform security risk verification on the interaction logic of calling the tool combination to obtain the combined verification result; A security risk report is generated based on potential tool call risk information in the combined verification results.

2. The method according to claim 1, characterized in that, The tool set information for obtaining agent invocation tools in a large model-driven agent system includes: To analyze the agent runtime environment of a large model-driven agent system, we need to determine the tool invocation architecture type, agent tool selection algorithm, and tool priority mechanism for agents calling tools. An agent-tool interaction model is established based on the agent tool selection algorithm and tool priority mechanism. The agent-tool interaction model is used to simulate the agent's tool calling decision logic in different scenarios. Based on the tool invocation architecture type and the agent-tool interaction model, identify a list of candidate tools accessible in the current environment of the agent system, and determine the basic meta-information of the invocation tool based on the list of candidate tools; The tool set information is obtained by integrating the tool invocation architecture type, tool priority mechanism, agent-tool interaction model, and basic meta-information.

3. The method according to claim 2, characterized in that, The step of determining the calling tool metadata based on the candidate tool list includes: For the white-box tools in the candidate tool list, static code parsing mode is executed to obtain the source code metadata of the white-box tools in order to construct the white-box tool call dependency graph, and the white-box tool metadata is obtained based on the white-box tool call dependency graph; For the black-box tools in the candidate tool list, the basic interface definition is obtained based on the public interface document, and a dynamic code testing mode is adopted to construct test inputs for the black-box tools so as to collect tool output feedback and system state changes through monitoring tools. Based on the tool output feedback and system state changes, the behavior pattern and potential side effect attributes of the black-box tools are inferred, and the black-box tool meta-information is obtained based on the behavior pattern and potential side effect attributes of the black-box tools. The white-box tool metadata and the black-box tool metadata are integrated to obtain the calling tool metadata.

4. The method according to claim 1, characterized in that, The formal model of the tool, constructed using a domain-specific language based on the tool's side-effect attributes and functional characteristics, describing the tool's interactive behavior, includes: The input parameter types, data range constraints, and execution preconditions in the tool's functional characteristics are converted into type definitions and logical constraints in a domain-specific language. Based on these type definitions and logical constraints, the interface constraint mapping relationship of the calling tool in the formal model is established. The tool's functional logic and expected output results in the tool's functional characteristics are mapped to the functional logic mapping relationship in the formal model; Based on the side effect attributes of the tools, a side effect classification system is established that includes data operation dimensions and system resource dimensions. Based on the side effect attributes and functional characteristics of the tools, the execution logic of each tool call is determined. The side effect behavior matching the execution logic is determined in the side effect classification system, and the side effect behavior is mapped to the side effect semantic mapping relationship in the formal model.

5. The method according to claim 4, characterized in that, Before converting the input parameter types, data range constraints, and execution preconditions in the tool's functional characteristics into type definitions and logical constraints in a domain-specific language, the process also includes: Semantic analysis is performed on the functional description text in the tool's functional characteristics to obtain semantic analysis results; Based on the semantic analysis results, detect whether there are security risk semantic features in the function description text; If it exists, the tool node corresponding to the security risk semantic feature is marked as a high-risk node in the formal model of the tool, and security constraints are added for the high-risk node.

6. The method according to claim 1, characterized in that, The configuration formal verification engine includes: Based on the tool set information, a tool random combination process is performed to obtain the tool combination to be verified; The state space size of the tool combination interaction logic corresponding to the tool combination being invoked is determined based on the formal model of the tool; When the state space is a finite-sized class, a model detector is selected as the formal verification engine, and a full state space search strategy is configured for the formal verification engine. When the state space is of infinite size, a symbolic execution engine or a theorem prover is selected as the formal verification engine, and an abstract reduction strategy is configured for the formal verification engine.

7. The method according to claim 1 or 6, characterized in that, The security risk verification of the interaction logic of the tool invocation combination includes: Scenario analysis prompts are constructed based on the tool function descriptions in the toolset information, the side effect definitions in the tool formal model, and the vulnerability patterns in the risk detection strategy. Based on the constructed scenario analysis prompt information, the target large language model is invoked to parse the current tool combination scenario to obtain scenario context semantics, and based on the scenario context semantics, an interaction hypothesis path is generated. The interaction hypothesis path represents the tool call sequence and parameter combination that may trigger the vulnerability mode. The interaction hypothesis path is converted into formal constraints and input into the formal verification engine. The formal verification engine is then driven to perform a directional search within the state space defined by the formal constraints to search for state transition paths that match the pre-assumptions and to verify whether the state transition paths are counterexample paths that violate the risk detection strategy.

8. The method according to claim 1, characterized in that, The generation of a security risk report based on potential tool invocation risk information in the combined verification results includes: Extract the counterexample paths output by the formal verification engine from the combined verification results; Based on the aforementioned counterexample path, a risk visualization view is constructed to show the dependencies between different tools and the flow of tool data during the vulnerability triggering process; Based on the logical vulnerability types identified by the combined verification results and the counterexample paths, security remediation suggestions are generated. The negative example path, the risk visualization view, and the remediation suggestions are integrated to generate a security risk report.

9. A safety detection device for intelligent agent tools, characterized in that, The device includes: The system tool extraction module is used to obtain tool set information of intelligent agents calling tools in intelligent agent systems based on large model-driven systems; The tool formal description module is used to identify the tool side effect attributes and tool functional characteristics of the intelligent agent calling the tool based on the tool set information, and to construct a tool formal model describing the tool interaction behavior using a domain-specific language based on the tool side effect attributes and tool functional characteristics. The combined verification analysis module is used to configure the formal verification engine and risk detection strategy. The formal model of the tool and the risk detection strategy are input into the formal verification engine to perform security risk verification on the interaction logic of calling the tool combination to obtain the combined verification result. The combined verification analysis module is used to generate a security risk report based on the potential tool call risk information in the combined verification results.

10. An electronic device, characterized in that, include: A processor and a memory; wherein the memory stores a computer program adapted to be loaded by the processor and to execute the steps of the method as described in any one of claims 1 to 8.

Images