Method, apparatus and device for security check of an agent and storage medium

Abstract

Embodiments of the present disclosure relate to a security verification method and apparatus for an agent, a device, and a storage medium. The method comprises: generating a first structured representation describing a dependency relationship in a task running process of the agent based on runtime sequence data corresponding to the task running process; the first structured representation comprising a plurality of running nodes corresponding to a plurality of processing functions and data called in the task running process; determining respective security description information of the plurality of running nodes in the first structured representation based on at least a tool security configuration related to available tools of the agent and a data security configuration related to accessible data objects of the agent; and performing security verification on the task running process of the agent based on the respective security description information of the plurality of running nodes. Thus, accurate modeling and effective verification of the agent running process can be achieved, thereby improving the coverage and accuracy of risk identification.

Description

Technical Field

[0001] The exemplary embodiments disclosed herein generally relate to the field of computer technology, and more specifically, to a method, apparatus, device, and storage medium for security verification of intelligent agents. Background Technology

[0002] Intelligent agents, as an extension of machine learning models, significantly enhance their ability to handle complex tasks by integrating functions such as tool invocation, memory management, and inference planning. They hold great promise for applications in office automation and intelligent interaction. However, compared to machine learning models, intelligent agents, with their more complex architecture and operational logic, expose a larger attack surface and thus face more severe security challenges. Summary of the Invention

[0003] In a first aspect of this disclosure, a method for security verification of an intelligent agent is provided. The method includes: generating a first structured representation describing dependencies during at least one task execution based on runtime sequence data corresponding to at least one task execution process of the intelligent agent; wherein the first structured representation includes multiple runtime nodes corresponding to: multiple processing functions invoked during at least one task execution process, and data during at least one task execution process; determining corresponding security description information for the multiple runtime nodes in the first structured representation based at least on tool security configurations related to the available tools of the intelligent agent and data security configurations related to the accessible data objects of the intelligent agent; and performing security verification on the at least one task execution process of the intelligent agent based on the corresponding security description information of the multiple runtime nodes.

[0004] In a second aspect of this disclosure, an apparatus for security verification of an intelligent agent is provided. The apparatus includes: a structured representation generation module configured to generate a first structured representation describing dependencies in at least one task execution process based on runtime sequence data corresponding to at least one task execution process of the intelligent agent; wherein the first structured representation includes multiple execution nodes corresponding to: multiple processing functions invoked in at least one task execution process, and data in at least one task execution process; a security description information determination module configured to determine corresponding security description information for the multiple execution nodes in the first structured representation based at least on tool security configurations related to the available tools of the intelligent agent and data security configurations related to accessible data objects of the intelligent agent; and a security verification module configured to perform security verification on at least one task execution process of the intelligent agent based on the corresponding security description information of the multiple execution nodes.

[0005] In a third aspect of this disclosure, an electronic device is provided. The device includes at least one processor; and at least one memory coupled to the at least one processor and storing instructions for execution by the at least one processor. When executed by the at least one processor, the instructions cause the device to perform the method of the first aspect.

[0006] In a fourth aspect of this disclosure, a computer-readable storage medium is provided. The computer-readable storage medium stores computer-executable instructions that can be executed by a processor to implement the method of the first aspect.

[0007] In a fifth aspect of this disclosure, a computer program product is provided. The computer program product includes computer-executable instructions that, when executed by a processor, implement the method according to a first aspect of this disclosure.

[0008] It should be understood that the content described in this content section is not intended to limit the key or essential features of the embodiments of this disclosure, nor is it intended to restrict the scope of this disclosure. Other features of this disclosure will become readily apparent from the following description. Attached Figure Description

[0009] The above and other features, advantages, and aspects of the embodiments of this disclosure will become more apparent from the accompanying drawings and the following detailed description. In the drawings, the same or similar reference numerals denote the same or similar elements, wherein:

[0010] Figure 1 A schematic diagram of an example environment according to an embodiment of the present disclosure is shown;

[0011] Figure 2 A flowchart illustrating an example process for security verification of an intelligent agent according to some embodiments of this disclosure is shown;

[0012] Figure 3 A schematic diagram of an example architecture for security verification of an intelligent agent according to some embodiments of the present disclosure is shown;

[0013] Figure 4 A schematic structural block diagram of a device for security verification of an intelligent agent according to some embodiments of the present disclosure is shown; and

[0014] Figure 5 A block diagram of an electronic device in which one or more embodiments of the present disclosure may be implemented is shown. Detailed Implementation

[0015] Embodiments of this disclosure will now be described in more detail with reference to the accompanying drawings. While some embodiments of this disclosure are shown in the drawings, it should be understood that this disclosure can be implemented in various forms and should not be construed as limited to the embodiments set forth herein. Rather, these embodiments are provided to provide a more thorough and complete understanding of this disclosure. It should be understood that the accompanying drawings and embodiments of this disclosure are for illustrative purposes only and are not intended to limit the scope of protection of this disclosure.

[0016] It should be noted that the headings of any section / subsection provided herein are not limiting. Various embodiments are described throughout this document, and embodiments of any type may be included under any section / subsection. Furthermore, embodiments described in any section / subsection may be combined in any way with any other embodiments described in the same section / subsection and / or different sections / subsections.

[0017] In the description of embodiments of this disclosure, the term "comprising" and similar terms should be understood as open-ended inclusion, i.e., "including but not limited to". The term "based on" should be understood as "at least partially based on". The term "one embodiment" or "the embodiment" should be understood as "at least one embodiment". The term "some embodiments" should be understood as "at least some embodiments". Other explicit and implicit definitions may also be included below. The terms "first", "second", etc., may refer to different or the same objects. Other explicit and implicit definitions may also be included below.

[0018] The embodiments of this disclosure may involve user data, data acquisition, and / or use. All of these aspects comply with applicable laws, regulations, and relevant provisions. In the embodiments of this disclosure, all data collection, acquisition, processing, manipulation, forwarding, and use are conducted with the user's knowledge and confirmation. Accordingly, in implementing the embodiments of this disclosure, the type, scope of use, and usage scenarios of any data or information that may be involved should be communicated to the user and their authorization obtained in accordance with relevant laws and regulations through appropriate means. The specific methods of notification and / or authorization may vary depending on the actual situation and application scenario, and the scope of this disclosure is not limited in this respect.

[0019] In this specification and the embodiments, any processing of personal information will be carried out only under the premise of legality (such as obtaining the consent of the personal information subject, or being necessary for the performance of a contract), and will only be carried out within the scope stipulated or agreed upon. A user's refusal to process personal information beyond what is necessary for basic functions will not affect the user's use of basic functions.

[0020] As briefly described above, intelligent agents face severe security challenges. For example, in scenarios where intelligent agents interact across domains (e.g., different business domains), the functions that intelligent agents rely on, such as tool invocation and memory management, are often distributed across business domains with different trust levels. In this case, highly sensitive information may flow from a high-trust domain to a low-trust domain (e.g., user account information used to log in to untrusted websites). Furthermore, malicious commands may infiltrate from a low-trust domain to a high-trust domain, leading to attacks such as privacy leaks and command poisoning. Simultaneously, due to the "blind execution" characteristic of intelligent agents, they lack the ability to introspect on commands. In this situation, attackers can disguise malicious commands as data injection into the execution process, thereby tampering with normal logic.

[0021] Based on differences in technical approaches, current security protection solutions can be broadly categorized into three types: model input / output filtering solutions (such as firewalls based on machine learning models), access control solutions, and trusted execution environment solutions. In model input / output filtering solutions, threats in the input and output content can be identified through vector similarity, rule matching, or machine learning models. In access control solutions, functions such as tool calls and memory management can be abstracted into entities or objects, thereby verifying the legitimacy of actions through an access control system. In trusted execution environment solutions, hardware isolation technology can be used to create a secure execution environment, thereby ensuring data security during the inference phase of the machine learning model.

[0022] The aforementioned solutions all rely on programming language syntax. However, intelligent agents are driven by natural language instructions, whose semantics are ambiguous, execution plans are dynamic, and tool calls span multiple trust domains. These characteristics prevent the aforementioned security solutions from directly and accurately analyzing the behavior of intelligent agents, thus resulting in limited security effectiveness.

[0023] Embodiments of this disclosure provide a scheme for security verification of an intelligent agent. According to this scheme, firstly, based on runtime sequence data corresponding to at least one task execution process of the intelligent agent, a first structured representation describing the dependencies in the at least one task execution process is generated. The first structured representation includes multiple nodes, which correspond to: multiple processing functions invoked during the at least one task execution process, and data during the at least one task execution process. Next, based at least on tool security configurations related to the available tools of the intelligent agent and data security configurations related to the accessible data objects of the intelligent agent, corresponding security description information for the multiple runtime nodes in the first structured representation is determined. Subsequently, based on the corresponding security description information of the multiple runtime nodes, security verification is performed on the at least one task execution process of the intelligent agent.

[0024] As will be more clearly understood from the following description, embodiments of this disclosure can map previously unquantifiable factors (such as natural language instructions, cross-tool calls, and side effects) into a structured representation (e.g., a first structured representation) based on runtime sequence data corresponding to at least one task execution process of the agent, thereby achieving accurate modeling and verifiable description of the task execution process. Furthermore, embodiments of this disclosure introduce prior security knowledge related to agent behavior during security verification through tool security configuration and data security configuration. In this way, potential risk paths and complex interaction scenarios that are difficult to identify using traditional solutions can be effectively covered, thereby significantly improving the coverage and accuracy of risk identification.

[0025] The following will further describe in detail various example implementations of this scheme with reference to the accompanying drawings. Figure 1 A schematic diagram of an example environment 100 according to an embodiment of the present disclosure is shown. (Refer to...) Figure 1 Example environment 100 may include electronic device 110, intelligent agent 120, and machine learning model 130. It should be understood that the structure and function of the various elements in environment 100 are described herein for illustrative purposes only and do not imply any limitation on the scope of this disclosure.

[0026] In example environment 100, agent 120 can perform reasoning and decision-making based on input instructions from a user or other system, using machine learning model 130, to complete a specific task. Agent 120 can be an application extension of machine learning model 130. For example, during at least one task execution, agent 120 can invoke corresponding tools or interfaces to assist in task completion. Machine learning model 130 can receive data from agent 120 as model input. Machine learning model 130 can be, for example, a large language model (LLM) or a large model capable of handling multimodal input (i.e., a multimodal large model). Machine learning model 130 can process the model input using trained model parameters to provide suggestions or guidance for the subsequent behavior of agent 120. Electronic device 110 can monitor at least one task execution of agent 120 and perform security checks on that at least one task execution. Electronic device 110 can present interface 150 to user 140. The security verification result of the electronic device 110 on the intelligent agent 120 can be displayed on the interface 150 for the user 140 to view and confirm.

[0027] In some embodiments, electronic device 110 may be any type of mobile terminal, fixed terminal, or portable terminal, including mobile phones, desktop computers, laptop computers, notebook computers, netbook computers, tablet computers, media computers, multimedia tablets, personal communication system (PCS) devices, personal navigation devices, personal digital assistants (PDAs), audio / video players, digital cameras / camcorders, positioning devices, television receivers, radio receivers, e-book devices, gaming devices, or any combination thereof, including accessories and peripherals of these devices or any combination thereof. In some embodiments, electronic device 110 may also support any type of user-facing interface (such as "wearable" circuitry).

[0028] Alternatively, in some embodiments, electronic device 110 may be a standalone physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server providing basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, content delivery networks, and big data and artificial intelligence platforms. Electronic device 110 may, for example, include computing systems / servers such as mainframes, edge computing nodes, computing devices in a cloud environment, etc.

[0029] A communication connection can be established between electronic device 110 and intelligent agent 120. This communication connection can be established via wired or wireless means. The communication connection may include, but is not limited to, Bluetooth, mobile network, Universal Serial Bus, and Wi-Fi connections; the embodiments of this disclosure are not limited in this respect. In the embodiments of this disclosure, electronic device 110 and intelligent agent 120 can achieve signaling interaction through their communication connection.

[0030] Figure 2 A flowchart of an example process 200 for security verification of an intelligent agent according to some embodiments of the present disclosure is shown. Figure 3 A schematic diagram of an example architecture 300 for security verification of an intelligent agent according to some embodiments of the present disclosure is shown. The following is in conjunction with... Figure 1 and Figure 3 Process 200 is described below. Process 200 can be implemented at electronic device 110 to perform security verification of at least one task execution process of intelligent agent 120.

[0031] Reference Figure 2In block 210, electronic device 110 (e.g., structured representation generation module 301 in electronic device 110) generates a structured representation (e.g., a first structured representation) describing the dependencies in at least one task execution process based on runtime sequence data corresponding to at least one task execution process of agent 120. In embodiments of this disclosure, the dependencies in at least one task execution process can be dependencies between various factors in that at least one task execution process. These factors can be processing functions invoked and data accessed (e.g., reading, modifying, adding, generating, etc.) in that at least one execution process. For example, the first structured representation includes multiple execution nodes, which correspond to the following: multiple processing functions invoked in the at least one task execution process, and data in the at least one task execution process. The multiple execution nodes and the edges between the multiple execution nodes can indicate the aforementioned dependencies.

[0032] At least one task execution process of agent 120 can refer to the execution process of a series of tasks performed by agent 120 in one or more interactions with a user. These tasks can include any appropriate reasoning or decision-making task, etc. For example, in each task execution process, agent 120 can invoke multiple processing functions. Multiple processing functions can include prompt processing functions, machine learning model 130 invocation functions (e.g., large models, such as LLM or multimodal large models), decision-making (also known as thought) functions, action execution functions, tool invocation functions, and observation information acquisition functions, etc. The runtime sequence data corresponding to at least one task execution process of agent 120 can include: state information and / or behavioral data generated during task execution, etc. State information can include, for example, the input of the processing function, the output of the processing function, the availability of the processing function, etc. Behavioral data can include, for example, whether the processing function was successfully invoked, whether the processing function was successfully completed, etc. The runtime sequence data in each task execution process can be represented in a serialized form. In some embodiments, runtime sequence data may include context generated during task execution, and context represented in serialized form may also be referred to as a context sequence.

[0033] The first structured representation can be a structured data form of dependencies during at least one task execution. A running node is the basic unit in the first structured representation. Dependencies during at least one task execution can be represented by edges connecting multiple running nodes. In some embodiments, the first structured representation can be a graph structure representation. Alternatively, in some embodiments, the first structured representation can be a tree structure representation, and so on. In some embodiments, the electronic device 110 can abstract dependencies during at least one task execution into a first structured representation using any suitable structured representation generation method.

[0034] In some embodiments, dependencies during at least one task execution can indicate control relationships between multiple processing functions performed by agent 120 during task execution. Control relationships can, for example, indicate the execution order (or control transfer) and control dependencies between multiple processing functions. In this case, the corresponding execution node in the first structured representation can correspond to a single processing function. Alternatively or additionally, in some embodiments, dependencies during at least one task execution can indicate the data flow of agent 120 during task execution. Data flow can, for example, indicate the direction of data flow across multiple processing functions. In this case, the corresponding execution node in the first structured representation can be the data itself and the processing function that processes that data.

[0035] In some embodiments, the first structured representation can simultaneously indicate the control relationships and data flow directions described above. In this case, the electronic device 110 can generate the first structured representation by fusing multiple structured representations (e.g., a second structured representation and a third structured representation) that respectively indicate the control relationships and data flow directions. For example, the electronic device 110 can generate a second structured representation describing the control relationships between multiple processing functions based on runtime sequence data corresponding to at least one task execution process of the agent 120. Then, the electronic device 110 can generate a third structured representation describing the data flow direction during task execution, at least based on the second structured representation. Subsequently, the electronic device 110 can generate the first structured representation by fusing the second and third structured representations.

[0036] As mentioned earlier, the first structured representation can be a graph structure representation. For example, the first structured representation can be a program dependency graph (PDG). In this case, both the second and third structured representations can be graph structure representations. For example, the second structured representation can be a control flow graph (CFG), and the third structured representation can be a data flow graph (DFG).

[0037] In some embodiments, the second structured representation uses processing functions (also referred to as basic blocks of the agent 120 program) as nodes (hereinafter also referred to as function nodes), and reflects the control relationships between multiple processing functions through edges connecting multiple function nodes. In some embodiments, the electronic device 110 can generate an initial second structured representation by calling a generation function of the second structured representation (e.g., a control flow graph initialization function or other appropriate function). Next, the electronic device 110 can parse the runtime sequence data corresponding to at least one task execution process of the agent 120 to determine the processing functions actually invoked by the agent 120 during task execution. For example, the processing functions actually invoked by the agent 120 during task execution may include prompt word processing functions, machine learning model invocation functions, decision-making functions, action execution functions, tool invocation functions, and observation information acquisition functions, etc. Subsequently, the electronic device 110 can transform these processing functions into function nodes of the second structured representation.

[0038] In some embodiments, the multiple processing functions may include action execution functions. Based on this, the electronic device 110 can determine control paths in the second structured representation for the action execution functions, thereby obtaining the execution backbone of the second structured representation. Specifically, the electronic device 110 can determine multiple functional nodes in the second structured representation corresponding to the multiple processing functions. Then, for the action function node corresponding to the action execution function among the multiple functional nodes, the electronic device 110 can determine at least one control path in the second structured representation including the action function node based on the control relationship between the action execution function and other processing functions besides the action execution function (e.g., decision-making function, prompt word processing function, and observation information acquisition function). For example, the electronic device 110 can connect the action function node and the corresponding other functional nodes with directed edges based on the actual execution order between the action execution function and other processing functions, thereby obtaining a control path. By using the action function node as a key node, it is helpful to identify effective control paths among the multiple functional nodes, thereby improving the generation efficiency of the second structured representation.

[0039] In some embodiments, the electronic device 110 can split an action function node into a first sub-node and a second sub-node. The first sub-node corresponds to the tool used by the action execution function. Such a first sub-node can be represented as Action Tool Name. The second sub-node corresponds to the input parameters of the tool, and such a second sub-node can be represented as Action Tool Param. Multiple processing functions may include decision functions. Based on this, the electronic device 110 can determine the control path between the first sub-node and the decision function node, as well as the control path between the second sub-node and the decision function node, based on the control relationship between the action execution function and the decision function. The decision function node can be the function node corresponding to the decision function among multiple function nodes. In this way, the control relationship between the action execution function and the decision function can be reflected in more detail in the second structured representation, thereby facilitating the accurate generation of the subsequent third structured representation and the first structured representation.

[0040] In some embodiments, the electronic device 110 can parse the actual execution order between the action execution function and the decision function. Further, the electronic device 110 can call a node splitting function or other appropriate function to split the action function node into independent first child nodes and second child nodes. Then, based on the actual execution order between the action execution function and the decision function, the electronic device 110 can connect the first child node and the decision function node, and the second child node and the decision function node, respectively, through directed edges, thereby obtaining the control paths between the first child node and the decision function node, and between the second child node and the decision function node. In some embodiments, in addition to the tool's input parameters, the second child node may also correspond to more parameters of the tool, such as the tool's return value, etc.

[0041] In addition to control paths, in some embodiments, the electronic device 110 may also construct control dependency edges between multiple functional nodes to indicate control dependencies between multiple processing functions. For example, the electronic device 110 may utilize a machine learning model (e.g., an LLM model different from the first machine learning model, also referred to herein as a second machine learning model) to further analyze the control relationships (e.g., control dependencies) between at least some functional nodes in the second structured representation. Furthermore, the electronic device 110 can construct control dependency edges based on the control dependencies. In some embodiments, the multiple processing functions may include an action execution function, a decision function, a cue word processing function, and an observation information acquisition function. Based on this, the control relationship (e.g., control dependency) between at least one of the decision function, cue word processing function, or observation information acquisition function and the action execution function may be determined using the second machine learning model.

[0042] In some embodiments, the electronic device 110 can use the action execution function, as well as the decision-making function, cue word processing function, and observation information acquisition function executed before the action execution function, as model inputs to the second machine learning model. Based on the model inputs, the second machine learning model can utilize a pre-trained inference strategy to output a predictive control dependency relationship between at least one of the decision-making function, cue word processing function, or observation information acquisition function and the action execution function. In addition to the control dependencies between the aforementioned functions, the second machine learning model can also output the control dependency relationship between the first child node and the second child node, and so on. This can be determined according to actual needs, and the embodiments of this disclosure will not be listed in detail here. With the help of the second machine learning model, the electronic device 110 can accurately capture the control dependencies between the aforementioned processing functions during the complex operation of the intelligent agent 120.

[0043] In some embodiments, after determining the control dependencies between processing functions, the electronic device 110 can construct control dependency edges between multiple functional nodes by calling a control dependency construction function or other appropriate functions. During this process, the electronic device 110 can construct control dependency edges between action function nodes and their related function nodes by backtracking through a control dependency analysis algorithm. By introducing control dependency edges, the causal logic of at least one task execution process can be reflected in the second structured representation, such as "which input led to the execution of which action". Combining control paths and control dependency edges, the second structured representation can not only reflect the execution order between multiple processing functions but also further reflect the control dependencies between them. Therefore, the control relationships between multiple processing functions can be comprehensively and accurately described.

[0044] After obtaining the second structured representation, the electronic device 110 can generate a third structured representation based on the second structured representation. In some embodiments, the electronic device 110 can extract at least one functional node related to the data flow from the multiple functional nodes of the second structured representation as part of the multiple data nodes of the third structured representation. Next, the electronic device 110 can create at least one additional node as another part of the multiple data nodes, at least based on external data objects accessed by available tools when invoked. Subsequently, the electronic device 110 can determine the connection relationships between the multiple data nodes based on the data dependencies during at least one task execution process to obtain the third structured representation.

[0045] In some embodiments, the third structured representation can use the data itself and the processing functions that process that data (e.g., tools invoked by a tool invocation function) as nodes (hereinafter also referred to as data nodes), and reflect the data flow through edges connecting multiple data nodes. For example, edges connecting multiple data nodes can represent the flow of data input to tools, the flow of data output by tools, and data dependencies between data, etc.

[0046] In some embodiments, the electronic device 110 may invoke a generation function of the third structured representation (e.g., a data flow graph copy function or other appropriate function) to extract at least one functional node related to the data flow from multiple functional nodes of the second structured representation. For example, the electronic device 110 may extract at least one functional node related to the data flow by removing purely logical functional nodes (e.g., machine learning model call functional nodes and decision functional nodes) from the second structured representation.

[0047] In some embodiments, electronic device 110 can determine the external data objects accessed by the available tools when invoked, based on the metadata of the available tools of agent 120 (which can be configured in tool security configuration 303). In some embodiments, electronic device 110 can identify files and / or links that are not present in the control paths mentioned above but are actually read and written by the available tools as external data objects. Furthermore, electronic device 110 can create corresponding additional nodes based on external data objects by calling the creation function of additional nodes (e.g., side-effect construction function or other appropriate function). In this way, electronic device 110 can dynamically supplement multiple data nodes with external data objects, thereby improving the comprehensiveness of data nodes. In some embodiments, for any data node, electronic device 110 can connect the data node to the data node corresponding to the available tool used to process the data node via data input edges and data output edges.

[0048] In some embodiments, electronic device 110 can determine data dependencies (also referred to as data dependencies) during at least one task execution based on the metadata of accessible data objects (which can be configured in data security configuration 304). Then, based on the data dependencies, electronic device 110 can determine the connection relationships (e.g., data dependency edges) between multiple data nodes by calling a data dependency construction function or other appropriate functions to obtain a third structured representation. Thus, the third structured representation can reflect the data flow process at the available tools, thereby accurately describing the data flow direction of agent 120 during at least one task execution.

[0049] In some embodiments, the electronic device 110 can generate a first structured representation by extracting key control relationships and data flows from the second and third structured representations and simplifying unnecessary content. This allows for a more effective description of the dependencies during at least one task execution of the agent 120.

[0050] In some embodiments, the electronic device 110 can take the second and third structured representations as inputs, and integrate the control dependency edges in the second structured representation, the data input edges, data output edges, and data dependency edges in the third structured representation into the first structured representation by calling the generation function of the first structured representation (e.g., the control flow graph copy function and the data flow graph copy function). Thus, the first structured representation can accurately reflect control causality and data flow in time sequence, thereby better describing the dependencies in at least one task execution process of the intelligent agent 120.

[0051] In this manner, embodiments of this disclosure employ three structured representations to collaboratively model dependencies in at least one task execution process of agent 120. These three structured representations can be, for example, control flow graphs, data flow graphs, and program dependency graphs. Based on the foregoing description, it is clear that embodiments of this disclosure first construct a control flow graph containing execution order and control causality. Subsequently, a data flow graph is derived, retaining only data-related entities. Finally, the dependencies of the two graphs are merged to generate a program dependency graph. In the above process, embodiments of this disclosure also introduce several technical means. For example, node splitting mechanisms (splitting action function nodes into first child nodes and second child nodes), language model-driven dependency inference (using LLM to infer control dependencies between multiple processing functions), and edge type normalization processing (ensuring semantic consistency between different graph structures).

[0052] Through the modeling method described above, the embodiments of this disclosure achieve a three-dimensional, integrated visual representation of "time sequence, control dependency, and data flow." This multi-dimensional, refined, and structured representation not only helps enhance the interpretability of at least one task execution process of the agent 120, but also provides effective support for refined security verification.

[0053] Continue to refer to Figure 2 After determining the first structured representation, in block 220, electronic device 110 (e.g., security verification module 302 in the electronic device) determines corresponding security description information for a plurality of running nodes in the first structured representation, based at least on tool security configuration 303 relating to available tools of agent 120 and data security configuration 304 relating to accessible data objects of agent 120. In some embodiments, tool security configuration 303 and data security configuration 304 may be stored in configuration information management module 305.

[0054] In some embodiments, the available tools may include components that the agent 120 can invoke. Alternatively or additionally, the available tools may include external interfaces that the agent 120 can invoke, and so on. In some embodiments, the accessible data object may include an accessible file of the agent 120. Alternatively or additionally, the accessible data object may include an accessible link of the agent 120, and so on.

[0055] In some embodiments, tool security configuration 303 may include a security identifier (e.g., a first security identifier) ​​for data involved in the available tool. Data security configuration 304 may include a security identifier (e.g., a second security identifier) ​​for accessible data objects. In some embodiments, the security identifier may be, for example, a pre-configured security statement or security label, etc. In some embodiments, at least one of the first security identifier or the second security identifier may indicate a pre-configured security sensitivity (e.g., a security sensitivity statement configured for data involved in the available tool or a security sensitivity statement configured for accessible data objects). Alternatively or additionally, at least one of the first security identifier or the second security identifier may indicate a pre-configured trust domain (e.g., a trust domain statement configured for data involved in the available tool or a trust domain statement configured for accessible data objects). In some embodiments, both tool security configuration 303 and data security configuration 304 may be stored in the form of a registry.

[0056] In some embodiments, the data involved in the available tool may include the input parameters of the available tool. Alternatively or additionally, the data involved in the available tool may include the return values ​​of the available tool. Alternatively or additionally, the data involved in the available tool may include data related to the side effects of the available tool. It should be noted that side effects here can refer to additional effects on the system state or external environment caused by the behavior of the available tool in addition to producing the expected results.

[0057] By introducing tool security configuration 303 and data security configuration 304, embodiments of this disclosure establish a unified prior knowledge system in the security scenario of agent 120. Specifically, tool security configuration 303 can provide a structured description of the input parameters, output results, and potential side effects of each available tool at the workflow granularity, and declare the expected security sensitivity (such as confidentiality level) and trust domain for each piece of data involved. Data security configuration 304 can dynamically maintain all data objects that appear or are generated during at least one task execution of agent 120, continuously recording their source, purpose, and current security label. Tool security configuration 303 and data security configuration 304 collaboratively construct a benchmark for subsequent security verification. Based on this, tool security configuration 303 and data security configuration 304 can combine multi-level confidentiality and integrity label definitions, as well as a trust domain declaration mechanism in execution runtime data, to achieve fine-grained security control over tool invocation and data flow processes.

[0058] In some embodiments, each running node may have corresponding security description information. In some embodiments, the corresponding security description information for multiple running nodes may indicate the node's security level. Alternatively or additionally, in some embodiments, the corresponding security description information for multiple running nodes may indicate node data integrity. Alternatively or additionally, in some embodiments, the corresponding security description information for multiple running nodes may indicate the node's trustworthiness level. Alternatively or additionally, in some embodiments, the corresponding security description information for multiple running nodes may indicate node access constraints related to runtime sequence data (e.g., context). In this way, the security attributes of the running nodes can be comprehensively and accurately described.

[0059] In some embodiments, security description information can be represented by multidimensional security labels. For example, node security level can indicate the confidentiality level of a running node. Node data integrity can indicate the data integrity level of a running node. Node security level and node data integrity can be simultaneously represented by the first dimension of the multidimensional security label. Node trust level can indicate the trust level of the execution environment in which the running node resides. Node trust level can be represented by the second dimension of the multidimensional security label. Node access constraints can indicate fine-grained access constraints that vary with runtime sequence data. For example, node access constraints can indicate "only whitelisted domains are allowed when the parameter is a link," etc. Node access constraints can be represented by the third dimension of the multidimensional security label. In this case, the security description information can also be referred to as a three-dimensional security label.

[0060] In some embodiments, when generating the aforementioned second and third structured representations, the electronic device 110 may first generate initial security description information (hereinafter also referred to as reference security description information) for the functional nodes and data nodes in these two structured representations. Furthermore, during the process of fusing the second and third structured representations to generate the first structured representation, the running nodes in the first structured representation may inherit the security description information of the corresponding nodes in the second and third structured representations. Specifically, for any running node among multiple running nodes (e.g., the first running node), this running node is generated during the fusion of the second and third structured representations by fusing the corresponding functional nodes in the second structured representation and the corresponding data nodes in the third structured representation. Based on this, the electronic device 110 may determine the reference security description information of each corresponding functional node and each corresponding data node based on at least one of tool security configuration 303 or data security configuration 304. Subsequently, the electronic device 110 may determine the security description information of the first running node by inheriting the reference security description information of each corresponding functional node and each corresponding data node during the fusion of the corresponding functional nodes and each corresponding data node.

[0061] In the process of constructing the first structured representation by integrating different structured representations (such as control flow graphs and data flow graphs), the running nodes inherit the security description information of the corresponding functional nodes and data nodes, thereby achieving automatic transfer and consistency maintenance of security attributes. This improves the accuracy and reliability of the structured representation in security analysis. Furthermore, embodiments of this disclosure also support dynamically deriving the security level of the integrated running nodes based on both tool security configuration 303 and data security configuration 304. This approach not only enhances the ability to identify potential risks in complex scenarios but also provides fine-grained data support for subsequent security verification.

[0062] In some embodiments, the node security level, node data integrity, and node trustworthiness (e.g., the first two dimensions of the multidimensional security label) in the security description information may be determined based on content related to at least one of confidentiality, data integrity, and trustworthiness in the tool security configuration 303. Alternatively or additionally, in some embodiments, the node security level, node data integrity, and node trustworthiness (e.g., the first two dimensions of the multidimensional security label) in the security description information may be determined based on content related to at least one of confidentiality, data integrity, and trustworthiness in the data security configuration 304. In some embodiments, node access constraints (e.g., the last dimension of the multidimensional security label) may be determined based on a configurable constraint generation strategy. Alternatively or additionally, in some embodiments, node access constraints may be determined based on metadata of the available tools.

[0063] In some embodiments, the constraint generation strategy can be maintained in the strategy configuration module 306. The node security level, node data integrity, and node trust level can be derived by inference based on security statements or security labels related to confidentiality, data integrity, and trust level in the tool security configuration 303 and data security configuration 304. In some embodiments, node access constraints can be automatically generated based on the constraint generation strategy defined by the user or the metadata of available tools stored in the tool security configuration 303.

[0064] In this way, the first two dimensions of the multidimensional security label are derived from tool security configuration 303 and data security configuration 304, ensuring consistency in the security description. The last dimension of the multidimensional security label can be generated on demand based on constraint generation strategies, thereby enhancing adaptability to complex scenarios.

[0065] In some embodiments, the electronic device 110 can determine the security description information of a third operating node connected to the second operating node among a plurality of operating nodes, based on the security description information of the second operating node among the plurality of operating nodes. In this way, the subsequent operating node can inherit the security description information of the previous operating node, thereby realizing the propagation and accumulation of security attributes in the first structured representation.

[0066] In some embodiments, a fourth operating node among multiple operating nodes is connected to a group of operating nodes within the multiple operating nodes. Based on this, the electronic device 110 can determine the security description information of the fourth operating node based on the security description information of the operating node with the highest security requirements among the group of operating nodes. In this way, when multiple operating nodes converge to the same subsequent node (such as the fourth operating node), the electronic device 110 can determine the security description information of the fourth operating node based on the security description information of the node with the highest security requirements among the group of preceding nodes. This effectively prevents the weakening or degradation of security attributes due to the fusion of multi-source information, thereby ensuring that the protection strength of highly sensitive information is not weakened throughout the entire process.

[0067] Continue to refer to Figure 2 After obtaining the security description information of each running node, in box 230, the electronic device 110 performs security verification on at least one task execution process of the intelligent agent 120 based on the corresponding security description information of multiple running nodes.

[0068] In some embodiments, for any running node, the electronic device 110 can perform corresponding verification operations based on each item in the security description information of that running node to determine whether the running node passes the security verification. In some embodiments, for the fifth running node, if the access process involved in the fifth running node does not meet the node access constraints (e.g., the running node is configured to "only allow access to whitelisted domains", but the actual access object is a non-whitelisted domain), the electronic device 110 can determine that the node access constraints are violated. In this case, the electronic device 110 can determine that the fifth running node has failed the security verification. If the data of the fifth running node is transmitted to the sixth running node among multiple running nodes and the node security level of the sixth running node is lower than that of the fifth running node (e.g., highly confidential data flows to a running node with a low security level), the electronic device 110 can determine that there is a risk of information leakage or unauthorized access. In this case, the electronic device 110 can determine that the fifth running node has failed the security verification. If data from the fifth operating node is transmitted to the sixth operating node, and the sixth operating node has a lower level of trust than the fifth operating node (e.g., highly trusted data flows to an untrusted operating node), electronic device 110 can determine that there is a risk of information leakage or unauthorized access. In this case, electronic device 110 can determine that the fifth operating node has failed security verification.

[0069] This dynamic propagation security verification method based on structured representation can not only cover the complex interaction logic during at least one task execution of agent 120, but also identify and block potential malicious behaviors at an early stage. Therefore, it can significantly improve the overall security of the system.

[0070] In some embodiments, if it is detected that agent 120 executes a new processing function during at least one task execution, electronic device 110 can update the first structured representation based on the new processing function. For example, electronic device 110 can insert the new processing function as a new execution node into the first structured representation. Subsequently, electronic device 110 can update the security verification of at least one task execution of agent 120 based on the updated first structured representation. For example, electronic device 110 can perform security verification in the updated first structured representation based on the security verification method described above. In this way, electronic device 110 can immediately cancel the current call when it discovers new highly sensitive data cross-domain leakage or low-trust instructions driving high-privilege operations, thereby achieving an immediate response to abnormal behavior.

[0071] In some embodiments, if at least one task execution of the intelligent agent 120 fails a security check, the electronic device 110 can mark the execution node that failed the security check in a first structured representation. This marking mechanism can visually identify the location of the violation, which can then be used for subsequent risk analysis and remediation guidance. In some embodiments, the electronic device 110 can also analyze the reasons for failing the security check and generate relevant reports, etc.

[0072] As can be clearly understood from the various embodiments described above, the embodiments of this disclosure provide a full-link security protection framework for natural language-driven intelligent agents 120 (LLM Agents). Tool security configuration 303 and data security configuration 304 provide a unified security verification foundation for the system, while the structured representation and security verification process respectively realize formal modeling and dynamic security assurance of the agent 120's behavior. The embodiments of this disclosure can solve the problem that traditional solutions lack formal behavioral descriptions and systematic security verification mechanisms when facing intelligent agents 120 with characteristics such as natural language driving, dynamically changing execution paths, and cross-trust domain calls. The embodiments of this disclosure map fuzzy natural language instructions, multi-step tool calls, and external side effects into control flow graphs, data flow graphs, and program dependencies. Figure 3 This complementary structure enables precise characterization of the interaction process of agent 120. Building upon this, embodiments of this disclosure introduce a tag-based security verification mechanism, enabling real-time identification of sensitive data leakage and unauthorized access by low-trust instructions. Furthermore, embodiments of this disclosure support incremental runtime verification and interpretive report generation, ensuring low intrusion into agent 120 logic while significantly improving the transparency and auditability of security decisions.

[0073] Embodiments of this disclosure also provide corresponding apparatus for implementing the above methods or processes. Figure 4 A schematic structural block diagram of a device 400 for security verification of an intelligent agent according to some embodiments of the present disclosure is shown. The device 400 may be implemented as or included in an electronic device 110. The various modules / components in the device 400 may be implemented by hardware, software, firmware, or any combination thereof.

[0074] Reference Figure 4 The apparatus 400 includes a structured representation generation module 410, a security description information determination module 420, and a security verification module 430. The structured representation generation module 410 is configured to generate a first structured representation describing the dependencies in at least one task execution process based on runtime sequence data corresponding to at least one task execution process of the agent; wherein the first structured representation includes multiple execution nodes corresponding to multiple processing functions invoked in the at least one task execution process, and data in the at least one task execution process. The security description information determination module 420 is configured to determine the corresponding security description information of the multiple execution nodes in the first structured representation based at least on tool security configurations related to the available tools of the agent and data security configurations related to the accessible data objects of the agent. The security verification module 430 is configured to perform security verification on the at least one task execution process of the agent based on the corresponding security description information of the multiple execution nodes.

[0075] In some embodiments, the structured representation generation module 410 is further configured to: generate a second structured representation describing the control relationships between multiple processing functions based on runtime sequence data; generate a third structured representation describing the data flow during at least one task execution process based on at least the second structured representation; and generate a first structured representation by fusing the second structured representation and the third structured representation.

[0076] In some embodiments, the plurality of processing functions include an action execution function, and the structured representation generation module 410 is further configured to: determine a plurality of functional nodes in the second structured representation corresponding to the plurality of processing functions based on the plurality of processing functions; and for the action function node in the plurality of functional nodes corresponding to the action execution function, determine at least one control path in the second structured representation including the action function node based on the control relationship between the action execution function and other processing functions in the plurality of processing functions besides the action execution function.

[0077] In some embodiments, the multiple processing functions further include a decision function, and the structured representation generation module 410 is further configured to: split the action function node into a first sub-node and a second sub-node, wherein the first sub-node corresponds to the tool to be used by the action execution function, and the second sub-node corresponds to the input parameters of the tool; and determine the control path between the first sub-node and the decision function node and the control path between the second sub-node and the decision function node based on the control relationship between the action execution function and the decision function, wherein the decision function node is the function node corresponding to the decision function among the multiple function nodes.

[0078] In some embodiments, the multiple processing functions include an action execution function, a decision-making function, a cue word processing function, and an observation information acquisition function, wherein the control relationship between at least one of the decision-making function, the cue word processing function, or the observation information acquisition function and the action execution function is determined using a machine learning model.

[0079] In some embodiments, the structured representation generation module 410 is further configured to: extract at least one functional node related to the data flow from the plurality of functional nodes of the second structured representation as part of the plurality of data nodes of the third structured representation; create at least one additional node as another part of the plurality of data nodes, based at least on external data objects accessed by available tools when invoked; and determine the connection relationships between the plurality of data nodes based on the data dependencies during at least one task execution process to obtain the third structured representation.

[0080] In some embodiments, for a first running node among a plurality of running nodes, the first running node is generated during the fusion process of the second structured representation and the third structured representation by fusing the corresponding functional node in the second structured representation and the corresponding data node in the third structured representation, and the structured representation generation module 410 is further configured to: determine the reference security description information of the corresponding functional node and the corresponding data node based on at least one of tool security configuration or data security configuration; and determine the security description information of the first running node by inheriting the reference security description information of the corresponding functional node and the corresponding data node during the fusion process of the corresponding functional node and the corresponding data node.

[0081] In some embodiments, the security description information determination module 420 is further configured to: determine the security description information of a third running node connected to the second running node among the plurality of running nodes based on the security description information of the second running node among the plurality of running nodes.

[0082] In some embodiments, after the fourth running node among the plurality of running nodes is connected to a group of running nodes among the plurality of running nodes, the security description information determination module 420 is further configured to: determine the security description information of the fourth running node based on the security description information of the running node with the highest security requirements among the group of running nodes.

[0083] In some embodiments, the tool security configuration includes a first security identifier for the data involved in the available tool, and the data security configuration includes a second security identifier for the accessible data object, wherein at least one of the first security identifier or the second security identifier indicates at least one of the following: security sensitivity or trust domain.

[0084] In some embodiments, the data involved in the available tool includes at least one of the following: input parameters of the available tool, return values ​​of the available tool, or data related to the side effects of the available tool. Accessible data objects include at least one of the following: an accessible file of the agent or an accessible link of the agent.

[0085] In some embodiments, the corresponding security description information of multiple running nodes indicates at least one of the following: node security level, node data integrity, node trust level, or node access constraints related to runtime sequence data.

[0086] In some embodiments, node security, node data integrity, and node trustworthiness are determined based on at least one of the following: content in the tool security configuration related to at least one of confidentiality, data integrity, and trustworthiness; or content in the data security configuration related to at least one of confidentiality, data integrity, and trustworthiness. Node access constraints are generated based on at least one of the following: configurable constraint generation strategies or metadata of available tools.

[0087] In some embodiments, the security verification module 430 is further configured to perform at least one of the following for the fifth running node among the running nodes: determining that the fifth running node has failed security verification in response to the access process involved in the fifth running node not meeting the node access constraints; determining that the fifth running node has failed security verification in response to the data of the fifth running node being transmitted to the sixth running node among the multiple running nodes and the node security level of the sixth running node being lower than that of the fifth running node; or determining that the fifth running node has failed security verification in response to the data of the fifth running node being transmitted to the sixth running node and the trust level of the sixth running node being lower than that of the fifth running node.

[0088] In some embodiments, the apparatus 400 further includes an update module. The update module is configured to: update a first structured representation based on the new processing function in response to detecting that the agent performs a new processing function during at least one task execution; and update the security verification of the agent's at least one task execution based on the updated first structured representation.

[0089] In some embodiments, the apparatus 400 further includes an annotation module. The annotation module is configured to: in response to at least one task execution process of the agent failing the security check, annotate the execution node that failed the security check in the first structured representation.

[0090] Figure 5 A block diagram is shown of an electronic device 500 in which one or more embodiments of the present disclosure may be implemented. The electronic device 500 may, for example, be used to implement... Figure 1 The electronic device 110 shown or such Figure 4 The device 400 shown. It should be understood that, Figure 5 The electronic device 500 shown is merely exemplary and should not be construed as limiting the functionality and scope of the embodiments described herein.

[0091] Reference Figure 5 Electronic device 500 is in the form of a general-purpose electronic device. Components of electronic device 500 may include, but are not limited to, one or more processors 510, memory 520, storage device 530, one or more communication units 540, one or more input devices 550, and one or more output devices 560. Processor 510 may be a physical or virtual processor and is capable of performing various processes according to programs stored in memory 520. In a multiprocessor system, multiple processors execute computer-executable instructions in parallel to improve the parallel processing capability of electronic device 500.

[0092] Electronic device 500 typically includes multiple computer storage media. Such media can be any available media accessible to electronic device 500, including but not limited to volatile and non-volatile media, removable and non-removable media. Memory 520 can be volatile memory (e.g., registers, cache, random access memory (RAM)), non-volatile memory (e.g., read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory), or some combination thereof. Storage device 530 can be removable or non-removable media and can include machine-readable media, such as flash drives, disks, or any other media capable of storing information and / or data and accessible within electronic device 500.

[0093] Electronic device 500 may further include additional removable / non-removable, volatile / non-volatile storage media. Although not explicitly stated... Figure 5 As shown, disk drives for reading from or writing to removable, non-volatile disks (e.g., "floppy disks") and optical disk drives for reading from or writing to removable, non-volatile optical disks can be provided. In these cases, each drive can be connected to a bus (not shown) via one or more data media interfaces. Memory 520 may include computer program product 525 having one or more program modules configured to perform various methods or actions of various embodiments of this disclosure.

[0094] Communication unit 540 enables communication with other electronic devices via a communication medium. Additionally, the functionality of components of electronic device 500 can be implemented using a single computing cluster or multiple computing machines capable of communicating via communication connections. Therefore, electronic device 500 can operate in a networked environment using logical connections to one or more other servers, network personal computers (PCs), or another network node.

[0095] Input device 550 can be one or more input devices, such as a mouse, keyboard, trackball, etc. Output device 560 can be one or more output devices, such as a monitor, speaker, printer, etc. Electronic device 500 can also communicate with one or more external devices (not shown) via communication unit 540 as needed. These external devices include storage devices, display devices, etc., and can communicate with one or more devices that enable user interaction with electronic device 500, or with any device that enables electronic device 500 to communicate with one or more other electronic devices (e.g., network card, modem, etc.). Such communication can be performed via input / output (I / O) interface (not shown).

[0096] According to an exemplary implementation of this disclosure, a computer-readable storage medium is provided that stores computer-executable instructions thereon, wherein the computer-executable instructions are executed by a processor to implement the methods described above. According to an exemplary implementation of this disclosure, a computer program product is also provided, which is tangibly stored on a non-transitory computer-readable medium and includes computer-executable instructions, which are executed by a processor to implement the methods described above.

[0097] Various aspects of this disclosure are described herein with reference to flowchart illustrations and / or block diagrams of methods, apparatuses, devices, and computer program products implemented according to this disclosure. It should be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer-readable program instructions.

[0098] These computer-readable program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing apparatus to produce a machine such that, when executed by the processor of the computer or other programmable data processing apparatus, they create means for implementing the functions / actions specified in one or more blocks of the flowchart and / or block diagram. These computer-readable program instructions can also be stored in a computer-readable storage medium that causes a computer, programmable data processing apparatus, and / or other device to operate in a particular manner; thus, the computer-readable medium storing the instructions comprises an article of manufacture that includes instructions for implementing aspects of the functions / actions specified in one or more blocks of the flowchart and / or block diagram.

[0099] Computer-readable program instructions can be loaded onto a computer, other programmable data processing apparatus, or other device to cause a series of operational steps to be performed on the computer, other programmable data processing apparatus, or other device to produce a computer-implemented process, thereby causing the instructions that execute on the computer, other programmable data processing apparatus, or other device to perform the functions / actions specified in one or more boxes of a flowchart and / or block diagram.

[0100] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this disclosure. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of an instruction, which contains one or more executable instructions for implementing the specified logical function. In some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutive blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, may be implemented using a dedicated hardware-based system that performs the specified function or action, or using a combination of dedicated hardware and computer instructions.

[0101] Various implementations of this disclosure have been described above. The foregoing description is exemplary and not exhaustive, nor is it limited to the disclosed implementations. Many modifications and variations will be apparent to those skilled in the art without departing from the scope and spirit of the described implementations. The terminology used herein is determined to best explain the principles, practical applications, or improvements to technology in the market, or to enable others skilled in the art to understand the various implementations disclosed herein.

Claims

1. A security verification method for intelligent agents, comprising: Based on the runtime sequence data corresponding to at least one task execution process of the agent, a first structured representation describing the dependencies in the at least one task execution process is generated; wherein, the first structured representation includes multiple runtime nodes corresponding to the following: multiple processing functions called in the at least one task execution process, and data in the at least one task execution process; Based at least on tool security configurations related to the available tools of the agent and data security configurations related to the accessible data objects of the agent, the corresponding security description information of the plurality of running nodes in the first structured representation is determined, wherein determining the corresponding security description information of the plurality of running nodes in the first structured representation includes: Based on at least one of the tool security configuration or the data security configuration, determine the reference security description information for each corresponding functional node and each corresponding data node; and During the fusion process of the corresponding functional nodes and the corresponding data nodes, the security description information of any one of the plurality of running nodes is determined by inheriting the reference security description information of each of the corresponding functional nodes and the corresponding data nodes; and Based on the corresponding security description information of the multiple running nodes, a security check is performed on the agent's at least one task execution process.

2. The method according to claim 1, wherein, Generating a first structured representation describing the dependencies during the execution of at least one task includes: Based on the runtime sequence data, a second structured representation describing the control relationships between the multiple processing functions is generated; Based at least on the second structured representation, generate a third structured representation describing the data flow during the at least one task execution; and The first structured representation is generated by fusing the second structured representation and the third structured representation.

3. The method according to claim 2, wherein, The plurality of processing functions include action execution functions, and generating a second structured representation describing the control relationships between the plurality of processing functions includes: Based on the aforementioned processing functions, multiple functional nodes corresponding to the processing functions in the second structured representation are determined; and For the action function node corresponding to the action execution function among the plurality of functional nodes, based on the control relationship between the action execution function and other processing functions among the plurality of processing functions, at least one control path including the action function node in the second structured representation is determined.

4. The method according to claim 3, wherein, The plurality of processing functions also include a decision-making function, and determining at least one control path in the second structured representation that includes the action function node includes: The action function node is divided into a first sub-node and a second sub-node, wherein the first sub-node corresponds to the tool to be used by the action execution function, and the second sub-node corresponds to the input parameters of the tool; and Based on the control relationship between the action execution function and the decision function, the control path between the first sub-node and the decision function node and the control path between the second sub-node and the decision function node are determined, wherein the decision function node is the function node corresponding to the decision function among the plurality of function nodes.

5. The method according to claim 2, wherein, The plurality of processing functions include an action execution function, a decision-making function, a prompt word processing function, and an observation information acquisition function. The control relationship between at least one of the decision-making function, the prompt word processing function, or the observation information acquisition function and the action execution function is determined using a machine learning model.

6. The method according to claim 2, wherein, Generating a third structured representation describing the data flow during the execution of at least one task includes: At least one functional node related to the data flow is extracted from the multiple functional nodes of the second structured representation and used as part of the multiple data nodes of the third structured representation; At least one additional node is created as another part of the plurality of data nodes, based on the external data object accessed by the available tool when it is invoked; and Based on the data dependencies during the at least one task execution process, the connection relationships between the multiple data nodes are determined to obtain the third structured representation.

7. The method according to claim 2, wherein, For the first running node among the plurality of running nodes, the first running node is generated during the fusion process of the second structured representation and the third structured representation by fusing the corresponding functional nodes in the second structured representation and the corresponding data nodes in the third structured representation, and the corresponding security description information of the plurality of running nodes in the first structured representation includes: Based on at least one of the tool security configuration or the data security configuration, determine the reference security description information for each of the corresponding functional node and the corresponding data node; and During the fusion process of the corresponding functional nodes and the corresponding data nodes, the security description information of the first running node is determined by inheriting the reference security description information of each of the corresponding functional nodes and the corresponding data nodes.

8. The method according to claim 1, wherein, Determining the corresponding security description information of the plurality of running nodes in the first structured representation includes: Based on the security description information of the second running node among the plurality of running nodes, the security description information of the third running node that is connected to the second running node among the plurality of running nodes is determined.

9. The method according to claim 1, wherein, After the fourth running node of the plurality of running nodes is connected to a group of running nodes of the plurality of running nodes, the corresponding security description information of the plurality of running nodes in the first structured representation is determined to include: Based on the security description information of the operating node with the highest security requirements among the group of operating nodes, the security description information of the fourth operating node is determined.

10. The method of claim 1, wherein the tool security configuration includes a first security identifier for data relating to the available tool, and the data security configuration includes a second security identifier for the accessible data object, wherein at least one of the first security identifier or the second security identifier indicates at least one of the following: Security sensitivity, or Trust domain.

11. The method according to claim 10, wherein, The data involved in the available tools includes at least one of the following: The input parameters of the available tools, The return value of the available tools, or Data relating to the side effects of the available tools; and The accessible data object includes at least one of the following: The accessible files of the intelligent agent, or The accessible links of the intelligent agent.

12. The method according to claim 1, wherein, The corresponding security description information of the plurality of running nodes indicates at least one of the following: Node security level Node data integrity Node trustworthiness, or Node access constraints related to the runtime sequence data.

13. The method of claim 12, wherein, The node security level, the node data integrity, and the node trust level are determined based on at least one of the following: The tool's security configuration includes elements related to at least one of confidentiality, data integrity, and trustworthiness, or The data security configuration includes elements related to at least one of confidentiality, data integrity, and trustworthiness, and The node access constraints are generated based on at least one of the following: Configurable constraint generation strategy, or Metadata of the available tools.

14. The method according to claim 12, wherein, Performing security checks on at least one task execution process of the agent includes: performing at least one of the following for the fifth execution node among the execution nodes: In response to the access process involving the fifth running node not meeting the node access constraints, it is determined that the fifth running node has failed the security check. In response to the data from the fifth running node being transmitted to the sixth running node among the plurality of running nodes, and the node security level of the sixth running node being lower than that of the fifth running node, it is determined that the fifth running node has failed the security check, or In response to the data being transmitted from the fifth running node to the sixth running node, and the trust level of the sixth running node being lower than that of the fifth running node, it is determined that the fifth running node has failed the security verification.

15. The method of claim 1, further comprising: In response to detecting that the agent performs a new processing function during the at least one task execution, the first structured representation is updated based on the new processing function; as well as Based on the updated first structured representation, the security verification of the agent's at least one task execution process is updated.

16. The method according to claim 1, further comprising: In response to the agent failing the security check at least once during a task execution process, the execution node that failed the security check is marked in the first structured representation.

17. A security verification device for an intelligent agent, comprising: The structured representation generation module is configured to generate a first structured representation describing the dependencies in at least one task execution process based on runtime sequence data corresponding to the intelligent agent; wherein, the first structured representation includes multiple runtime nodes corresponding to the following: multiple processing functions called in the at least one task execution process, and data in the at least one task execution process; The security description information determination module is configured to determine the corresponding security description information of the plurality of running nodes in the first structured representation based at least on tool security configurations related to the available tools of the agent and data security configurations related to the accessible data objects of the agent, wherein the security description information determination module is further configured to: Based on at least one of the tool security configuration or the data security configuration, determine the reference security description information for each corresponding functional node and each corresponding data node; and During the fusion process of the corresponding functional nodes and the corresponding data nodes, the security description information of any one of the plurality of running nodes is determined by inheriting the reference security description information of each of the corresponding functional nodes and the corresponding data nodes; and The security verification module is configured to perform security verification on the agent's at least one task execution process based on the corresponding security description information of the plurality of running nodes.

18. An electronic device comprising: At least one processor; as well as At least one memory coupled to the at least one processor and storing instructions for execution by the at least one processor, the instructions causing the electronic device to perform the method according to any one of claims 1 to 16 when executed by the at least one processor.

19. A computer-readable storage medium having stored thereon computer-executable instructions that can be executed by a processor to implement the method according to any one of claims 1 to 16.

20. A computer program product comprising computer-executable instructions, wherein the computer-executable instructions, when executed by a processor, implement the method according to any one of claims 1 to 16.

Images