Method, device and equipment for security verification of intelligent agent and storage medium

Abstract

The embodiment of the invention relates to a security verification method and device for an intelligent agent, equipment and a storage medium. The method comprises the steps of generating a first structured representation for describing a dependency relationship in a task running process based on running time sequence data corresponding to at least one task running process of an intelligent agent; the first structured representation comprises a plurality of operation nodes corresponding to the following items: a plurality of processing functions and data called in the task operation process; determining corresponding security description information of a plurality of operation nodes in the first structured representation at least based on a tool security configuration related to an available tool of the agent and a data security configuration related to an accessible data object of the agent; and based on the corresponding security description information of the plurality of operation nodes, executing security verification on the task operation process of the intelligent agent. Therefore, accurate modeling and effective verification of the operation process of the intelligent agent can be realized, so that the coverage rate and accuracy of risk identification are improved.

Description

TECHNICAL FIELD

[0001] Example embodiments of the present disclosure generally relate to the field of computer technology, and more particularly, to a method, apparatus, device and storage medium for security verification of an intelligent agent. BACKGROUND

[0002] As an upper application expansion of machine learning models, intelligent agents significantly improve the processing capability of machine learning models on complex tasks through the integration of tool calls, memory management, reasoning planning and other functions. Intelligent agents have broad prospects in office automation and intelligent interaction scenarios. However, compared with machine learning models, intelligent agents have a more complex architecture and running logic, exposing a larger attack surface and thus facing more severe security challenges. SUMMARY

[0003] In a first aspect of the present disclosure, a method for security verification of an intelligent agent is provided. The method comprises: generating, based on runtime sequence data corresponding to at least one task running process of the intelligent agent, a first structured representation describing dependency relationships in the at least one task running process; wherein the first structured representation comprises a plurality of running nodes corresponding to: a plurality of processing functions called in the at least one task running process, and data in the at least one task running process; determining, based at least on tool security configurations related to available tools of the intelligent agent and data security configurations related to accessible data objects of the intelligent agent, respective security description information of the plurality of running nodes in the first structured representation; and performing, based on the respective security description information of the plurality of running nodes, security verification on the at least one task running process of the intelligent agent.

[0004] In a second aspect of the present disclosure, an apparatus for security verification of an intelligent agent is provided. The apparatus comprises: a structured representation generation module configured to generate, based on runtime sequence data corresponding to at least one task running process of the intelligent agent, a first structured representation describing dependency relationships in the at least one task running process; wherein the first structured representation comprises a plurality of running nodes corresponding to: a plurality of processing functions called in the at least one task running process, and data in the at least one task running process; a security description information determination module configured to determine, based at least on tool security configurations related to available tools of the intelligent agent and data security configurations related to accessible data objects of the intelligent agent, respective security description information of the plurality of running nodes in the first structured representation; and a security verification module configured to perform, based on the respective security description information of the plurality of running nodes, security verification on the at least one task running process of the intelligent agent.

[0005] In a third aspect of the disclosure, an electronic device is provided. The device includes at least one processor; and at least one memory coupled to the at least one processor and storing instructions for execution by the at least one processor. The instructions, when executed by the at least one processor, cause the device to perform the method of the first aspect.

[0006] In a fourth aspect of the disclosure, a computer-readable storage medium is provided. The computer-readable storage medium has stored thereon computer-executable instructions that are executable by a processor to implement the method of the first aspect.

[0007] In a fifth aspect of the disclosure, a computer program product is provided. The computer program product includes computer-executable instructions that, when executed by a processor, implement the method according to the first aspect of the disclosure.

[0008] It should be understood that the contents described in this section are not intended to limit the key features or important features of the embodiments of the disclosure, nor are they used to limit the scope of the disclosure. Other features of the disclosure will become apparent through the following description. BRIEF DESCRIPTION OF DRAWINGS

[0009] The above and other features, advantages, and aspects of embodiments of the present disclosure will become more apparent by describing in detail exemplary embodiments thereof with reference to the attached drawings in which:

[0010] Figure 1 A schematic diagram illustrating an example environment according to embodiments of the disclosure is shown;

[0011] Figure 2 A flowchart illustrating an example process of security verification of an agent according to some embodiments of the disclosure is shown;

[0012] Figure 3 A schematic diagram illustrating an example architecture of security verification of an agent according to some embodiments of the disclosure is shown;

[0013] Figure 4 A schematic structural block diagram of an apparatus for security verification of an agent according to some embodiments of the disclosure is shown; and

[0014] Figure 5 A block diagram of an electronic device in which one or more embodiments of the disclosure can be implemented is shown. DETAILED DESCRIPTION

[0015] Embodiments of the present disclosure will be described below in greater detail with reference to the accompanying drawings. While certain embodiments of the present disclosure are shown in the drawings, it is understood that the present disclosure can be embodied in various forms and should not be construed as being limited to the embodiments set forth herein; rather, these embodiments are provided so as to more completely and thoroughly understand the present disclosure. It is understood that the drawings and embodiments of the present disclosure are for exemplary purposes only and are not intended to limit the scope of the present disclosure.

[0016] It should be noted that the titles of any sections / sub-sections provided herein are not limiting. Various embodiments are described throughout, and any type of embodiment can be included under any section / sub-section. Moreover, embodiments described in any section / sub-section can be combined with any other embodiments described in the same section / sub-section and / or different section / sub-section in any manner.

[0017] In the description of embodiments of the present disclosure, the term "includes" and its derivatives, such as "including," should be understood in an open, inclusive sense, that is, "including, but not limited to." The term "based on" should be understood as "based at least in part on." The term "one embodiment" or "an embodiment" should be understood as "at least one embodiment." The term "some embodiments" should be understood as "at least some embodiments." Other explicitly and implicitly recited definitions can also be found below. The terms "first," "second," and the like can refer to different or identical objects. Other explicitly and implicitly recited definitions can also be found below.

[0018] Data of users, acquisition and / or use of data, etc. can be involved in embodiments of the present disclosure. These aspects all comply with corresponding laws and regulations and relevant provisions. In embodiments of the present disclosure, all collection, acquisition, processing, processing, forwarding, use, etc. of data are performed on the premise that users are aware of and confirm. Accordingly, when implementing embodiments of the present disclosure, the type of data or information that can be involved, the range of use, the scenario of use, etc. should be informed to users and authorized by users in a proper manner according to relevant laws and regulations. The specific informing and / or authorization manner can vary according to actual situations and application scenarios, and the scope of the present disclosure is not limited in this respect.

[0019] In the present specification and embodiments, if personal information processing is involved, it will be processed on the premise of legality (for example, obtaining the consent of the subject of personal information, or being necessary for the performance of a contract, etc.), and only within the prescribed or agreed range. Users refuse to process personal information other than the necessary information required for basic functions, which will not affect the user's use of basic functions.

[0020] As briefly described above, agents face severe security challenges. For example, in the scenario where agents interact across domains (e.g., different business domains), the tool invocation and memory management functions that agents rely on are often distributed across business domains of different trust levels. In this case, sensitive information can flow from a high-trust domain to a low-trust domain (e.g., user account information used for login on an untrusted website). In addition, malicious instructions can penetrate from a low-trust domain to a high-trust domain, resulting in attacks such as privacy leakage and instruction poisoning. At the same time, due to the "blind execution" feature of agents, agents lack introspection capabilities for instructions. In this case, an attacker can disguise malicious instructions as data injection into the execution flow, thereby tampering with normal logic.

[0021] According to the difference in technical path, the current security protection scheme is generally divided into three categories: model input / output filtering type protection scheme (such as protection scheme based on machine learning model firewall), permission control type protection scheme and trusted execution environment type protection scheme. In the model input / output filtering type protection scheme, the threats in the input / output content can be identified by vector similarity, rule matching or machine learning model. In the permission control type protection scheme, tool invocation and memory management functions can be abstracted as entities or objects, so as to verify the legality of the behavior through the permission system. In the trusted execution environment type protection scheme, a secure execution environment can be created by using hardware isolation technology, so as to protect the data security of machine learning models in the reasoning phase.

[0022] The above schemes all rely on the syntax of the programming language. However, agents are driven by natural language instructions, the semantics of which have ambiguity, the execution plan has dynamic nature, and the tool invocation spans multiple trust domains. These characteristics make it impossible for the above security protection schemes to accurately analyze the behavior of agents, resulting in limited security protection effect.

[0023] Embodiments of the present disclosure provide a scheme for security verification of an agent. According to the scheme, first, based on runtime sequence data corresponding to at least one task running process of the agent, a first structured representation describing the dependency relationship in the at least one task running process is generated. The first structured representation includes a plurality of nodes corresponding to a plurality of processing functions called in the at least one task running process and data in the at least one task running process. Next, at least based on tool security configurations related to available tools of the agent and data security configurations related to accessible data objects of the agent, corresponding security description information of a plurality of running nodes in the first structured representation is determined. Then, based on the corresponding security description information of the plurality of running nodes, security verification is performed on the at least one task running process of the agent.

[0024] It will be more clearly understood through the description below that the embodiments of the present disclosure can uniformly map the factors that were originally unquantifiable in the task running process of the intelligent agent (such as natural language instructions, cross-tool calls and external side effects (Side Effects)) into a structured representation (such as a first structured representation) based on the runtime timing data corresponding to at least one task running process of the intelligent agent, thereby achieving accurate modeling and verifiable description of the task running process. On this basis, the embodiments of the present disclosure introduce security prior knowledge related to the behavior of the intelligent agent in the security verification process through tool security configuration and data security configuration. In this way, potential risk paths and complex interaction scenarios that are difficult to identify with traditional solutions can be effectively covered, thereby significantly improving the coverage and accuracy of risk identification.

[0025] Various example implementations of this solution will be described in detail below with reference to the accompanying drawings. Figure 1 Schematic diagram of an example environment 100 according to an embodiment of the present disclosure is shown. Figure 1 , the example environment 100 may include an electronic device 110, an agent 120, and a machine learning model 130. It should be understood that the structure and function of each element in the environment 100 are described herein for exemplary purposes only and do not imply any limitation on the scope of the present disclosure.

[0026] In the example environment 100, the agent 120 can perform reasoning and decision-making based on input instructions from a user or other system with the help of a machine learning model 130 to complete a specific task. The agent 120 can be an upper-layer application extension of the machine learning model 130. For example, during at least one task execution by the agent 120, the agent 120 can call corresponding tools or interfaces to assist in completing the task. The machine learning model 130 can receive data from the agent 120 as model input. The machine learning model 130 can, for example, be a large language model (LLM) or a large model capable of processing multimodal input (i.e., a multimodal large model). The machine learning model 120 can process the model input using trained model parameters to provide recommendations or guidance for the subsequent behavior of the agent 120. The electronic device 110 can monitor the at least one task execution process of the agent 120 and perform security verification on the at least one task execution process of the agent 120. The electronic device 110 can present an interface 150 to the user 140. The security verification result of the electronic device 110 on the intelligent agent 120 can be presented on the interface 150 for the user 140 to view and confirm.

[0027] In some embodiments, the electronic device 110 can be any type of mobile terminal, fixed terminal, or portable terminal including a mobile handset, a desktop computer, a laptop computer, a notebook computer, a subnotebook computer, a netbook computer, a tablet computer, a media player, a multimedia player, a personal navigation device, a personal digital assistant (PDA), an audio / video player, a digital camera / camcorder, a positioning device, a television receiver, a radio broadcast receiver, an electronic book device, a game device, or any combinations of the aforementioned devices, including accessories and peripherals for these devices, or any combinations thereof. In some embodiments, the electronic device 110 can also support any type of interface to the user (such as "wearable" circuitry, etc.).

[0028] Alternatively, in some embodiments, the electronic device 110 can be a standalone physical server, or a server cluster or distributed system composed of multiple physical servers, or a cloud server providing basic cloud computing services such as cloud service, cloud database, cloud computing, cloud function, cloud storage, network service, cloud communication, middleware service, domain name service, security service, content distribution network, and big data and artificial intelligence platform, etc. The electronic device 110 may, for example, include a computing system / server such as a mainframe, an edge computing node, a computing device in a cloud environment, etc.

[0029] A communication connection can be established between the electronic device 110 and the agent 120. The communication connection can be established by wired or wireless means. The communication connection can include, but is not limited to, a Bluetooth connection, a mobile network connection, a universal serial bus connection, a wireless fidelity connection, etc., and embodiments of the present disclosure are not limited in this regard. In embodiments of the present disclosure, the electronic device 110 and the agent 120 can achieve signaling interaction through the communication connection therebetween.

[0030] Figure 2 A flowchart of an example process 200 of security verification of an agent according to some embodiments of the present disclosure is shown, Figure 3 A schematic diagram of an example architecture 300 of security verification of an agent according to some embodiments of the present disclosure is shown. The following describes the architecture 300 in conjunction with the process 200, Figure 1 and Figure 3 The process 200 is described. The process 200 can be implemented at the electronic device 110, thereby achieving security verification of at least one task running process of the agent 120.

[0031] Reference is made to Figure 2At block 210, the electronic device 110 (e.g., the structured representation generation module 301 in the electronic device 110) generates a structured representation (e.g., a first structured representation) describing dependencies in the at least one task running process of the agent 120 based on runtime sequence data corresponding to the at least one task running process. In embodiments of the present disclosure, the dependencies in the at least one task running process can be dependencies between various factors in the at least one task running process. The factors can be processing functions invoked in the at least one running process and data accessed (e.g., read, modified, added, generated, etc.). For example, the first structured representation includes a plurality of running nodes corresponding to a plurality of processing functions invoked in the at least one task running process and data in the at least one task running process. The plurality of running nodes and edges between the plurality of running nodes can indicate the dependencies described above.

[0032] The at least one task running process of the agent 120 can refer to a running process of a series of tasks performed by the agent 120 in one or more interactions with a user. The tasks can include any suitable reasoning or decision-making tasks, etc. For example, in each task running process, the agent 120 can invoke a plurality of processing functions. The plurality of processing functions can include a prompt processing function, an invocation function of a machine learning model 130 (e.g., a large model such as an LLM or a multi-modal large model), a decision (also referred to as thought) function, an action execution function, a tool invocation function, and an observation acquisition function, etc. The runtime sequence data corresponding to the at least one task running process of the agent 120 can include state information and / or behavior data generated in the task running process, etc. The state information may, for example, include inputs of processing functions, outputs of processing functions, availability of processing functions, etc. The behavior data may, for example, include whether the processing functions are successfully invoked, whether the processing functions are successfully ended, etc. The runtime sequence data in each task running process can be represented in a serialized form. In some embodiments, the runtime sequence data can include contexts generated in the task running process, and the context represented in a serialized form can also be referred to as a context sequence.

[0033] The first structured representation can be a structured data form of the dependency relationship in the at least one task running process. A running node is a basic unit in the first structured representation. The dependency relationship in the at least one task running process can be represented by edges connected between multiple running nodes. In some embodiments, the first structured representation can be a graph structure representation. Alternatively, in some embodiments, the first structured representation can be a tree structure representation, and the like. In some embodiments, the electronic device 110 can abstract the dependency relationship in the at least one task running process into the first structured representation by any appropriate structured representation generation manner.

[0034] In some embodiments, the dependency relationship in the at least one task running process can indicate a control relationship between multiple processing functions performed by the agent 120 in the task running process. The control relationship can indicate, for example, an execution order (or control passing) and a control dependency relationship between the multiple processing functions, and the like. In this case, a corresponding running node in the first structured representation can correspond to a single processing function. Alternatively or additionally, in some embodiments, the dependency relationship in the at least one task running process can indicate a data flow direction in the task running process of the agent 120. The data flow direction can indicate, for example, a flow direction of data on the multiple processing functions, and the like. In this case, a corresponding running node in the first structured representation can be the data itself and a processing function processing the data, and the like.

[0035] In some embodiments, the first structured representation can indicate both the control relationship and the data flow direction described above. In this case, the electronic device 110 can generate the first structured representation by fusing multiple structured representations (e.g., a second structured representation and a third structured representation) respectively indicating the control relationship and the data flow direction. For example, the electronic device 110 can generate, based on the runtime sequence data corresponding to the at least one task running process of the agent 120, a second structured representation describing a control relationship between the multiple processing functions. Then, the electronic device 110 can generate, based at least on the second structured representation, a third structured representation describing a data flow direction in the task running process. Subsequently, the electronic device 110 can generate the first structured representation by fusing the second structured representation and the third structured representation.

[0036] As mentioned above, the first structured representation can be a graph structure representation. For example, the first structured representation can be a Program Dependency Graph (PDG). In this case, the second structured representation and the third structured representation can both be graph structure representations. For example, the second structured representation can be a Control Flow Graph (CFG), and the third structured representation can be a Data Flow Graph (DFG).

[0037] In some embodiments, the second structured representation takes processing functions (also referred to as basic blocks of the agent 120 program) as nodes (also referred to as function nodes hereinafter), and reflects the control relationship between the processing functions through edges connecting between the function nodes. In some embodiments, the electronic device 110 can generate an initial second structured representation by invoking a generation function (such as a control flow graph initialization function or other appropriate function) of the second structured representation. Next, the electronic device 110 can parse the runtime sequence data corresponding to at least one task running process of the agent 120, so as to determine the processing functions actually invoked by the agent 120 in the task running process. For example, the processing functions actually invoked by the agent 120 in the task running process can include a prompt word processing function, a machine learning model invocation function, a decision function, an action execution function, a tool invocation function, and an observation information acquisition function, etc. Then, the electronic device 110 can convert these processing functions into function nodes of the second structured representation.

[0038] In some embodiments, the processing functions can include an action execution function. On this basis, the electronic device 110 can determine a control path in the second structured representation for the action execution function, so as to obtain an execution backbone of the second structured representation. Specifically, the electronic device 110 can determine a plurality of function nodes in the second structured representation corresponding to the processing functions. Then, for an action function node in the plurality of function nodes corresponding to the action execution function, the electronic device 110 can determine at least one control path in the second structured representation including the action function node based on the control relationship between the action execution function and other processing functions (such as a decision function, a prompt word processing function, and an observation information acquisition function) in the plurality of processing functions except the action execution function. For example, the electronic device 110 can connect the action function node and the corresponding other function nodes with directed edges based on the actual execution order between the action execution function and the other processing functions, so as to obtain a control path. By taking the action function node as a key node, it is helpful to sort out the effective control path in the plurality of function nodes, so as to improve the generation efficiency of the second structured representation.

[0039] In some embodiments, the electronic device 110 can split the action function node into a first sub-node and a second sub-node. The first sub-node corresponds to a tool to be used by the action execution function. Such a first sub-node can be denoted as Action ToolName. The second sub-node corresponds to an input parameter of the tool, such a second sub-node can be denoted as Action ToolParam. The plurality of processing functions can include a decision function. On this basis, the electronic device 110 can determine a control path between the first sub-node and the decision function node and a control path between the second sub-node and the decision function node based on the control relationship between the action execution function and the decision function. The decision function node can be a function node in the plurality of function nodes corresponding to the decision function. In this way, the control relationship between the action execution function and the decision function can be more detailed reflected in the second structured representation, thereby facilitating accurate generation of the subsequent third structured representation and the first structured representation.

[0040] In some embodiments, the electronic device 110 can parse the actual execution order between the action execution function and the decision function. Further, the electronic device 110 can call a node splitting function or other appropriate function to split the action function node into independent first and second sub-nodes. Then, the electronic device 110 can connect the first sub-node and the decision function node and the second sub-node and the decision function node through directed edges based on the actual execution order between the action execution function and the decision function, thereby obtaining the control path between the first sub-node and the decision function node and the control path between the second sub-node and the decision function node. In some embodiments, in addition to the input parameter of the tool, the second sub-node can also correspond to more parameters of the tool, such as the return value of the tool, and the like.

[0041] In addition to the control path, in some embodiments, the electronic device 110 can also construct control dependency edges between the plurality of function nodes for indicating control dependency relationships between the plurality of processing functions. For example, the electronic device 110 can further analyze the control relationship (e.g., control dependency relationship) between at least part of the function nodes in the second structured representation using a machine learning model (e.g., an LLM model different from the first machine learning model, also referred to herein as a second machine learning model). In turn, the electronic device 110 can construct control dependency edges according to the control dependency relationship. In some embodiments, the plurality of processing functions can include an action execution function, a decision function, a prompt word processing function, and an observation information acquisition function. On this basis, the control relationship (e.g., control dependency relationship) between at least one of the decision function, the prompt word processing function, or the observation information acquisition function and the action execution function can be determined using the second machine learning model.

[0042] In some embodiments, the electronic device 110 can take the action execution function and the decision function, the prompt word processing function, and the observation information obtaining function executed before the action execution function as model inputs of the second machine learning model. The second machine learning model can output a predicted control dependency relationship between at least one of the decision function, the prompt word processing function, or the observation information obtaining function and the action execution function based on the model inputs, using a pre-trained inference strategy. In addition to the control dependency relationship between each of the above functions, the second machine learning model can also output a control dependency relationship between the first sub-node and the second sub-node, and so on. This can be determined according to actual needs, and embodiments of the present disclosure will not be listed one by one here. With the aid of the second machine learning model, the electronic device 110 can accurately capture the control dependency relationship between each of the above processing functions in the complex running process of the agent 120.

[0043] In some embodiments, after determining the control dependency relationship between each processing function, the electronic device 110 can construct control dependency edges between multiple function nodes based on the control dependency relationship by calling a control dependency construction function or other appropriate function. In this process, the electronic device 110 can construct control dependency edges between the action function node and its related function nodes by backtracking the action function node and its related control paths through a control dependency analysis algorithm. By introducing the control dependency edges, the causal logic of at least one task running process, such as “which input led to the execution of which action”, can be reflected in the second structured representation. In combination with the control paths and the control dependency edges, the second structured representation can not only reflect the execution order between multiple processing functions, but also further reflect the control dependency relationship between multiple processing functions. Thus, the control relationship between multiple processing functions can be comprehensively and accurately described.

[0044] After obtaining the second structured representation, the electronic device 110 can generate a third structured representation based on the second structured representation. In some embodiments, the electronic device 110 can extract at least one function node related to data flow from the multiple function nodes of the second structured representation as part of multiple data nodes of the third structured representation. Next, the electronic device 110 can create at least one additional node as another part of the multiple data nodes based on at least an external data object accessible by the available tool when invoked. Then, the electronic device 110 can determine a connection relationship between the multiple data nodes based on the dependency relationship of the data in the at least one task running process to obtain the third structured representation.

[0045] In some embodiments, the third structured representation can reflect the data flow direction by connecting edges between multiple data nodes, where each data node is a node (also referred to as a data node hereinafter) representing data itself and a processing function (e.g., a tool invoked by a tool invocation function) processing the data. For example, the edges connecting between multiple data nodes can represent the flow of data input to a tool, the flow of data output by a tool, and the data dependency between data and data, etc.

[0046] In some embodiments, the electronic device 110 can invoke a generation function (e.g., a data flow graph copying function or other appropriate function) of the third structured representation to extract at least one function node related to the data flow direction from the multiple function nodes of the second structured representation. For example, the electronic device 110 can extract at least one function node related to the data flow direction from the second structured representation by pruning pure logic function nodes (e.g., machine learning model invocation function nodes and decision function nodes, etc.) in the second structured representation.

[0047] In some embodiments, the electronic device 110 can determine external data objects accessed by the available tools when invoked based on the metadata of the available tools (which can be configured in the tool security configuration 303). In some embodiments, the electronic device 110 can determine files and / or links, etc. that do not appear in the control path mentioned above but are actually read and written by the available tools as external data objects. In turn, the electronic device 110 can create corresponding additional nodes based on the external data objects by invoking a creation function (e.g., a side effect building function or other appropriate function) of the additional nodes. In this way, the electronic device 110 can dynamically supplement the external data objects into the multiple data nodes, thereby improving the comprehensiveness of the data nodes. In some embodiments, for any data node, the electronic device 110 can connect the data node to the data node corresponding to the available tool processing the data node through a data input edge and a data output edge.

[0048] In some embodiments, the electronic device 110 can determine the dependency of data (also referred to as data dependency) in the at least one task run based on the metadata of the accessible data objects (which can be configured in the data security configuration 304). In turn, the electronic device 110 can determine the connection relationship (e.g., a data dependency edge) between multiple data nodes based on the data dependency by invoking a data dependency building function or other appropriate function, to obtain the third structured representation. In this way, the third structured representation can reflect the data flow process at the available tools, thereby accurately describing the data flow direction of the agent 120 in the at least one task run.

[0049] In some embodiments, the electronic device 110 can generate the first structured representation by extracting key control relationships and data flow from the second structured representation and the third structured representation, and simplifying unnecessary content. In this way, the dependency relationships in the at least one task run of the agent 120 can be more effectively described.

[0050] In some embodiments, the electronic device 110 can take the second structured representation and the third structured representation as inputs, and integrate the control dependency edges in the second structured representation, the data input edges, the data output edges, and the data dependency edges in the third structured representation into the first structured representation by invoking a generation function (e.g., a control flow graph replication function and a data flow graph replication function) of the first structured representation. In this way, the first structured representation can accurately reflect the control causality and the data flow in the time sequence, thereby better describing the dependency relationships in the at least one task run of the agent 120.

[0051] In this way, the embodiments of the present disclosure employ three structured representations to collaboratively model the dependency relationships in the at least one task run of the agent 120. The three structured representations may, for example, be a control flow graph, a data flow graph, and a program dependency graph. Based on the foregoing description, it can be clearly understood that the embodiments of the present disclosure first construct a control flow graph containing the execution order and the control causality. Then a data flow graph is derived, which only retains data-related entities. Finally, the dependency relationships of the two graphs are fused to generate a program dependency graph. In the above process, the embodiments of the present disclosure also introduce a number of technical means. For example, a node splitting mechanism (splitting an action function node into a first sub-node and a second sub-node), a language model driven dependency inference (using an LLM to infer the control dependency relationships between multiple processing functions), and an edge type normalization process (ensuring semantic consistency between different graph structures).

[0052] Through the above modeling method, the embodiments of the present disclosure realize a three-dimensional integrated visual representation of “time sequence, control dependency, and data flow”. This multi-dimensional and refined structured representation not only helps to enhance the explainability of the at least one task run of the agent 120, but also provides effective support for fine-grained security verification.

[0053] With continued reference to Figure 2 After determining the first structured representation, at block 220, the electronic device 110 (e.g., a security verification module 302 in the electronic device) determines respective security description information of a plurality of run nodes in the first structured representation based at least on a tool security configuration 303 related to available tools of the agent 120 and a data security configuration 304 related to accessible data objects of the agent 120. In some embodiments, the tool security configuration 303 and the data security configuration 304 can be stored in a configuration information management module 305.

[0054] In some embodiments, the available tools can include components that the agent 120 is able to invoke. Alternatively or additionally, the available tools can include external interfaces that the agent 120 is able to invoke, and so on. In some embodiments, the accessible data objects can include files that are accessible to the agent 120. Alternatively or additionally, the accessible data objects can include links that are accessible to the agent 120, and so on.

[0055] In some embodiments, the tool security configuration 303 can include a security identification (e.g., a first security identification) for data involved by the available tools. The data security configuration 304 can include a security identification (e.g., a second security identification) for the accessible data objects. In some embodiments, the security identification can be, for example, a preconfigured security claim or a security label, and so on. In some embodiments, at least one of the first security identification or the second security identification can indicate a preconfigured security sensitivity (e.g., a security sensitivity claim configured for the data involved by the available tools or a security sensitivity claim configured for the accessible data objects). Alternatively or additionally, at least one of the first security identification or the second security identification can indicate a preconfigured trust domain (e.g., a trust domain claim configured for the data involved by the available tools or a trust domain claim configured for the accessible data objects). In some embodiments, the tool security configuration 303 and the data security configuration 304 can each be stored in the form of a registry.

[0056] In some embodiments, the data involved by the available tools can include input parameters of the available tools. Alternatively or additionally, the data involved by the available tools can include return values of the available tools. Alternatively or additionally, the data involved by the available tools can include data related to side effects of the available tools. It is noted that the side effects here can refer to additional impacts on the system state or external environment caused by the behavior of the available tools in addition to the expected results.

[0057] By introducing the tool security configuration 303 and the data security configuration 304, embodiments of the present disclosure establish a unified priori knowledge system in the agent 120 security scenario. Specifically, the tool security configuration 303 can structurally describe the input parameters, output results and potential side effects of each available tool at the workflow granularity, and declare the expected security sensitivity (such as the confidentiality level) and trust domain of each item of data involved. The data security configuration 304 can dynamically maintain all data objects that appear or are generated in the process of at least one task execution of the agent 120, and continuously record their sources, uses and current security labels. The tool security configuration 303 and the data security configuration 304 cooperatively build a benchmark for subsequent security checking. On this basis, the tool security configuration 303 and the data security configuration 304 can realize fine-grained security control over tool calling and data flow process by combining multi-level confidentiality and integrity label definitions and implementing trust domain declaration mechanisms in runtime sequence data.

[0058] In some embodiments, each running node can correspond to security description information. In some embodiments, the respective security description information of the plurality of running nodes can indicate a node security level. Alternatively or additionally, in some embodiments, the respective security description information of the plurality of running nodes can indicate a node data integrity. Alternatively or additionally, in some embodiments, the respective security description information of the plurality of running nodes can indicate a node trust level. Alternatively or additionally, in some embodiments, the respective security description information of the plurality of running nodes can indicate a node access constraint related to runtime sequence data (e.g., context). In this way, the security properties of the running nodes can be comprehensively and accurately described.

[0059] In some embodiments, the security description information can be represented by a multi-dimensional security label. For example, the node security level can indicate a confidentiality level of the running node. The node data integrity can indicate a data integrity level of the running node. The node security level and the node data integrity can be simultaneously represented by a first dimension of the multi-dimensional security label. The node trust level can indicate a trust level of an execution environment in which the running node is located. The node trust level can be represented by a second dimension of the multi-dimensional security label. The node access constraint can indicate a fine-grained access constraint of the running node varying with the runtime sequence data. For example, the node access constraint can indicate “only allow access to white list domain names when the parameter is a link”, and the like. The node access constraint can be represented by a third dimension of the multi-dimensional security label. In this case, the security description information can also be referred to as a three-dimensional security label.

[0060] In some embodiments, the electronic device 110 can generate initial security description information (hereinafter also referred to as reference security description information) for the function nodes and the data nodes in the second structured representation and the third structured representation before generating the second structured representation and the third structured representation mentioned in the foregoing. Then, in the process of fusing the second structured representation and the third structured representation to generate the first structured representation, the operation nodes in the first structured representation can inherit the security description information of the corresponding nodes in the second structured representation and the third structured representation. Specifically, for any operation node (for example, the first operation node) in the plurality of operation nodes, the operation node is generated by fusing the corresponding function node in the second structured representation and the corresponding data node in the third structured representation in the fusion process of the second structured representation and the third structured representation. On this basis, the electronic device 110 can determine the reference security description information of the corresponding function node and the corresponding data node based on at least one of the tool security configuration 303 or the data security configuration 304. Then, the electronic device 110 can determine the security description information of the first operation node by inheriting the reference security description information of the corresponding function node and the corresponding data node in the fusion process of the corresponding function node and the corresponding data node.

[0061] In the process of fusing different structured representations (such as the control flow graph and the data flow graph) to construct the first structured representation, the operation nodes achieve automatic transmission and consistent maintenance of security attributes by inheriting the security description information of the corresponding function nodes and the data nodes. Therefore, the accuracy and reliability of the structured representation in security analysis can be improved. In addition, the embodiments of the present disclosure also support the dual basis of the tool security configuration 303 and the data security configuration 304 to dynamically derive the security level of the fused operation node. This way not only enhances the ability to identify potential risks in complex scenarios, but also provides fine-grained data support for subsequent security verification.

[0062] In some embodiments, in the security description information, the node security level, the node data integrity, and the node trust level (e.g., the first two dimensions of the multi-dimensional security label) can be determined based on the content in the tool security configuration 303 related to at least one of the confidentiality level, the data integrity, and the trust level. Alternatively or additionally, in some embodiments, in the security description information, the node security level, the node data integrity, and the node trust level (e.g., the first two dimensions of the multi-dimensional security label) can be determined based on the content in the data security configuration 304 related to at least one of the confidentiality level, the data integrity, and the trust level. In some embodiments, the node access constraint (e.g., the last dimension of the multi-dimensional security label) can be determined based on a configurable constraint generation policy. Alternatively or additionally, in some embodiments, the node access constraint can be determined based on the metadata of the available tools.

[0063] In some embodiments, the constraint generation policy can be maintained in the policy configuration module 306. The node security level, the node data integrity, and the node trust level can be derived based on the security claims or security labels in the tool security configuration 303 and the data security configuration 304 related to the confidentiality level, the data integrity, and the trust level. In some embodiments, the node access constraint can be automatically generated based on a constraint generation policy defined by the user or the metadata of the available tools stored in the tool security configuration 303.

[0064] In this way, the first two dimensions of the multi-dimensional security label are derived from the tool security configuration 303 and the data security configuration 304, ensuring the consistency of the security description. While the last dimension of the multi-dimensional security label can be generated on demand based on the constraint generation policy, etc., thereby enhancing the adaptability to complex scenarios.

[0065] In some embodiments, the electronic device 110 can determine, based on the security description information of a second running node among the plurality of running nodes, the security description information of a third running node connected to the second running node among the plurality of running nodes. In this way, the security description information of the previous running node can be inherited by the subsequent running node, thereby achieving the propagation and accumulation of security attributes in the first structured representation.

[0066] In some embodiments, a fourth running node in the plurality of running nodes is connected to a group of running nodes in the plurality of running nodes. On this basis, the electronic device 110 can determine the security description information of the fourth running node based on the security description information of the running node with the highest security requirement in the group of running nodes. In this way, in the case where a plurality of running nodes converge to the same subsequent node (such as the fourth running node), the electronic device 110 can determine the security description information of the fourth running node according to the security description information of the node with the highest security requirement in the group of predecessor nodes. In this way, it can effectively prevent the weakening or degradation of security attributes caused by multi-source information fusion, thereby ensuring that the protection strength of high-sensitive information in the entire process will not be weakened.

[0067] With reference back to Figure 2 After obtaining the security description information of each running node, the electronic device 110 performs security verification on at least one task running process of the agent 120 based on the corresponding security description information of the plurality of running nodes at block 230.

[0068] In some embodiments, for any running node, the electronic device 110 can perform a corresponding verification operation based on each item of content in the security description information of the running node, to determine whether the running node passes the security verification. In some embodiments, for a fifth running node in the running nodes, if the access process involved in the fifth running node does not meet the node access constraint condition (for example, the running node is configured as “only allow access to white list domain name”, but the actual access object is a non-white list domain name), the electronic device 110 can determine that the node access constraint condition is violated. In this case, the electronic device 110 can determine that the fifth running node does not pass the security verification. If the data of the fifth running node is transmitted to a sixth running node in the plurality of running nodes and the node security level of the sixth running node is lower than that of the fifth running node (for example, high confidentiality data flows to a running node with low security level), the electronic device 110 can determine that there is a risk of information leakage or unauthorized access. In this case, the electronic device 110 can determine that the fifth running node does not pass the security verification. If the data of the fifth running node is transmitted to the sixth running node and the trust level of the sixth running node is lower than that of the fifth running node (for example, high trust data flows to an untrusted running node), the electronic device 110 can determine that there is a risk of information leakage or unauthorized access. In this case, the electronic device 110 can determine that the fifth running node does not pass the security verification.

[0069] This dynamic propagation security verification method based on structured representation not only covers the complex interaction logic in the at least one task running process of the agent 120, but also identifies and blocks potential malicious behavior at an early stage. In this way, the overall security of the system can be significantly improved.

[0070] In some embodiments, if the electronic device 110 detects that the agent 120 performs a new processing function during at least one task execution process, the electronic device 110 can update the first structured representation based on the new processing function. For example, the electronic device 110 can insert the new processing function as a new running node into the first structured representation. Then, the electronic device 110 can update the security check of the at least one task execution process of the agent 120 based on the updated first structured representation. For example, the electronic device 110 can perform the security check in the updated first structured representation based on the security check manner described above. In this way, the electronic device 110 can immediately cancel the current call when discovering that new high-sensitive data is leaked across domains or low-trust instructions drive high-privilege operations, thereby achieving immediate response to abnormal behaviors.

[0071] In some embodiments, if the at least one task execution process of the agent 120 fails the security check, the electronic device 110 can mark the running node that fails the security check in the first structured representation. This marking mechanism can identify the location of the violation in a visual manner, which can be used for subsequent risk analysis and repair guidance. In some embodiments, the electronic device 110 can also analyze the reason why the security check fails and generate a related report, etc.

[0072] According to the various embodiments described above, it can be clearly understood that the embodiments of the present disclosure provide a full-link security protection framework for natural language driven agents 120 (LLM Agent). The tool security configuration 303 and the data security configuration 304 provide a unified security check basis for the system, while the structured representation and the security check process respectively implement the formal modeling of the behavior of the agent 120 and the dynamic security guarantee. The embodiments of the present disclosure can solve the problem that the traditional scheme lacks formal behavior description and systematic security check mechanism when facing agents 120 with characteristics such as natural language driving, dynamic execution path, cross-trust domain calling, etc. The embodiments of the present disclosure map the fuzzy natural language instructions, multi-step tool calling, and external side effects into control flow graphs, data flow graphs, and program dependency graphs, etc. through a unified mapping mechanism, thereby achieving a precise characterization of the interaction process of the agent 120. Figure 3 The embodiments of the present disclosure introduce a security check mechanism based on label propagation, thereby enabling real-time identification of sensitive data leakage and low-trust instruction overreach, etc. At the same time, the embodiments of the present disclosure also support incremental runtime checking and explanatory report generation, which not only ensures low invasiveness to the logic of the agent 120, but also significantly improves the transparency and auditability of security decisions.

[0073] Embodiments of the present disclosure also provide a corresponding apparatus for implementing the above method or process. Figure 4 A schematic structural block diagram of an apparatus 400 for security verification of an agent according to some embodiments of the present disclosure is shown. The apparatus 400 can be implemented as or included in the electronic device 110. Various modules / components in the apparatus 400 can be implemented by hardware, software, firmware, or any combination thereof.

[0074] With reference to Figure 4 The apparatus 400 includes a structured representation generation module 410, a security description information determination module 420, and a security verification module 430. The structured representation generation module 410 is configured to generate, based on runtime sequence data corresponding to at least one task execution process of an agent, a first structured representation describing dependency relationships in the at least one task execution process; wherein the first structured representation includes a plurality of execution nodes corresponding to a plurality of processing functions invoked in the at least one task execution process and data in the at least one task execution process. The security description information determination module 420 is configured to determine, based on at least tool security configurations related to available tools of the agent and data security configurations related to accessible data objects of the agent, respective security description information of the plurality of execution nodes in the first structured representation. The security verification module 430 is configured to perform security verification on the at least one task execution process of the agent based on the respective security description information of the plurality of execution nodes.

[0075] In some embodiments, the structured representation generation module 410 is further configured to generate, based on the runtime sequence data, a second structured representation describing control relationships between the plurality of processing functions, generate, based on at least the second structured representation, a third structured representation describing data flow in the at least one task execution process, and generate the first structured representation by fusing the second structured representation and the third structured representation.

[0076] In some embodiments, the plurality of processing functions includes an action execution function, and the structured representation generation module 410 is further configured to determine, based on the plurality of processing functions, a plurality of function nodes corresponding to the plurality of processing functions in the second structured representation, and determine, for an action function node corresponding to the action execution function in the plurality of function nodes, at least one control path including the action function node in the second structured representation based on control relationships between the action execution function and other processing functions in the plurality of processing functions except the action execution function.

[0077] In some embodiments, the plurality of processing functions further comprises a decision function, and the structured representation generation module 410 is further configured to: split the action function node into a first sub-node and a second sub-node, wherein the first sub-node corresponds to a tool to be used by the action execution function, and the second sub-node corresponds to an input parameter of the tool; determine a control path between the first sub-node and a decision function node and a control path between the second sub-node and the decision function node based on a control relationship between the action execution function and the decision function, wherein the decision function node is a function node in the plurality of function nodes corresponding to the decision function.

[0078] In some embodiments, the plurality of processing functions comprises an action execution function, a decision function, a prompt word processing function, and an observation information acquisition function, and the control relationship between at least one of the decision function, the prompt word processing function, or the observation information acquisition function and the action execution function is determined using a machine learning model.

[0079] In some embodiments, the structured representation generation module 410 is further configured to: extract at least one function node related to data flow from the plurality of function nodes of the second structured representation as part of the plurality of data nodes of the third structured representation; create at least one additional node as another part of the plurality of data nodes based on at least external data objects accessed by the available tools when invoked; and determine a connection relationship between the plurality of data nodes based on a dependency relationship of data in at least one task running process to obtain the third structured representation.

[0080] In some embodiments, for a first running node in the plurality of running nodes, the first running node is generated by fusing a corresponding function node in the second structured representation and a corresponding data node of the third structured representation in a fusion process of the second structured representation and the third structured representation, and the structured representation generation module 410 is further configured to: determine reference security description information of the corresponding function node and the corresponding data node based on at least one of a tool security configuration or a data security configuration; and determine security description information of the first running node by inheriting the reference security description information of the corresponding function node and the corresponding data node in the fusion process of the corresponding function node and the corresponding data node.

[0081] In some embodiments, the security description information determination module 420 is further configured to: determine security description information of a third running node connected after the second running node in the plurality of running nodes based on the security description information of the second running node.

[0082] In some embodiments, a fourth running node in the plurality of running nodes is connected to a group of running nodes in the plurality of running nodes, and the security description information determination module 420 is further configured to determine the security description information of the fourth running node based on the security description information of a running node in the group of running nodes that has the highest security requirement.

[0083] In some embodiments, the tool security configuration comprises a first security identification for data involved by the available tool, the data security configuration comprises a second security identification for an accessible data object, and at least one of the first security identification or the second security identification indicates at least one of a security sensitivity or a trust domain.

[0084] In some embodiments, the data involved by the available tool comprises at least one of an input parameter of the available tool, a return value of the available tool, or data related to a side effect of the available tool. The accessible data object comprises at least one of an accessible file of the agent or an accessible link of the agent.

[0085] In some embodiments, the respective security description information of the plurality of running nodes indicates at least one of a node security level, a node data integrity, a node trust level, or a node access constraint related to runtime data.

[0086] In some embodiments, the node security level, the node data integrity, and the node trust level are determined based on at least one of content in the tool security configuration related to at least one of a confidentiality level, a data integrity, and a trust level, or content in the data security configuration related to at least one of a confidentiality level, a data integrity, and a trust level. The node access constraint is generated based on at least one of a configurable constraint generation policy or metadata of the available tool.

[0087] In some embodiments, the security check module 430 is further configured to perform at least one of the following for a fifth running node in the running nodes: determine that the fifth running node fails the security check in response to an access procedure involved by the fifth running node not satisfying the node access constraint, determine that the fifth running node fails the security check in response to data of the fifth running node being transmitted to a sixth running node in the plurality of running nodes and a node security level of the sixth running node being lower than a node security level of the fifth running node, or determine that the fifth running node fails the security check in response to data of the fifth running node being transmitted to the sixth running node and a trust level of the sixth running node being lower than a trust level of the fifth running node.

[0088] In some embodiments, the apparatus 400 further includes an update module. The update module is configured to: in response to detecting that the agent performs a new processing function during at least one task execution, update the first structured representation based on the new processing function; and update the security check of the at least one task execution process of the agent based on the updated first structured representation.

[0089] In some embodiments, the apparatus 400 further includes a marking module configured to: in response to at least one task execution process of the agent failing to pass the safety check, mark the execution nodes that failed the safety check in the first structured representation.

[0090] Figure 5 1 is a block diagram of an electronic device 500 in which one or more embodiments of the present disclosure may be implemented. The electronic device 500 may be used to implement, for example, Figure 1 The electronic device 110 shown or Figure 4 The device 400 shown. It should be understood that Figure 5 The illustrated electronic device 500 is merely exemplary and should not be construed as limiting the functionality and scope of the embodiments described herein.

[0091] Reference Figure 5 , electronic device 500 is in the form of a general electronic device. Components of electronic device 500 may include, but are not limited to, one or more processors 510, memory 520, storage device 530, one or more communication units 540, one or more input devices 550, and one or more output devices 560. Processor 510 may be a real or virtual processor and is capable of performing various processes according to a program stored in memory 520. In a multi-processor system, multiple processors execute computer-executable instructions in parallel to increase the parallel processing capabilities of electronic device 500.

[0092] The electronic device 500 typically includes a plurality of computer storage media. Such media can be any available media accessible to the electronic device 500, including but not limited to volatile and non-volatile media, removable and non-removable media. The memory 520 can be a volatile memory (e.g., registers, cache, random access memory (RAM)), a non-volatile memory (e.g., read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory), or some combination thereof. The storage device 530 can be a removable or non-removable medium and can include a machine-readable medium, such as a flash drive, a disk, or any other medium that can be used to store information and / or data and can be accessed within the electronic device 500.

[0093] The electronic device 500 may further include additional removable / non-removable, volatile / non-volatile storage media. Figure 5 As shown in FIG, a magnetic disk drive for reading from or writing to a removable, non-volatile magnetic disk (e.g., a "floppy disk") and an optical disk drive for reading from or writing to a removable, non-volatile optical disk may be provided. In these cases, each drive may be connected to a bus (not shown) by one or more data media interfaces. Memory 520 may include a computer program product 525 having one or more program modules configured to perform various methods or actions of various embodiments of the present disclosure.

[0094] The communication unit 540 enables communication with other electronic devices via a communication medium. Additionally, the functions of the components of the electronic device 500 can be implemented in a single computing cluster or multiple computing machines that can communicate via a communication connection. Thus, the electronic device 500 can operate in a networked environment using a logical connection with one or more other servers, a network personal computer (PC), or another network node.

[0095] Input device 550 may be one or more input devices, such as a mouse, keyboard, or trackball. Output device 560 may be one or more output devices, such as a display, a speaker, or a printer. Electronic device 500 may also communicate with one or more external devices (not shown) via communication unit 540 as needed, such as a storage device, a display device, or the like, with one or more devices that allow a user to interact with electronic device 500, or with any device that allows electronic device 500 to communicate with one or more other electronic devices (e.g., a network card, a modem, etc.). Such communication may be performed via an input / output (I / O) interface (not shown).

[0096] According to an exemplary implementation of the present disclosure, a computer-readable storage medium is provided, on which computer-executable instructions are stored, wherein the computer-executable instructions are executed by a processor to implement the method described above. According to an exemplary implementation of the present disclosure, a computer program product is also provided, which is tangibly stored on a non-transitory computer-readable medium and includes computer-executable instructions, and the computer-executable instructions are executed by a processor to implement the method described above.

[0097] Various aspects of the present disclosure are described herein with reference to flowcharts and / or block diagrams of methods, apparatuses, devices, and computer program products implemented according to the present disclosure. It should be understood that each block of the flowcharts and / or block diagrams, and combinations of blocks in the flowcharts and / or block diagrams, can be implemented by computer-readable program instructions.

[0098] These computer-readable program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, thereby producing a machine such that when these instructions are executed by the processor of the computer or other programmable data processing device, a device is generated that implements the functions / actions specified in one or more blocks in the flowchart and / or block diagram. These computer-readable program instructions can also be stored in a computer-readable storage medium, where these instructions cause the computer, programmable data processing device, and / or other device to operate in a specific manner. Thus, the computer-readable medium storing the instructions comprises an article of manufacture that includes instructions for implementing various aspects of the functions / actions specified in one or more blocks in the flowchart and / or block diagram.

[0099] Computer-readable program instructions can be loaded onto a computer, other programmable data processing apparatus, or other device so that a series of operational steps are performed on the computer, other programmable data processing apparatus, or other device to produce a computer-implemented process, thereby causing the instructions executed on the computer, other programmable data processing apparatus, or other device to implement the functions / actions specified in one or more boxes in the flowchart and / or block diagram.

[0100] The flow charts and block diagrams in the accompanying drawings show the possible architecture, functions and operations of the systems, methods and computer program products according to multiple implementations of the present disclosure. In this regard, each box in the flow chart or block diagram can represent a part for a module, program segment or instruction, and a part for a module, program segment or instruction comprises one or more executable instructions for realizing the logical function of the specification. In some alternative implementations, the functions marked in the box can also occur in a sequence different from that marked in the accompanying drawings. For example, two continuous boxes can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram and / or flow chart, and the combination of the boxes in the block diagram and / or flow chart can be realized by a special hardware-based system that performs the function or action of the specification, or can be realized by a combination of special hardware and computer instructions.

[0101] While various implementations of the present disclosure have been described above, the foregoing description is intended to be illustrative, non-exhaustive, and not limited to the disclosed implementations. Many modifications and variations will be apparent to those skilled in the art without departing from the scope and spirit of the described implementations. The terminology used herein is intended to best explain the principles of the implementations, their practical applications, or improvements to existing technologies, or to enable others skilled in the art to understand the various implementations disclosed herein.

Claims

1. A security verification method for an intelligent agent, comprising: Generate, based on runtime timing data corresponding to at least one task execution process of an agent, a first structured representation describing dependency relationships during the at least one task execution process; wherein the first structured representation includes a plurality of execution nodes corresponding to the following items: a plurality of processing functions called during the at least one task execution process, and data during the at least one task execution process; Determining corresponding security description information of the plurality of execution nodes in the first structured representation based at least on a tool security configuration related to available tools of the agent and a data security configuration related to accessible data objects of the agent; and Based on the corresponding security description information of the multiple running nodes, a security check is performed on the at least one task running process of the agent.

2. The method according to claim 1, wherein Generating a first structured representation describing the dependency relationship during the at least one task execution includes: generating, based on the runtime timing data, a second structured representation describing a control relationship between the plurality of processing functions; generating, based at least on the second structured representation, a third structured representation describing a data flow during the at least one task execution; and The first structured representation is generated by fusing the second structured representation and the third structured representation.

3. The method according to claim 2, wherein: The plurality of processing functions include action execution functions, and generating a second structured representation describing a control relationship between the plurality of processing functions includes: Based on the plurality of processing functions, determining a plurality of function nodes in the second structured representation corresponding to the plurality of processing functions; and For the action function node corresponding to the action execution function among the multiple function nodes, based on the control relationship between the action execution function and other processing functions among the multiple processing functions except the action execution function, determine at least one control path including the action function node in the second structured representation.

4. The method according to claim 3, wherein: The plurality of processing functions further include a decision function, and determining at least one control path in the second structured representation that includes the action function node comprises: Splitting the action function node into a first sub-node and a second sub-node, wherein the first sub-node corresponds to a tool to be used by the action execution function, and the second sub-node corresponds to an input parameter of the tool; and Based on the control relationship between the action execution function and the decision function, determine the control path between the first sub-node and the decision function node and the control path between the second sub-node and the decision function node, wherein the decision function node is a function node corresponding to the decision function among the multiple function nodes.

5. The method according to claim 2, wherein: The multiple processing functions include an action execution function, a decision-making function, a prompt word processing function and an observation information acquisition function. The control relationship between at least one of the decision-making function, the prompt word processing function or the observation information acquisition function and the action execution function is determined using a machine learning model.

6. The method according to claim 2, wherein: Generating a third structured representation describing the data flow during the at least one task execution includes: Extracting at least one functional node related to the data flow from the plurality of functional nodes in the second structured representation as part of the plurality of data nodes in the third structured representation; creating at least one additional node as another portion of the plurality of data nodes based at least on an external data object accessed by the available tool when invoked; and Based on the dependency relationship of the data during the at least one task execution, the connection relationship between the multiple data nodes is determined to obtain the third structured representation.

7. The method according to claim 2, wherein: For a first running node among the multiple running nodes, the first running node is generated by fusing corresponding function nodes in the second structured representation and corresponding data nodes in the third structured representation during a fusion process of the second structured representation and the third structured representation, and determining corresponding security description information of the multiple running nodes in the first structured representation includes: Determining reference security description information of each of the corresponding function node and the corresponding data node based on at least one of the tool security configuration or the data security configuration; and During the fusion process of the corresponding functional node and the corresponding data node, the security description information of the first running node is determined by inheriting the respective reference security description information of the corresponding functional node and the corresponding data node.

8. The method according to claim 1, wherein Determining corresponding security description information of the plurality of running nodes in the first structured representation includes: Based on the security description information of a second running node among the plurality of running nodes, the security description information of a third running node among the plurality of running nodes that is connected to the second running node is determined.

9. The method according to claim 1, wherein A fourth running node among the plurality of running nodes is connected to a group of running nodes among the plurality of running nodes, and determining corresponding security description information of the plurality of running nodes in the first structured representation includes: The security description information of the fourth running node is determined based on the security description information of the running node with the highest security requirement in the group of running nodes.

10. The method according to claim 1, wherein The tool security configuration includes a first security identifier for data involved in the available tool, and the data security configuration includes a second security identifier for the accessible data object, wherein at least one of the first security identifier and the second security identifier indicates at least one of the following: security sensitivity, or Trust domain.

11. The method according to claim 10, wherein: The data involved in the available tools include at least one of the following: The input parameters of the available tools are: The return value of the available tools, or Data on the side effects of the available tools; and The accessible data object includes at least one of the following: The agent's accessible files, or Accessible links for the agent.

12. The method according to claim 1, wherein The corresponding security description information of the multiple running nodes indicates at least one of the following: Node security level, Node data integrity, Node credibility, or Node access constraints related to the runtime data.

13. The method according to claim 12, wherein: The node security level, the node data integrity, and the node trustworthiness are determined based on at least one of the following: Content related to at least one of confidentiality, data integrity, and trustworthiness in the tool security configuration, or The content of the data security configuration related to at least one of confidentiality, data integrity and credibility, and The node access constraint is generated based on at least one of the following: Configurable constraint generation strategies, or Metadata about the available tools.

14. The method according to claim 12, wherein: Performing a security check on the at least one task execution process of the agent includes: performing at least one of the following on a fifth execution node among the execution nodes: In response to an access process involving the fifth operating node not satisfying the node access constraint condition, determining that the fifth operating node fails the security check, In response to data of the fifth running node being transmitted to a sixth running node among the plurality of running nodes, and the node security level of the sixth running node being lower than the node security level of the fifth running node, determining that the fifth running node fails the security check, or In response to data of the fifth running node being transmitted to the sixth running node, and the trustworthiness of the sixth running node being lower than the trustworthiness of the fifth running node, it is determined that the fifth running node fails the security check.

15. The method according to claim 1, further comprising: In response to detecting that the agent performs a new processing function during the at least one task execution, updating the first structured representation based on the new processing function; as well as Based on the updated first structured representation, the safety check of the at least one task execution process of the agent is updated.

16. The method according to claim 1, further comprising: In response to at least one task execution process of the agent failing to pass the safety check, the execution nodes that failed to pass the safety check are marked in the first structured representation.

17. A security verification device for an intelligent agent, comprising: A structured representation generation module is configured to generate, based on runtime timing data corresponding to at least one task execution process of an agent, a first structured representation describing dependency relationships during the at least one task execution process; wherein the first structured representation includes a plurality of execution nodes corresponding to the following items: a plurality of processing functions called during the at least one task execution process, and data during the at least one task execution process; a security description information determination module configured to determine corresponding security description information of the plurality of execution nodes in the first structured representation based on at least a tool security configuration related to available tools of the agent and a data security configuration related to accessible data objects of the agent; and The safety verification module is configured to perform a safety verification on the at least one task running process of the agent based on the corresponding security description information of the multiple running nodes.

18. An electronic device comprising: at least one processor; as well as At least one memory is coupled to the at least one processor and stores instructions for execution by the at least one processor, the instructions causing the electronic device to perform the method according to any one of claims 1 to 16 when executed by the at least one processor.

19. A computer-readable storage medium having computer-executable instructions stored thereon, wherein the computer-executable instructions can be executed by a processor to implement the method according to any one of claims 1 to 16.

20. A computer program product comprising computer executable instructions, wherein the computer executable instructions, when executed by a processor, implement the method according to any one of claims 1 to 16.

Images