Methods, devices, equipment, media and products for detecting security risks in intelligent agents

Abstract

This application provides a method, apparatus, device, medium, and product for detecting security vulnerabilities in intelligent agents. It relates to the field of information security technology. The method includes: determining the intent type, perplexity value, and back-translation similarity of input text; determining that the input text passes detection if all corresponding preset conditions are met; using a preset classification model to determine the security category of each MCP tool to be used based on its metadata, to obtain a target MCP tool list; generating a global tool dependency graph based on the detected input text and the target MCP tool list; using a preset graph neural network model to construct a security detection graph based on the global tool dependency graph, to verify whether the global tool dependency graph can be executed; and controlling the intelligent agent to perform operations based on the global tool dependency graph in response to successful verification. This application achieves comprehensive coverage of security vulnerabilities during intelligent agent runtime through comprehensive detection of input text, MCP tools, and inference chains.

Description

Technical Field

[0001] This application relates to the field of information security technology, and in particular to a method, apparatus, equipment, medium and product for detecting security vulnerabilities in intelligent agents. Background Technology

[0002] With the rapid development of large language models and intelligent agent paradigms, large model intelligent agents have evolved from simple question-and-answer assistants into autonomous systems capable of perceiving their environment, planning autonomously, and invoking external tools to execute complex tasks. Leveraging MCP tool integration technology, they achieve cross-system collaborative operations, efficiently completing diverse tasks such as expense reimbursement processing, schedule management, and data interaction. However, the complex interactive capabilities of intelligent agents also bring new security challenges. Attackers can induce illegal operations through adversarial input, tampering with tool metadata, and injecting malicious commands. These attacks are highly covert, employ diverse methods, and are difficult to counter with traditional defenses.

[0003] Currently, four core methods have been developed for security protection of large-scale intelligent agents. Input / output security filtering mechanisms use sensitive word libraries, regular expression rules, and classification models to detect and intercept user input and model output in real time. Prompt word security hardening strategies embed security rules into prompt words to constrain model inference logic and reduce risks. The principle of least privilege strictly controls access permissions for intelligent agent tools, granting only the minimum set of permissions necessary for the task. Tool signature and metadata verification mechanisms verify the integrity of tool metadata through digital signatures to ensure the trustworthiness of the source.

[0004] However, the input risk detection is not robust enough and struggles to identify emerging attacks such as multi-turn combinations and semantic spoofing. It cannot prevent tool poisoning caused by tool description tampering, and it cannot identify threats such as inference hijacking and decision deviation in runtime inference trajectories. Summary of the Invention

[0005] The intelligent agent security vulnerability detection method, device, equipment, medium and product provided in the embodiments of this application are used to solve the problem of insufficient detection of intelligent agents' full-link security vulnerabilities.

[0006] In a first aspect, embodiments of this application provide a method for detecting security vulnerabilities in intelligent agents, the method comprising:

[0007] Determine the intent type, perplexity value, and back-translation similarity of the input text;

[0008] If the intent type, perplexity value, and back-translation similarity all meet the corresponding preset conditions, then the input text detection is determined to be successful.

[0009] A pre-defined classification model is used to determine the security category of each MCP tool to be used based on the metadata of each MCP tool to be used, so as to obtain a list of target MCP tools;

[0010] A global tool dependency graph is generated based on the detected input text and the target MCP tool list;

[0011] A preset graph neural network model is used to construct a security detection graph based on the global tool dependency graph to verify whether the global tool dependency graph can be executed;

[0012] Upon successful verification, the control agent performs operations based on the global tool dependency graph.

[0013] In one possible implementation, determining the intent type, perplexity value, and back-translation similarity of the input text includes:

[0014] The intent type of the input text is determined based on its semantic features;

[0015] The conditional probability value of each tag is calculated based on the word segmentation results of the input text using a preset language model;

[0016] The perplexity value of the input text is determined based on the conditional probability value of each marker;

[0017] The input text is translated multiple times according to a preset language chain order to obtain the target back-translated text;

[0018] Calculate the semantic vector cosine similarity between the target back-translated text and the input text to determine the back-translation similarity.

[0019] In one possible implementation, the step of using a preset classification model to determine the security category of each MCP tool to be used based on the metadata of each MCP tool to be used, in order to obtain a list of target MCP tools, includes:

[0020] Obtain metadata for at least one MCP tool to be used from the MCP server;

[0021] Convert the text content of the metadata into a sequence of tags;

[0022] The security category of each MCP tool to be used is determined by using a pre-defined classification model based on the label sequence of each MCP tool to be used.

[0023] If an MCP tool with the default security category exists, delete that MCP tool to obtain a list of target MCP tools.

[0024] In one possible implementation, the global tool dependency graph includes tools, execution order, and dependencies; the step of constructing a security detection graph based on the global tool dependency graph using a preset graph neural network model to verify whether the global tool dependency graph is executable includes:

[0025] A pre-defined graph neural network model is used to construct a security detection graph with tools as nodes and dependencies between tools as edges.

[0026] Average pooling is performed on the feature vectors corresponding to each node to obtain the feature vectors corresponding to the inference trajectory;

[0027] The probability value of each preset detection category is calculated based on the feature vector corresponding to the inference trajectory;

[0028] The global tool dependency graph is determined to be executable based on the probability value.

[0029] In one possible implementation, it also includes:

[0030] If it is determined that the global tool dependency graph cannot be executed, then the gradient sensitivity of each node is calculated;

[0031] The current sensitivity threshold is determined based on the gradient sensitivity of each node;

[0032] If the gradient sensitivity of any node is greater than the current sensitivity threshold, the node is determined to be a potential security risk node.

[0033] The global tool dependency graph is modified based on the security vulnerability nodes so that operations can be performed based on the modified global tool dependency graph.

[0034] In one possible implementation, it also includes:

[0035] In response to the completion of the operation, labels are added to the global tool dependency graph based on the execution result;

[0036] Replanning is performed based on the tagged global tool dependency graph to generate an updated global tool dependency graph;

[0037] Based on the updated global tool dependency graph, security checks and control execution operations are repeatedly performed until the task for the currently input text is completed.

[0038] In one possible implementation, it also includes:

[0039] In response to obtaining the output text, determine whether the output text contains preset sensitive words according to the preset sensitive series table;

[0040] If it does not exist, calculate the probability of semantic security risks in the output text;

[0041] If the probability of the semantic security risk is less than or equal to a preset probability threshold, then the inconsistency value between the input text and the output text is calculated.

[0042] Based on the semantic security risk probability and inconsistency value, it is determined whether the output text can be output safely.

[0043] Secondly, embodiments of this application provide a smart agent security hazard detection device, comprising:

[0044] The determination module is used to determine the intent type, perplexity value, and back-translation similarity of the input text;

[0045] The determining module is further configured to determine that the input text detection has passed if the intent type, perplexity value, and back-translation similarity all meet the corresponding preset conditions.

[0046] The determining module is further configured to use a preset classification model to determine the security category of each MCP tool to be used based on the metadata of each MCP tool to be used, so as to obtain a list of target MCP tools;

[0047] The generation module is used to generate a global tool dependency graph based on the detected input text and the target MCP tool list;

[0048] A construction module is used to construct a security detection graph based on the global tool dependency graph using a preset graph neural network model, in order to verify whether the global tool dependency graph can be executed;

[0049] The control module is used to control the agent to perform operations based on the global tool dependency graph in response to successful verification.

[0050] Thirdly, embodiments of this application provide an intelligent agent security vulnerability detection device, including: a memory and a processor;

[0051] The memory stores computer-executed instructions;

[0052] The processor executes computer execution instructions stored in the memory, causing the processor to perform the first aspect and / or various possible implementations of the first aspect as described above.

[0053] Fourthly, embodiments of this application provide a computer-readable storage medium storing computer-executable instructions, which, when executed by a processor, are used to implement the first aspect and / or various possible implementations of the first aspect.

[0054] Fifthly, embodiments of this application provide a computer program product, including a computer program that, when executed by a processor, implements the first aspect and / or various possible implementations of the first aspect.

[0055] The intelligent agent security vulnerability detection method, apparatus, device, medium, and product provided in this application comprehensively judge whether the input text passes detection by determining the intent type, perplexity value, and back-translation similarity of the input text and setting corresponding preset conditions. Multi-dimensional evaluation effectively filters out abnormal, malicious, or unacceptable inputs, reducing false positives and false negatives, improving the robustness of input detection, preventing attackers from bypassing defenses using a single technique, and improving detection accuracy. Metadata contains key information about MCP tools; classification based on metadata can more accurately identify tools with security vulnerabilities, avoiding security risks caused by problems with the tools themselves, improving the security of tool calls, and preventing intelligent agents from executing illegal operations due to metadata pollution. A global tool dependency graph is generated based on the detected input text and the target MCP tool list, effectively reducing the risk of intent deviation and malicious instruction penetration after multiple rounds of thinking and execution, and helping to discover potential security vulnerabilities and conflicts in advance. A preset graph neural network model is used to construct a security detection graph based on the global tool dependency graph, and the executable nature of the global tool dependency graph is verified. Through a verification mechanism, it is ensured that only global tool dependency graphs that have undergone rigorous security checks can be executed, effectively preventing the execution of malicious code or unsafe operations, blocking attackers from gradually penetrating the attack chain through intermediate tool calls, and ensuring the secure operation of the system. This solution comprehensively covers security vulnerabilities during agent runtime, providing a systematic solution for the secure operation of highly autonomous agents. Attached Figure Description

[0056] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application.

[0057] Figure 1 This application provides an illustration of an method for detecting security risks in intelligent agents.

[0058] Figure 2 This is a flowchart illustrating a method for detecting security risks in intelligent agents according to an embodiment of this application.

[0059] Figure 3 A flowchart illustrating a method for detecting security risks in intelligent agents according to another embodiment of this application;

[0060] Figure 4 This is a schematic diagram of the structure of an intelligent agent security hazard detection device provided in an embodiment of this application;

[0061] Figure 5 This is a schematic diagram of the structure of an intelligent agent security hazard detection device provided in an embodiment of this application.

[0062] The accompanying drawings illustrate specific embodiments of this application, which will be described in more detail below. These drawings and descriptions are not intended to limit the scope of the concept in any way, but rather to illustrate the concept of this application to those skilled in the art through reference to particular embodiments. Detailed Implementation

[0063] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numbers in different drawings denote the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this application. Rather, they are merely examples of apparatuses and methods consistent with some aspects of this application as detailed in the appended claims.

[0064] It should be noted that in the embodiments of this application, certain software, components, models and other existing solutions in the industry may be mentioned. These should be regarded as exemplary and are only intended to illustrate the feasibility of implementing the technical solution of this application. However, they do not mean that the applicant has used or necessarily used the solution.

[0065] To clearly understand the technical solution of this application, the solutions of the prior art will be described in detail first.

[0066] With the rapid evolution of large language models and intelligent agent technologies, large model intelligent agents have evolved from basic question-and-answer formats to intelligent autonomous systems capable of perceiving their environment, autonomously planning tasks, and invoking external tools. These systems rely on MCP tool integration technology to achieve cross-system collaboration, efficiently completing complex tasks such as financial reimbursement, schedule management, and multi-source data interaction. However, their diverse interactive capabilities have also given rise to new security threats. Attackers can induce intelligent agents to perform illegal operations by constructing adversarial inputs, tampering with MCP tool metadata, and injecting malicious commands. These attacks are characterized by strong concealment and diverse methods, making them difficult to effectively counter with traditional security defenses. Currently, the industry's security protection for large model intelligent agents mainly consists of four core technical solutions: input / output security filtering mechanisms that conduct real-time detection and interception of user input and intelligent agent output through sensitive word libraries, regular rules, and classification models; prompt word security hardening strategies that embed security criteria in prompt words to constrain model inference logic; the principle of minimization of permissions, strictly controlling tool access permissions and only granting the minimum set of permissions required for the task; and tool signature and metadata verification mechanisms that verify the integrity of metadata through digital signatures to ensure the trustworthiness of the tool's origin. However, existing protection solutions still have significant shortcomings. The robustness of input risk detection is insufficient, making it difficult to identify new threats such as multi-round combined attacks and semantic spoofing attacks. They cannot effectively prevent tool poisoning caused by the tampering of tool descriptions. At the same time, they lack effective monitoring of the inference trajectory of intelligent agents during runtime, making it difficult to identify potential security risks such as inference hijacking and decision deviation.

[0067] Therefore, to address the insufficient robustness of intelligent agent input risk detection in the face of existing technical problems, this paper proposes a three-dimensional detection approach for input text: intent type, perplexity value, and back-translation similarity. This approach identifies abnormal input intent, constructed low-probability expressions, and semantic spoofing attacks, respectively. A collaborative verification mechanism based on preset conditions for each dimension is then used to determine the input detection results, intercepting multi-round combinations and semantic spoofing attacks at the input source, thus filling the gap in protection against single-dimensional detection. To address the lack of semantic-level malicious injection detection for MCP tool metadata, this paper designs security category judgment rules for tool metadata based on the semantic recognition capabilities of a pre-trained classification model. Metadata such as tool descriptions and parameter specifications are used as model input. The model output filters out safe and harmless MCP tools and forms a target tool list, ensuring a natural match between tool usage permissions and security attributes, achieving precise filtering of malicious tools. To further address the issue of insufficient monitoring of inference trajectories during agent runtime, a global tool dependency graph is generated based on the detected input text and the target MCP tool list. Since this dependency graph integrates legitimate input requirements and safe tool resources, a security detection graph is constructed on it through a preset graph neural network model to complete executability verification, ensuring dynamic monitoring and security verification of the inference trajectory, and ultimately achieving full-link security protection of input detection, tool verification, and inference monitoring.

[0068] Figure 1 This is an application scenario diagram illustrating the intelligent agent security vulnerability detection method provided in this application, such as... Figure 1 As shown in the diagram, the scenario corresponding to the intelligent agent security vulnerability detection method provided in this application includes: a terminal device 101 and a server 102. The intelligent agent security vulnerability detection device can be integrated into the server 102. It is understood that an intelligent agent system is deployed in the server 102.

[0069] Specifically, the user inputs a task request, i.e., input text, through terminal device 101. After receiving the input text, server 102 determines the intent type, perplexity value, and back-translation similarity of the input text; if all three dimensions meet the corresponding preset conditions, the input text is deemed to have passed detection. Simultaneously, server 102 obtains the metadata of each Model Context Protocol (MCP) tool to be used from the MCP server, and uses a preset classification model to determine the security category of each MCP tool based on its metadata, thus obtaining a list of target MCP tools. Then, server 102 generates a global tool dependency graph based on the detected input text and the list of target MCP tools; it then uses a preset graph neural network model to construct a security detection graph based on the global tool dependency graph to verify whether the global tool dependency graph can be executed. Finally, if the verification passes, the control agent performs the operation based on the global tool dependency graph. After execution, the output text is obtained, and server 102 sends the output text to terminal device 101 for the user to view.

[0070] The technical solution of this application and how the technical solution of this application solves the above-mentioned technical problems are described in detail below with specific embodiments. These specific embodiments can be combined with each other, and the same or similar concepts or processes may not be described again in some embodiments. The embodiments of this application will now be described with reference to the accompanying drawings.

[0071] Figure 2 This is a flowchart illustrating a method for detecting security vulnerabilities in intelligent agents according to an embodiment of this application, as shown below. Figure 2 As shown, the execution subject of this embodiment is an intelligent agent security vulnerability detection device. This device can be implemented through a computer program, or through a medium storing the relevant computer program, such as a USB flash drive and / or optical disc, or through a physical device integrating or installing the relevant computer program, such as a chip or intelligent agent security vulnerability detection equipment. The intelligent agent security vulnerability detection equipment can be a server, server cluster, etc. The intelligent agent security vulnerability detection method provided in this embodiment includes the following steps:

[0072] S201. Determine the intent type, perplexity value, and back-translation similarity of the input text.

[0073] Among them, intent type refers to the core request category of user input text determined by semantic analysis, such as legitimate intents like querying orders or changing passwords, or abnormal intents like unauthorized access to data or injection of malicious commands.

[0074] The perplexity score measures how well the input text conforms to the rules of natural language expression. The lower the score, the more deliberate and simplistic the text expression is, which may be an adversarial input constructed by attackers to bypass detection. The higher the score, the more the text conforms to the natural and diverse expression habits of humans, and the higher the probability of compliance.

[0075] Back-translation similarity refers to converting the input text into an intermediate language, such as English, using a translation tool, and then back-translating it into the original language, such as Chinese, to obtain the back-translated text. The semantic similarity between the original input text and the back-translated text is then calculated.

[0076] Specifically, the input text is first preprocessed to remove redundant spaces, special symbols and other interference information. Then, a pre-trained semantic classification model, such as a BERT-like model, is used to perform semantic parsing. Based on the semantic features, the corresponding intent category and confidence level are output. If the confidence level is higher than a preset threshold, it is determined to be the intent type; if the confidence level is lower than the threshold, it is temporarily judged as an ambiguous intent.

[0077] Furthermore, the input text is segmented into words to form an ordered sequence of tokens. A pre-trained language model is used to calculate the conditional probability of each token. The model relies on language patterns formed through training on massive amounts of natural text to output the conditional probability value corresponding to each token. The log probabilities of all tokens are then summed and divided by the total number of tokens to obtain the average log probability. The average log probability is then negativeized and exponentially calculated to obtain the final perplexity value.

[0078] Furthermore, a bidirectional translation operation is performed on the input text, translating the original input text into a preset intermediate language, which can be multiple languages, i.e., multiple translations, such as Chinese, English, and Japanese, to obtain intermediate translated text; then the intermediate translated text is translated back into the original language to generate back-translated text. Subsequently, the semantic similarity between the original input text and the back-translated text is calculated to obtain the back-translation similarity.

[0079] S202. If the intent type, perplexity value, and back-translation similarity all meet the corresponding preset conditions, then the input text detection is deemed successful.

[0080] Among them, the preset conditions refer to the judgment criteria set for the three detection dimensions of intent type, perplexity value, and back-translation similarity, such as preset intent type, preset threshold, etc.

[0081] Specifically, the preset conditions for intent types can include preset intent types such as privacy breach, value bias, false information, and harmless input. A preset large model can be used to directly output the intent type based on the input text. If the output intent type is a preset type with potential security risks, it is determined that the condition is not met. Alternatively, a preset confidence threshold can be set. During the determination, if the confidence level of the intent output by the model is higher than the preset threshold, the preset condition for the intent type is met; if the confidence level is lower than the threshold, it is determined that the condition is not met.

[0082] Optionally, the preset condition for the perplexity value can be a preset threshold. During the judgment, the calculated perplexity value of the input text is compared with the preset threshold. If the perplexity value is higher than the threshold, it means that the text conforms to the natural expression rules of humans and meets the preset condition; if it is lower than the threshold, it means that the text expression is deliberately monotonous and there is a suspicion of adversarial input, and the condition is not met.

[0083] Optionally, the preset condition for back-translation similarity can be a semantic consistency threshold. During the determination, the calculated back-translation similarity value is compared with the semantic consistency threshold. If the similarity value is higher than the threshold, it means that the original input text has no obvious semantic spoofing, the semantic consistency meets the standard, and the preset condition is satisfied; if it is lower than the threshold, it means that the text may have undergone semantic spoofing, which poses an attack risk and does not meet the condition.

[0084] It should be noted that intent detection is performed first during the determination process. If the preset conditions are not met, the rejection mechanism is triggered. If the preset conditions are met, the perplexity analysis and back-translation verification are performed in parallel. The rejection mechanism is triggered when either dimension meets the preset conditions.

[0085] S203. Using a preset classification model, determine the security category of each MCP tool to be used based on the metadata of each MCP tool to be used, so as to obtain a list of target MCP tools.

[0086] Among them, the pre-trained classification model refers to the semantic classification model that has been trained in advance and is used for security judgment of MCP tool metadata. Such as the fine-tuned version of pre-trained language models like BERT and RoBERTa, the model has learned the security features of a large number of labeled samples and can output the security category corresponding to the MCP tool metadata.

[0087] Among them, MCP tool metadata refers to the descriptive information of MCP tools, including tool function descriptions, parameter descriptions, permission requirements, calling constraints, and return result formats.

[0088] Among them, security category refers to the classification of the security status of MCP tool metadata, including safe and harmless, indirect injection risk, tool hijacking risk, and unauthorized attack risk.

[0089] The target MCP tool list refers to the set of MCP tools that are safe and harmless after being selected by a preset classification model. The list includes key information such as tool name, calling interface, and compliance metadata summary.

[0090] Specifically, the intelligent agent, acting as an MCP client, establishes a secure connection with the remote MCP server through identity authentication to obtain the complete metadata of the MCP tool to be used. The metadata is then cleaned, removing redundant formatting tags, invalid characters, and other interfering information, standardizing text representation, and extracting core semantic content to form structured metadata text. Each preprocessed MCP tool metadata entry is then input into a pre-defined classification model. The model extracts security-related features from the metadata through semantic parsing, matches them with a trained feature library, and outputs the corresponding security category and category confidence score.

[0091] Optionally, a classification confidence threshold, such as 0.8, can be preset. When the security category of the tool's metadata is determined to be safe and harmless and the confidence level is higher than the threshold, the tool is considered compliant and usable. If it is determined to be in another risk category, or if the confidence level of the safe and harmless category is lower than the threshold, the corresponding tool will be removed.

[0092] Furthermore, information on all compliance tools is compiled, summarizing key information such as tool name, API call, and compliance metadata, to generate a target MCP tool list.

[0093] S204. Generate a global tool dependency graph based on the detected input text and the target MCP tool list.

[0094] The global tool dependency graph is a structured graphical representation of the MCP tools required by an agent to perform a task, the order of tool calls, and logical relationships. It is an abstract mapping of the task execution process.

[0095] Among them, the input text that passes the detection refers to the input text that is determined to be free of adversarial attacks, semantic spoofing, and legitimate intent through collaborative verification of three dimensions: intent type, perplexity value, and back-translation similarity.

[0096] Specifically, the input text that passes the testing is analyzed to extract the core task requirements and tool invocation needs. Based on natural language processing technology, the semantic logic of the input text is deeply deconstructed to clarify the task objectives, execution steps, constraints, and required tool capabilities. Based on the deconstructed task flow and tool matching results, the invocation conditions and input / output parameters of each compliant tool are analyzed one by one to define the dependency types and constraints between tools, including input / output dependencies (the return result of a preceding tool is a mandatory parameter for subsequent tools), execution order dependencies (subsequent tools can only start after the preceding tool has completed execution and the result is valid), and conditional dependencies (subsequent tools are triggered only when the preceding tool executes successfully; failure terminates the corresponding chain). Simultaneously, invalid associations and circular dependencies between tools are eliminated to ensure that the dependency logic is clear and conflict-free.

[0097] For example, if the input text is "Query the Finance Department's Q1 2025 expense reimbursement orders, export them to Excel, and send them to the director's email," then the process is broken down into three core task stages: order query, Excel export, and email sending. The order of each stage and the input / output requirements are clearly defined. For instance, the order query results are used as input for the export tool, the exported file is used as input for the email sending tool, and access restrictions are imposed, limiting access to only the Finance Department's dedicated order query tool. During this process, the target MCP tool list is simultaneously linked to initially match the compliance tools corresponding to each task stage, establishing a correspondence between task stages and compliance tools.

[0098] Furthermore, taking the compliance tools in the target MCP tool list as the core, each tool is encapsulated as an independent node. Each node embeds key information such as the tool name, function description, parameter requirements, and calling constraints. Based on the identified dependencies, directed edges connect the corresponding tool nodes. The edge attributes annotate the dependency type and association details, such as the name of the passed parameter and whether it is a mandatory dependency, forming a complete graph structure.

[0099] S205. A security detection graph is constructed based on the global tool dependency graph using a preset graph neural network model to verify whether the global tool dependency graph can be executed.

[0100] Among them, the pre-set graph neural network model refers to a graph neural network model that has been trained in advance for the security detection scenario of tool call links of intelligent agents, such as the GAT graph attention network. The model has learned the security features and executable logic of nodes and edges in the tool dependency graph, and can perform feature extraction, risk modeling and executable verification on structured tool dependency graphs.

[0101] Among them, the security detection graph refers to a special detection graph formed by encoding the security features of nodes and edges and reconstructing the structure based on the global tool dependency graph and using a graph neural network model.

[0102] Specifically, first, all core elements are extracted from the global tool dependency graph, including all tool information and dependency attributes. Tools are treated as nodes, and dependencies between tools as edges. Feature mining is performed on nodes and edges separately, including compliance attributes such as tool function type, calling permissions, parameter requirements, and return value type; and association attributes such as dependency type, parameter passing rules, and execution order constraints. A pre-defined graph neural network model is loaded. The structured parsed global tool dependency graph data, along with the extracted and standardized node and edge features, are input into the pre-defined graph neural network model. Based on the original tool nodes, each node is assigned a quantized security feature vector encoded by the model; based on the original dependency edges, each edge is assigned a quantized association feature vector encoded by the model. The nodes and edges with feature vectors are reconnected to form a structure consistent with the original dependency graph. Figure 1 A security detection map that is consistent with each element and has quantified features.

[0103] Furthermore, the model uses the feature vectors of nodes in the security detection graph to determine whether each tool node is a valid and compliant node, and whether there are invalid nodes such as mismatch between tool functionality and task requirements, or missing tool call permissions. If an invalid node is found, the verification is directly deemed unsuccessful. Next, based on the feature vectors of edges in the detection graph and the connection relationships between nodes, the model verifies whether all dependencies conform to the basic logic of tool calls, focusing on identifying logical conflicts such as circular dependencies, unordered dependencies, and parameter passing mismatches, or unordered dependencies where sequentially executed tools are not ordered according to the task flow. If any logical conflict exists, the verification is deemed unsuccessful. Finally, the model combines the task objective of the input text to verify whether the tool call chain in the security detection graph can completely cover all stages of task execution, and whether there are missing links such as key task stages lacking corresponding tool nodes or key parameter passing lacking corresponding dependency edges. If a core link is missing, the verification is deemed unsuccessful.

[0104] S206. In response to successful verification, the control agent performs operations based on the global tool dependency graph.

[0105] Among them, execution operation refers to the intelligent agent following the tool calling logic and execution rules of the global tool dependency graph, sequentially scheduling and calling compliant tools in the target MCP tool list to complete the entire process of implementation actions from task initiation to goal achievement.

[0106] Specifically, the verified global tool dependency graph is passed to the agent. The agent traverses the entire chain to extract core content such as scheduling order, interface information, and dependency rules, and transforms it into a set of recognizable and executable instructions. Simultaneously, it initializes parameters such as business logic and identity authentication based on input requirements and binds them to corresponding tool nodes. The tool scheduling engine is then started, and tools are invoked in an orderly manner according to the dependency graph topology, following the principle of calling tools with no dependencies first and those with dependencies later. Nodes without prerequisite dependencies are directly invoked and their status monitored. Nodes with dependencies need to verify the successful execution of the preceding tools before passing valid results as input parameters. Parallel nodes are invoked synchronously, while conditional nodes are triggered or skipped according to rules.

[0107] The intelligent agent security vulnerability detection method provided in this application comprehensively judges whether the input text passes detection by determining the intent type, perplexity value, and back-translation similarity of the input text and setting corresponding preset conditions. Multi-dimensional evaluation effectively filters out abnormal, malicious, or unacceptable inputs, reducing false positives and false negatives, improving the robustness of input detection, preventing attackers from bypassing defenses using a single technique, and improving detection accuracy. Metadata contains key information about MCP tools; classification based on metadata can more accurately identify tools with security vulnerabilities, avoiding security risks caused by problems with the tools themselves, improving the security of tool calls, and preventing intelligent agents from executing illegal operations due to metadata pollution. A global tool dependency graph is generated based on the detected input text and the target MCP tool list, effectively reducing the risk of intent deviation and malicious instruction penetration after multiple rounds of thinking and execution, and helping to discover potential security vulnerabilities and conflicts in advance. A preset graph neural network model is used to construct a security detection graph based on the global tool dependency graph, and the executable nature of the global tool dependency graph is verified. Through a verification mechanism, it is ensured that only global tool dependency graphs that have undergone rigorous security checks can be executed, effectively preventing the execution of malicious code or unsafe operations, blocking attackers from gradually penetrating the attack chain through intermediate tool calls, and ensuring the secure operation of the system. This solution comprehensively covers security vulnerabilities during agent runtime, providing a systematic solution for the secure operation of highly autonomous agents.

[0108] As an optional implementation, based on the above embodiments, determining the intent type, perplexity value, and back-translation similarity of the input text includes:

[0109] The intent type of the input text is determined based on its semantic features;

[0110] The conditional probability value of each tag is calculated based on the word segmentation results of the input text using a pre-defined language model;

[0111] The perplexity value of the input text is determined based on the conditional probability value of each marker;

[0112] The input text is translated multiple times according to the preset language chain order to obtain the target back-translated text;

[0113] Calculate the semantic vector cosine similarity between the target back-translated text and the input text to determine the back-translation similarity.

[0114] Here, a token refers to the smallest semantic / syntactic unit obtained after the input text has been segmented.

[0115] The conditional probability value refers to the probability of a given tag appearing, predicted by a pre-defined language model, based on the sequence of all preceding tags in the input text.

[0116] The preset language chain order refers to a pre-defined multi-round translation language conversion path, which is a fixed language conversion order, such as Chinese → English → French → Chinese.

[0117] Among them, the target back-translated text refers to the text obtained by converting the input text back to the original language after multiple rounds of translation according to the preset language chain sequence.

[0118] Semantic vector cosine similarity refers to the vector similarity value calculated using the cosine algorithm after converting two texts into semantic vectors of fixed dimensions.

[0119] Specifically, the input text is first preprocessed, and a language model is used to extract semantic features, including core semantic features such as action direction, target audience, and operational purpose. Based on these semantic features, intent classification is performed, and the model directly outputs the type name, such as value bias, false information, or harmless input.

[0120] Furthermore, the preprocessed input text is segmented into words, as shown in the segmentation result: Load a pre-trained language model for the target language, input the ordered sequence of tokens after word segmentation into the model, and use the language model to calculate the conditional probability of each token. The average log probability is obtained by taking the logarithmic probability and summing them. Finally, the perplexity is obtained by negativeening the mean log probability and exponentializing it. Confusion level The calculation formula is as follows:

[0121] ;

[0122] .

[0123] Furthermore, the original language type of the input text is confirmed, and it is matched with a pre-defined language chain order. A high-precision machine translation tool is used to translate the input text round by round according to the language chain order, with each round using the translation result of the previous round as input. After completing all rounds of translation, the translation result of the last round is converted back to the original language of the input text to obtain the final target back-translated text. A pre-trained semantic encoding model is loaded, and the original input text and the target back-translated text are respectively input into the semantic encoding model to generate unique fixed-dimensional semantic vectors for each text. The cosine similarity algorithm is used to calculate the similarity between the two semantic vectors. By calculating the cosine value of the angle between the vectors, the semantic fit between the two texts is quantified, and the calculated cosine similarity value is the back-translation similarity of the input text.

[0124] The intelligent agent security vulnerability detection method provided in this application analyzes input text from different perspectives to form a complementary defense system, thereby improving the accuracy and robustness of detection. It clearly defines the intent type of the input text, coarsely screening its security, and immediately rejecting responses upon detecting security vulnerabilities, thus improving detection efficiency. Normal human expression is natural and diverse, resulting in high perplexity; however, carefully crafted attack inputs, in order to bypass simple keyword filtering, often use very specific expressions, leading to abnormally low perplexity in the language model. By calculating the perplexity value of the input text, implicit risk inputs can be effectively identified. Back-translation verification, through multilingual cyclic translation, can reveal the true semantic intent of the input. By cyclically translating the text across multiple languages, utilizing the semantic differences between different languages ​​and the inherent consistency checks of the translation model, the true meaning of the text is revealed.

[0125] As an optional implementation, based on the above embodiments, a preset classification model is used to determine the security category of each MCP tool to be used based on the metadata of each MCP tool to be used, so as to obtain a list of target MCP tools, including:

[0126] Obtain metadata for at least one MCP tool to be used from the MCP server;

[0127] Convert the text content of metadata into a sequence of tags;

[0128] The security category of each MCP tool to be used is determined by using a pre-defined classification model based on the label sequence of each MCP tool to be used.

[0129] If an MCP tool with the default security category exists, delete that MCP tool to obtain a list of target MCP tools.

[0130] The tag sequence refers to the ordered combination of the smallest semantic units formed after the text content of the MCP tool metadata is split according to natural language semantic rules.

[0131] Among them, the preset security category refers to the non-compliance risk category defined in advance for the risk scenarios of MCP tools, such as indirect injection risk, tool hijacking risk, and unauthorized attack risk.

[0132] Specifically, based on the tool capability requirements decomposed from the detected input text, a precise metadata retrieval request is initiated to the MCP server to obtain metadata for at least one MCP tool to be used. Each piece of MCP tool metadata is preprocessed to remove redundant formatting symbols, invalid placeholders, duplicate descriptions, and other interfering information, standardizing text representation while retaining the core semantic content of the metadata. Then, following natural language semantic segmentation rules, the preprocessed metadata text is segmented into a series of minimal tags with independent semantics, while maintaining the semantic order of the tags consistent with the original text, forming an ordered tag sequence.

[0133] Furthermore, a pre-trained classification model specifically designed for MCP tool metadata security assessment scenarios is loaded. The labeled sequence of each MCP tool to be used is input into this model one by one. The model extracts and analyzes semantic features from the labeled sequences, matching the extracted features with a security category feature library learned within the model. Based on the fit of the feature match, the security category corresponding to the MCP tool is output. If the confidence level is higher than a preset threshold, it is directly identified as belonging to that security category; if the confidence level is lower than the threshold, it is marked as a suspected risk category. The security category assessment results for all MCP tools to be used are checked one by one. If a certain MCP tool's security category belongs to a preset security category or is marked as a suspected risk category, the tool is directly removed from the list of tools to be used, eliminating all tools with potential security risks. After this process, only safe and harmless MCP tools remain in the list of tools to be used. Information on these compliant tools is then compiled, summarizing key information such as tool name, API calls, and core functions, ultimately forming a target MCP tool list.

[0134] The intelligent agent security risk detection method provided in this application determines the security category of each MCP tool to be used based on metadata using a preset classification model, which can accurately identify tools with security risks. Timely deletion of these tools that fall into the preset unsafe category can prevent potentially malicious tools from entering the system and avoid damage to system data and functions.

[0135] As an optional implementation, based on the above embodiments, the global tool dependency graph includes tools, execution order, and dependencies; a preset graph neural network model is used to construct a security detection graph based on the global tool dependency graph to verify whether the global tool dependency graph can be executed, including:

[0136] A pre-defined graph neural network model is used to construct a security detection graph with tools as nodes and dependencies between tools as edges.

[0137] Average pooling is performed on the feature vectors corresponding to each node to obtain the feature vectors corresponding to the inference trajectory;

[0138] The probability value of each preset detection category is calculated based on the feature vector corresponding to the inference trajectory;

[0139] Determine whether the global tool dependency graph can be executed based on the probability value.

[0140] The feature vector corresponding to the inference trajectory refers to the global feature vector obtained by averaging the feature vectors of all tool nodes in the security detection graph.

[0141] Among them, the preset detection category refers to the detection and judgment category set in advance for the global tool dependency graph, including injection attack, privilege escalation attack, safe and harmless, etc.

[0142] The global tool dependency graph is a tool dependency graph generated in one go using the Plan and Execute agent paradigm to generate tool execution plans.

[0143] Specifically, compliant MCP tools in the global tool dependency graph are used as nodes in the security detection graph, and each node is assigned a feature vector h encoded by the model. i The various dependencies between tools are then used as edges in the security detection graph. Each edge is labeled with attributes such as dependency type and parameter passing rules, and assigned a corresponding feature vector. Following the original topology of the global tool dependency graph, the model structurally connects the feature-laden nodes and edges to construct a complete security detection graph.

[0144] Furthermore, the feature vector h of all tool nodes in the security detection graph is extracted. i A mean pooling operation is performed on all node feature vectors, summing each vector dimension by dimension. The summation result is then divided by the total number of nodes to obtain a global feature vector h that incorporates information from all nodes in the entire graph. g h g The calculation formula is as follows:

[0145] .

[0146] Furthermore, give h gEach dimension's value is indexed to amplify the numerical differences; then, the index result for each dimension is divided by the sum of the index results for all dimensions to obtain the probability corresponding to that dimension.

[0147] For example, the preset detection categories are injection attack, privilege escalation attack, and safe and harmless. First, the 768-dimensional h... g It is compressed into a 3D vector through a fully connected layer, and then softmax is applied. Assume h... g After passing through a fully connected layer, a 3D vector is obtained: z = [0.09, 0.19, 0.29], corresponding to injection attacks, privilege escalation attacks, and safe / harmless attacks, respectively. Each value is then exponentialized: e ^ 0.09 ≈ 1.094, e ^ 0.19 ≈ 1.209, e ^ 0.29 ≈ 1.336. Calculate the total: 1.094 + 1.209 + 1.336 = 3.639. Calculate the probability for each category: Injection attack probability = 1.094 / 3.639 ≈ 0.301 (30.1%); Unauthorized attack probability = 1.209 / 3.639 ≈ 0.332 (33.2%); Safe and harmless probability = 1.336 / 3.639 ≈ 0.367 (36.7%). The highest risk category has a probability P(risk) = 33.2%, which is an unauthorized attack, therefore it is determined that execution is not allowed.

[0148] The intelligent agent security vulnerability detection method provided in this application constructs a security detection graph by treating tools as nodes and dependencies as edges. The model can capture complex interaction patterns between tools, thereby accurately identifying potential security vulnerabilities or malicious behavior patterns. Preset detection categories can cover various security aspects. Based on the feature vectors corresponding to the inference trajectory, the probability value of each preset detection category is calculated, enabling a comprehensive assessment of the risk level of the global tool dependency graph under different security scenarios. This avoids the omission of security vulnerabilities due to the limitations of a single security detection method. The graph neural network model, by analyzing the connections and feature vectors between nodes in the security detection graph, can verify whether the execution order of the tools conforms to logic and business requirements, preventing system malfunctions or erroneous results due to incorrect execution order.

[0149] As an optional implementation, based on the above embodiments, it further includes:

[0150] If it is determined that the global tool dependency graph cannot be executed, then calculate the gradient sensitivity of each node;

[0151] The current sensitivity threshold is determined based on the gradient sensitivity of each node;

[0152] If the gradient sensitivity of any node is greater than the current sensitivity threshold, the node is identified as a potential security risk node.

[0153] The global tool dependency graph is modified based on the nodes with security vulnerabilities, so that operations can be performed based on the modified global tool dependency graph.

[0154] The current sensitivity threshold refers to the judgment value dynamically calculated based on the gradient sensitivity distribution of all nodes in the current global tool dependency graph, and there is no fixed value.

[0155] Among them, the security vulnerability node refers to the tool node whose gradient sensitivity exceeds the current sensitivity threshold. This node is the core cause that makes the global tool dependency graph unexecutable.

[0156] Specifically, once the global tool dependency graph is determined to be unexecutable by a pre-defined graph neural network model, the gradient sensitivity calculation process is immediately initiated. Using the unexecutable probability value output by the model as the target value, partial derivatives are calculated on the feature vector of each tool node to obtain the rate of change of each dimension of the node's feature vector with respect to the unexecutable probability value. Then, norm calculation is used to convert the multidimensional rate of change into a single value, which is the gradient sensitivity G of the corresponding node. i Gradient sensitivity G i The calculation formula is as follows:

[0157] .

[0158] Furthermore, the mean μ and standard deviation σ of the gradient sensitivity of all nodes are statistically analyzed, and the threshold is calculated according to a preset formula. The threshold is... Where k is a hyperparameter, set within the range of 1.5 to 3. The gradient sensitivity of each tool node is compared one by one with the calculated current sensitivity threshold. If the gradient sensitivity value of a node is greater than the current sensitivity threshold, it indicates that the node is a critical abnormal node causing the global tool dependency graph to be unexecutable, and is directly identified as a security vulnerability node. If the gradient sensitivity of a node is less than or equal to the current sensitivity threshold, it indicates that the node has no significant negative impact on overall executability, and is identified as a normal node. After completing the comparison of all nodes, all security vulnerability nodes are summarized to form a list of vulnerability nodes, and the gradient sensitivity value of each node is marked.

[0159] Furthermore, if a node is detected as a security risk node, then the current node and the edges connected to it are deleted. If a node is a necessary node on the inference trajectory, deleting this node would truncate the inference trajectory, so the trajectory from this node to the end node is also deleted, and only the trajectory from the starting node to this risk node is retained.

[0160] The intelligent agent security vulnerability detection method provided in this application calculates the gradient sensitivity of each node, quantifies its impact on the execution result of the global tool dependency graph, reveals the key role and impact of changes in the tool during execution, and helps to accurately locate the root cause of non-executable problems. Security vulnerability nodes are weak points in the system prone to security issues; identification allows for early prevention. For example, nodes with high gradient sensitivity and involving sensitive operations are easily attacked, and timely handling can improve security. Furthermore, complex node dependencies mean that security vulnerability nodes can propagate security problems; correcting the dependency graph can block vulnerability propagation and ensure overall system security.

[0161] As an optional implementation, based on the above embodiments, it further includes:

[0162] In response to the completion of the operation, labels are added to the global tool dependency graph based on the execution result;

[0163] Replanning is performed based on the tagged global tool dependency graph to generate an updated global tool dependency graph;

[0164] Based on updating the global tool dependency graph, security checks and control execution operations are repeatedly performed until the task for the currently input text is completed.

[0165] Among them, the label refers to the stateful description identifier assigned to the global tool dependency graph based on the actual execution result of the tool call, including information such as the execution result type, node execution status, and link validity.

[0166] Replanning refers to the process of reorganizing and adjusting the nodes, execution order, and dependency relationships of the original dependency graph based on a labeled global tool dependency graph, combined with the problems reported from the execution results and the remaining requirements of the tasks.

[0167] The completion of the task for the current input text refers to the intelligent agent completing the entire process of tool invocation and operation execution according to the core requirements of the input text, and outputting a complete result that meets the user's needs.

[0168] Specifically, after the operation is completed, the execution result is analyzed in all dimensions, including the overall execution status (e.g., partial completion, single node execution success / failure, link interruption) and node execution details (e.g., whether the tool node returned a result normally, whether the parameter passing was valid, and whether the execution timed out). Based on the analysis results, multi-dimensional labels are added to the global tool dependency graph. These labels are directly associated with the corresponding tool node, dependency edge, and the topology of the entire graph.

[0169] Furthermore, based on the tagged global tool dependency graph and combined with the remaining task requirements of the current input text, a full-link replanning is carried out. First, the anomalies and incomplete steps marked in the tags are identified. Anomalies are addressed through node replacement, link adjustment, and rule optimization, such as replacing failed nodes with backup compliant nodes from the target MCP tool list. For incomplete steps, compliant tool nodes are matched from the target MCP tool list and added to the original dependency graph. The execution order and dependencies between new and existing nodes are then re-examined to improve the link structure and generate an updated global tool dependency graph adapted to the remaining task requirements.

[0170] Furthermore, the generated updated global tool dependency graph is re-input into the preset graph neural network model, and a full-dimensional verification is performed according to the original security detection process. This includes steps such as security detection graph construction, inference trajectory feature vector calculation, and preset detection category probability value determination to verify whether the updated dependency graph is executable. If the verification passes, the control agent initiates a new round of execution operations based on the updated global tool dependency graph. The execution process still follows the core rules of tool scheduling, real-time monitoring, and anomaly handling. If the verification fails, the updated global tool dependency graph is corrected according to the process of gradient sensitivity calculation, security vulnerability node identification, and dependency graph correction until the verification passes before execution.

[0171] The intelligent agent security vulnerability detection method provided in this application, based on the above embodiments, adds labels to the global tool dependency graph after the operation is completed. This records information such as the effect of tool combinations and problematic stages, accumulating execution experience for the dependency graph and providing a reference for subsequent replanning and execution. Replanning is performed based on the labeled dependency graph to generate an updated graph. The system can dynamically adjust the tool execution order and dependencies according to actual execution conditions to meet actual needs. If a particular tool is found to be highly efficient in a specific order, its position can be adjusted during replanning to optimize the process and reduce resource and time waste. Repeated security detection based on the updated graph allows for continuous monitoring of system security. New threats constantly emerge, and old vulnerabilities may become more prominent due to environmental changes; continuous detection can promptly identify and control new security issues. If security vulnerabilities are found in the updated graph, they can be corrected in a timely manner to prevent the problem from escalating and ensure stable system operation. Furthermore, the recorded data is a knowledge asset that can be used to analyze system performance and other issues, providing a basis for continuous improvement.

[0172] As an optional implementation, it also includes:

[0173] In response to the obtained output text, determine whether the output text contains preset sensitive words based on the preset sensitive series list;

[0174] If it does not exist, calculate the probability of semantic security risks in the output text;

[0175] If the probability of semantic security risks is less than or equal to a preset probability threshold, then the inconsistency value between the input text and the output text is calculated.

[0176] Whether the output text can be safely output is determined based on the probability of semantic security risks and inconsistency values.

[0177] The pre-defined sensitive word list refers to a set of sensitive words that are formulated in advance based on the application scenarios of intelligent agents, industry compliance requirements, and security protection needs.

[0178] Among them, the probability of semantic security risks refers to the probability that the output text has compliance risks or semantic security risks as determined by the semantic analysis model.

[0179] Among them, the preset probability threshold refers to the judgment threshold set for the probability of semantic security risks, which is a value pre-configured based on historical detection data and security protection level.

[0180] The inconsistency value refers to an indicator that quantifies the degree of deviation between the core message of the input text and the semantic content of the output text.

[0181] Specifically, after the operation is completed, the generated output text is obtained and preprocessed to remove redundant spaces and meaningless special characters. An efficient keyword matching algorithm is then used to perform a full-text scan of the preprocessed output text, comparing each word in the text with a pre-defined list of sensitive words. If no pre-defined sensitive words are detected, the process proceeds to the next step; if pre-defined sensitive words are detected, it indicates that the text poses an explicit risk, and a pre-defined safe response is returned directly.

[0182] Furthermore, if no preset sensitive words are detected, the probability of semantic security risks is calculated. A pre-trained semantic analysis model trained for output text security judgment is loaded. This model has learned the semantic features of massive amounts of compliant and non-compliant output texts and can accurately identify the overall semantic risks behind explicit sensitive words. Output text containing sensitive words is input into this model, and the model outputs the probability of semantic security risks corresponding to the output text based on the semantic feature matching results. The calculated probability of semantic security risks is compared with a preset probability threshold. If the probability is greater than the preset threshold, it means that although the output text does not contain explicit sensitive words, there are high semantic security risks, and it is judged as unsafe to output, triggering a security response. If the probability is less than or equal to the preset threshold, it means that the semantic risk of the output text is within a controllable range, and further input-output consistency verification is required to check for deviations such as semantic compliance but irrelevant answers.

[0183] Furthermore, a pre-trained semantic encoding model is loaded, and the original input text and the current output text are input into the model respectively. The model performs independent semantic encoding on the two texts, generating semantic vectors. The similarity value between the two semantic vectors is calculated using a cosine similarity algorithm. The final inconsistency value is obtained by subtracting the similarity value from 1. If the inconsistency value is greater than a preset inconsistency threshold, a safety response is triggered; otherwise, it is added to the semantic security risk probability. If the sum of the two is greater than a preset deviation threshold, a safety response is triggered; if the sum of the two is less than or equal to the preset deviation threshold, the output text is output safely.

[0184] Optionally, a context-sensitive intent verification mechanism can be introduced, dynamically adjusting the intent relevance threshold based on user input, historical interaction records, and tool call history. For example, when the user input is to summarize employee preferences, the output can be set to not contain financial operation-related words, such as transfer or payment; when such words are detected in the output, a security response is triggered even if the semantic similarity does not exceed the limit. A sequence-to-sequence model, such as T5, is used to generate a summary of the user input intent and semantically compare it with the output content, dynamically adjusting the verification strategy based on tool call history.

[0185] The intelligent agent security vulnerability detection method provided in this application, based on a preset sensitive series list, can quickly determine whether the output text contains preset sensitive words and identify sensitive information content. Upon discovery of sensitive words, timely interception or desensitization can be performed to prevent information leakage and ensure information security. Calculating the semantic security vulnerability probability of the output text allows analysis of potential semantic attacks. Some attacks do not directly use obvious sensitive words but instead use obscure semantics to convey malicious information and deceive; semantic analysis can detect such threats, allowing for early prevention and avoiding user harm. Calculating the inconsistency value between the input and output text allows assessment of whether the output text matches the intent of the input text. Large inconsistencies may indicate processing errors or deviations, leading to inaccurate content; this allows for timely detection and correction of problems, ensuring content quality. Furthermore, judging whether the output text can be safely output based on the semantic security vulnerability probability and inconsistency value provides feedback to the content generation model, helping it to adjust and optimize, improving the quality and reliability of the generated content.

[0186] Figure 3 This is a flowchart illustrating a method for detecting security risks in intelligent agents according to another embodiment of this application, as shown below. Figure 3 As shown, the intelligent agent security risk detection method provided in this embodiment includes the following steps:

[0187] S301. Determine the intent type of the input text based on its semantic features.

[0188] S302. Calculate the conditional probability value of each tag based on the word segmentation results of the input text using a preset language model.

[0189] S303. Determine the perplexity value of the input text based on the conditional probability value of each marker.

[0190] S304. Perform multiple rounds of translation on the input text according to the preset language chain order to obtain the target back-translated text.

[0191] S305. Calculate the semantic vector cosine similarity between the target back-translated text and the input text to determine the back-translation similarity.

[0192] S306. If the intent type, perplexity value, and back-translation similarity all meet the corresponding preset conditions, then the input text detection is deemed successful.

[0193] S307. Obtain metadata for at least one MCP tool to be used from the MCP server.

[0194] S308. Convert the text content of the metadata into a sequence of tags.

[0195] S309. Using a preset classification model, determine the security category of each MCP tool to be used based on the label sequence of each MCP tool to be used.

[0196] S310. If there is an MCP tool with a security category of the preset security category, delete the MCP tool to obtain a list of target MCP tools.

[0197] S311. Generate a global tool dependency graph based on the detected input text and the target MCP tool list.

[0198] S312. A security detection graph is constructed using a pre-defined graph neural network model with tools as nodes and dependencies between tools as edges.

[0199] S313. Perform average pooling on the feature vectors corresponding to each node to obtain the feature vectors corresponding to the inference trajectory.

[0200] S314. Calculate the probability value of each preset detection category based on the feature vector corresponding to the inference trajectory.

[0201] S315. Determine whether the global tool dependency graph can be executed based on the probability value.

[0202] S316. In response to successful verification, the control agent performs operations based on the global tool dependency graph.

[0203] S317. If it is determined that the global tool dependency graph cannot be executed, then calculate the gradient sensitivity of each node.

[0204] S318. Determine the current sensitivity threshold based on the gradient sensitivity of each node.

[0205] S319. If the gradient sensitivity of any node is greater than the current sensitivity threshold, the node is determined to be a node with potential security risks.

[0206] S320. Correct the global tool dependency graph based on the security vulnerability nodes so that the operation can be performed based on the corrected global tool dependency graph.

[0207] S321. Add labels to the global tool dependency graph based on the execution results.

[0208] S322. Replan based on the tagged global tool dependency graph to generate an updated global tool dependency graph, and execute S311-S315 until the task of the current input text is completed.

[0209] S323. In response to obtaining the output text, determine whether the output text contains preset sensitive words based on the preset sensitive series table.

[0210] S324. If it does not exist, calculate the probability of semantic security risks in the output text.

[0211] S325. If the probability of semantic security risks is less than or equal to the preset probability threshold, then calculate the inconsistency value between the input text and the output text.

[0212] S326. Determine whether the output text can be output safely based on the probability of semantic security risks and inconsistency values.

[0213] It should be noted that the execution order of S302-S303 and S304-S305 is not important, as is the execution order of S316 and S317-S320.

[0214] In this embodiment, the implementation method and technical effect of S301-S326 are similar to those of the corresponding solutions in the above embodiments, and will not be repeated here.

[0215] Figure 4 This is a schematic diagram of the intelligent agent security hazard detection device provided in this application, as shown below. Figure 4 As shown, the intelligent agent security hazard detection device 40 provided in this embodiment includes: a determination module 41, a generation module 42, a construction module 43, and a control module 44.

[0216] The system includes the following modules: Determination module 41, which determines the intent type, perplexity value, and back-translation similarity of the input text; determination module 41, which determines that the input text passes detection if the intent type, perplexity value, and back-translation similarity all meet the corresponding preset conditions; determination module 41, which uses a preset classification model to determine the security category of each MCP tool to be used based on the metadata of each MCP tool to be used, in order to obtain a list of target MCP tools; generation module 42, which generates a global tool dependency graph based on the detected input text and the list of target MCP tools; construction module 43, which uses a preset graph neural network model to construct a security detection graph based on the global tool dependency graph, in order to verify whether the global tool dependency graph can be executed; and control module 44, which controls the agent to perform operations based on the global tool dependency graph in response to successful verification.

[0217] The intelligent agent security risk detection device provided in this embodiment can perform... Figure 2 The implementation principles and technical effects of the methods shown are similar, and will not be repeated here.

[0218] Optionally, the determining module 41, when determining the intent type, perplexity value, and back-translation similarity of the input text, is specifically used for: determining the intent type of the input text based on the semantic features of the input text; calculating the conditional probability value of each tag based on the word segmentation results of the input text using a preset language model; determining the perplexity value of the input text based on the conditional probability value of each tag; performing multiple rounds of translation on the input text according to the preset language chain order to obtain the target back-translated text; and calculating the semantic vector cosine similarity between the target back-translated text and the input text to determine the back-translation similarity.

[0219] Optionally, the determining module 41, when determining the security category of each MCP tool to be used based on the metadata of each MCP tool to be used using a preset classification model to obtain a list of target MCP tools, specifically performs the following: obtaining the metadata of at least one MCP tool to be used from the MCP server; converting the text content of the metadata into a tag sequence; determining the security category of each MCP tool to be used based on the tag sequence of each MCP tool to be used using a preset classification model; and deleting any MCP tool whose security category is a preset security category to obtain a list of target MCP tools.

[0220] Optionally, the global tool dependency graph includes tools, execution order, and dependencies; the construction module 43, when using a preset graph neural network model to construct a security detection graph based on the global tool dependency graph to verify whether the global tool dependency graph can be executed, is specifically used for: constructing a security detection graph using a preset graph neural network model with tools as nodes and dependencies between tools as edges; performing average pooling on the feature vectors corresponding to each node to obtain the feature vectors corresponding to the inference trajectory; calculating the probability value of each preset detection category based on the feature vectors corresponding to the inference trajectory; and determining whether the global tool dependency graph can be executed based on the probability value.

[0221] Optionally, the intelligent agent security hazard detection device provided in this embodiment also includes a calculation module and a correction module.

[0222] Correspondingly, the calculation module is used to calculate the gradient sensitivity of each node if it is determined that the global tool dependency graph cannot be executed; the determination module 41 is also used to determine the current sensitivity threshold based on the gradient sensitivity of each node; in response to the gradient sensitivity of any node being greater than the current sensitivity threshold, the node is determined to be a security vulnerability node; the correction module is used to correct the global tool dependency graph according to the security vulnerability node, so that the execution operation can be performed based on the corrected global tool dependency graph.

[0223] Optionally, the intelligent agent security risk detection device provided in this embodiment also includes an adding module and an execution module.

[0224] Accordingly, the addition module is used to add labels to the global tool dependency graph based on the execution result after the execution operation is completed; the generation module 42 is also used to replan based on the tagged global tool dependency graph to generate an updated global tool dependency graph; the execution module is used to repeatedly execute the security detection and control execution operation based on the updated global tool dependency graph until the task of the current input text is completed.

[0225] Optionally, the determining module 41 is further configured to, in response to obtaining the output text, determine whether the output text contains preset sensitive words according to the preset sensitive series table; the calculation module is further configured to, if not, calculate the semantic security risk probability of the output text; if the semantic security risk probability is less than or equal to the preset probability threshold, calculate the inconsistency value between the input text and the output text; the determining module 41 is further configured to, based on the semantic security risk probability and the inconsistency value, determine whether the output text can be output safely.

[0226] Figure 5 This is a structural schematic diagram of the intelligent agent security hazard detection device provided in this application. Figure 5 As shown, the intelligent agent security risk detection device 50 provided in this embodiment includes a processor 51 and a memory 52. ​​The processor 51 and the memory 52 are connected via a bus and communicate with each other.

[0227] In the specific implementation process, the processor 51 executes the computer execution instructions stored in the memory 52, causing the processor 51 to perform the above-described method.

[0228] The specific implementation process of processor 51 can be found in the above method embodiments, and its implementation principle and technical effect are similar. It will not be repeated here.

[0229] In the above embodiments, it should be understood that the processor 51 can be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), etc. The general-purpose processor can be a microprocessor or any conventional processor. The steps of the method disclosed in this invention can be directly implemented by a hardware processor, or implemented by a combination of hardware and software modules within the processor.

[0230] The memory 52 may include random access memory (RAM) and may also include non-volatile memory (NVM), such as at least one disk storage device.

[0231] The bus can be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus, or an Extended Industry Standard Architecture (EISA) bus, etc. Buses can be categorized as address buses, data buses, control buses, etc. For ease of illustration, the buses shown in the accompanying drawings are not limited to a single bus or a single type of bus.

[0232] This application also provides a computer program product, including a computer program that, when executed by a processor, implements the above-described method.

[0233] This application also provides a computer-readable storage medium storing computer-executable instructions, which, when executed by a processor, implement the above-described method.

[0234] The aforementioned readable storage medium can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic storage, flash memory, magnetic disk, or optical disk. The readable storage medium can be any available medium accessible to a general-purpose or special-purpose computer.

[0235] An exemplary readable storage medium is coupled to a processor, enabling the processor to read information from and write information to the readable storage medium. Of course, the readable storage medium can also be a component of the processor. The processor and the readable storage medium can reside in an Application Specific Integrated Circuit (ASIC). Alternatively, the processor and the readable storage medium can exist as discrete components in the device.

[0236] The division of units is merely a logical functional division; in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be indirect coupling or communication connection through some interfaces, devices, or units, and may be electrical, mechanical, or other forms.

[0237] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0238] In addition, the functional units in the various embodiments of the present invention can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit.

[0239] If a function is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this invention, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of the various embodiments of this invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0240] Those skilled in the art will understand that all or part of the steps of the above-described method embodiments can be implemented by hardware related to program instructions. The aforementioned program can be stored in a computer-readable storage medium. When executed, the program performs the steps of the above-described method embodiments; and the aforementioned storage medium includes various media capable of storing program code, such as ROM, RAM, magnetic disks, or optical disks.

[0241] Finally, it should be noted that other embodiments of the invention will readily occur to those skilled in the art upon consideration of the specification and practice of the invention disclosed herein. This invention is intended to cover any variations, uses, or adaptations of the invention that follow the general principles of the invention and include common knowledge or customary techniques in the art not disclosed herein, and is not limited to the precise structures described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from its scope. The scope of the invention is limited only by the appended claims.

Claims

1. A method for detecting security risks in intelligent agents, characterized in that, The method includes: Determine the intent type, perplexity value, and back-translation similarity of the input text; If the intent type, perplexity value, and back-translation similarity all meet the corresponding preset conditions, then the input text detection is determined to be successful. A pre-defined classification model is used to determine the security category of each MCP tool to be used based on the metadata of each MCP tool to be used, so as to obtain a list of target MCP tools; A global tool dependency graph is generated based on the detected input text and the target MCP tool list; A preset graph neural network model is used to construct a security detection graph based on the global tool dependency graph to verify whether the global tool dependency graph can be executed; Upon successful verification, the control agent performs operations based on the global tool dependency graph.

2. The method according to claim 1, characterized in that, The determination of the intent type, perplexity value, and back-translation similarity of the input text includes: The intent type of the input text is determined based on its semantic features; The conditional probability value of each tag is calculated based on the word segmentation results of the input text using a preset language model; The perplexity value of the input text is determined based on the conditional probability value of each marker; The input text is translated multiple times according to a preset language chain order to obtain the target back-translated text; Calculate the semantic vector cosine similarity between the target back-translated text and the input text to determine the back-translation similarity.

3. The method according to claim 1, characterized in that, The method employs a pre-defined classification model to determine the security category of each MCP tool to be used based on its metadata, thereby obtaining a list of target MCP tools, including: Obtain metadata for at least one MCP tool to be used from the MCP server; Convert the text content of the metadata into a sequence of tags; The security category of each MCP tool to be used is determined by using a pre-defined classification model based on the label sequence of each MCP tool to be used. If an MCP tool with the default security category exists, delete that MCP tool to obtain a list of target MCP tools.

4. The method according to claim 1, characterized in that, The global tool dependency graph includes tools, execution order, and dependencies; the step of constructing a security detection graph based on the global tool dependency graph using a preset graph neural network model to verify whether the global tool dependency graph can be executed includes: A pre-defined graph neural network model is used to construct a security detection graph with tools as nodes and dependencies between tools as edges. Average pooling is performed on the feature vectors corresponding to each node to obtain the feature vectors corresponding to the inference trajectory; The probability value of each preset detection category is calculated based on the feature vector corresponding to the inference trajectory; The global tool dependency graph is determined to be executable based on the probability value.

5. The method according to claim 4, characterized in that, Also includes: If it is determined that the global tool dependency graph cannot be executed, then the gradient sensitivity of each node is calculated; The current sensitivity threshold is determined based on the gradient sensitivity of each node; If the gradient sensitivity of any node is greater than the current sensitivity threshold, the node is determined to be a potential security risk node. The global tool dependency graph is modified based on the security vulnerability nodes so that operations can be performed based on the modified global tool dependency graph.

6. The method according to claim 5, characterized in that, Also includes: In response to the completion of the operation, labels are added to the global tool dependency graph based on the execution result; Replanning is performed based on the tagged global tool dependency graph to generate an updated global tool dependency graph; Based on the updated global tool dependency graph, security checks and control execution operations are repeatedly performed until the task for the currently input text is completed.

7. The method according to claim 1, characterized in that, Also includes: In response to obtaining the output text, determine whether the output text contains preset sensitive words according to the preset sensitive series table; If it does not exist, calculate the probability of semantic security risks in the output text; If the probability of the semantic security risk is less than or equal to a preset probability threshold, then the inconsistency value between the input text and the output text is calculated. Based on the semantic security risk probability and inconsistency value, it is determined whether the output text can be output safely.

8. A device for detecting potential safety hazards in intelligent agents, characterized in that, include: The determination module is used to determine the intent type, perplexity value, and back-translation similarity of the input text; The determining module is further configured to determine that the input text detection has passed if the intent type, perplexity value, and back-translation similarity all meet the corresponding preset conditions. The determining module is further configured to use a preset classification model to determine the security category of each MCP tool to be used based on the metadata of each MCP tool to be used, so as to obtain a list of target MCP tools; The generation module is used to generate a global tool dependency graph based on the detected input text and the target MCP tool list; A construction module is used to construct a security detection graph based on the global tool dependency graph using a preset graph neural network model, in order to verify whether the global tool dependency graph can be executed; The control module is used to control the agent to perform operations based on the global tool dependency graph in response to successful verification.

9. A smart agent security hazard detection device, characterized in that, include: Memory, processor; The memory stores computer-executed instructions; The processor executes computer execution instructions stored in the memory, causing the processor to perform the method as described in any one of claims 1-7.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer-executable instructions, which, when executed by a processor, are used to implement the method as described in any one of claims 1-7.

11. A computer program product, characterized in that, Includes a computer program that, when executed by a processor, implements the method according to any one of claims 1-7.

Images