Static permission policy automatic generation method for LLM intelligent agent tool invocation

Abstract

This invention discloses an automated method and apparatus for generating static permission policies for LLM (Limited Management Model) agent tool calls, relating to the fields of computer security and artificial intelligence. The method includes: acquiring target tool definition information and inputting it into a tool description receiving module to complete the basic definition of abstract security areas and sensitive data metadata; generating static risk scenarios through a static risk scenario generation module; inputting the static risk scenarios into a static policy generation module to convert them into directly executed single-step denial policies; for complex risks involving multiple tools working together, taking a specific target tool as the final destination, generating multi-step tool call hazard trajectories through a multi-step hazard trajectory generation module; inputting the tool call hazard trajectories into a multi-step policy reinforcement module to generate reinforcement policies and construct execution rules; and performing real-time review through a runtime rule execution module to complete the tool call. Using this invention can improve the protection capabilities of sensitive data in agent systems.

Description

Technical Field

[0001] This invention relates to the fields of computer security and artificial intelligence technology, and in particular to a method and apparatus for automatically generating static permission policies for LLM agent tool calls. Background Technology

[0002] In recent years, large language models have made rapid progress in natural language understanding, task planning, content generation, and code understanding. Building on this foundation, an increasing number of systems are beginning to construct agents with tool-invoking capabilities. This allows large language models to move beyond text-based question answering and accomplish more complex and executive-oriented tasks by invoking external tools. For example, an agent can invoke file reading and writing tools to read local documents, invoke command execution tools to execute system commands, invoke email tools to send messages, invoke search tools to retrieve external information, and invoke cameras or scanning tools to extract information from documents and forms.

[0003] The characteristics of the constructed intelligent agent system include: (1) Multi-tool collaboration capability: The same task often requires multiple tools to work together continuously. For example, first read the file, then summarize the content, then write a new file, and finally output it through email, upload or external service; (2) Complex execution chain: Tool calls are not isolated, but have a dependency relationship. The output of the previous tool usually becomes the input of the next tool, thus forming a complete task execution process; (3) Continuously expanding tool capabilities: With the development of intelligent office, terminal control, business process automation, government and enterprise digital assistants, the types of tools that intelligent agents can access are increasing, involving multiple fields such as file system, terminal command, network service, document processing, message communication, image recognition, audio transcription and personal assistant. As tool capabilities increase, the security risks of intelligent agent systems have significantly risen, especially in scenarios involving file system access, command execution, data writing, data transmission, and external service calls. Without effective access control, the following risks may arise: reading, modifying, or deleting sensitive files; outputting sensitive data such as ID cards, transcripts, tax records, and health information to insecure locations; creating scripts, configuration files, or command files in high-privilege areas; bypassing single-step restrictions through combinations of multiple tools, leading to indirect data leaks or privilege breaches; and causing destructive operations or system state tampering through command execution tools. Therefore, establishing refined, automatically generated, and runtime-executable access control schemes for tool calls by intelligent agents based on large language models has become a key technical issue in the security of intelligent agent systems.

[0004] Current technologies propose a programmable permission control mechanism for agents based on large language models. Its core is to express tool call permission policies using a domain-specific language and monitor, allow, or block each tool call during agent runtime. AgentSpec also uses a policy language to describe security constraints and applies them to runtime constraint enforcement. This approach emphasizes using structured rules to impose runtime restrictions on agents based on large language models. These rules include triggering conditions, predicates, and execution mechanisms, covering different scenarios such as code execution, embodied agents, and autonomous driving, while emphasizing rule interpretability and runtime efficiency. These permission policy technologies focus on how to express rules and how to execute them at runtime. However, as the number of tools increases, parameter structures diversify, and tools continue to expand, high human maintenance costs remain. Furthermore, they lack consideration for situations involving sensitive data during tool calls and preventative measures against potential dangers associated with multi-step tool calls. Summary of the Invention

[0005] To address the shortcomings of existing technologies, such as the reliance on manual coding for permission policies in LLM agent tool calls, which makes automatic generation difficult as tool types and parameter formats expand, and the inability of existing single-step static review mechanisms to identify complex security risks arising from sensitive data propagation, out-of-bounds access, data leakage, and indirect dangerous operations in multi-tool collaboration, and the instability, uninterpretability, and difficulty in auditing results when relying solely on large models for security judgments during runtime, this invention provides a method and apparatus for automatically generating static permission policies for LLM agent tool calls. The technical solution is as follows: On the one hand, a method for automatically generating static permission policies for LLM agent tool calls is provided. This method is implemented by a static permission policy automatic generation device for LLM agent tool calls, and includes: S1. Construct an automated generation framework for tool static permission policies for LLM intelligent agent systems; wherein, the framework includes: a tool description receiving module, a static risk scenario generation module, a static policy generation module, a multi-step danger trajectory generation module, a multi-step policy reinforcement module, and a runtime rule execution module; S2. Obtain the target tool definition information and input it into the tool description receiving module. Combine the predefined abstract security area system and sensitive data metadata system to perform security semantic mapping on the parameters, input objects and output objects of the target tool. Input the target tool definition information into the static risk scenario generation module to generate static risk scenarios. S3. Input the static risk scenario into the static policy generation module to convert the static risk scenario into a single-step rejection policy that can be directly executed by the rule engine; S4. For complex risks involving multiple tools, with a specific target tool as the final destination, the target tool definition information, system tool definition set, static risk scenarios, and predefined abstract security area system and sensitive data metadata system are input into the multi-step danger trajectory generation module to automatically generate the danger trajectory of multi-step tool calls. S5. Input the dangerous trajectory and target tool definition information of the tool call into the multi-step strategy reinforcement module, generate a reinforcement strategy based on the single-step rejection strategy, and construct the execution rules; S6. Input the execution rules into the runtime rule execution module for real-time review by the rule engine and complete the tool call.

[0006] On the other hand, an automated static permission policy generation apparatus for LLM agent tool calls is provided. This apparatus is applied to the automated static permission policy generation method for LLM agent tool calls. The apparatus includes: The tool description receiving module is used to obtain the target tool definition information and input it into the tool description receiving module. Combined with the predefined abstract security area system and sensitive data metadata system, it performs security semantic mapping on the parameters, input objects and output objects of the target tool. The static risk scenario generation module is used to input the target tool definition information into the static risk scenario generation module to generate static risk scenarios; The static policy generation module is used to input static risk scenarios into the static policy generation module and convert the static risk scenarios into single-step rejection policies that can be directly executed by the rule engine. The multi-step hazard trajectory generation module is used to address complex risks involving multiple tools working together. It takes a specific target tool as the final destination and inputs the target tool definition information, system tool definition set, static risk scenario, and predefined abstract security area system and sensitive data metadata system into the multi-step hazard trajectory generation module to automatically generate hazard trajectories for multi-step tool calls. The multi-step strategy reinforcement module is used to input the dangerous trajectory of the tool call and the target tool definition information into the multi-step strategy reinforcement module, generate reinforcement strategies based on the single-step rejection strategy, and construct execution rules; The runtime rule execution module is used to input execution rules into the runtime rule execution module for real-time review by the rule engine and to complete tool calls.

[0007] On the other hand, a static permission policy automatic generation device for LLM agent tool calls is provided. The static permission policy automatic generation device for LLM agent tool calls includes: a processor; a memory, wherein the memory stores computer-readable instructions, and when the computer-readable instructions are executed by the processor, any one of the methods described above for the automatic generation of static permission policies for LLM agent tool calls is implemented.

[0008] On the other hand, a computer-readable storage medium is provided, wherein at least one instruction is stored therein, the at least one instruction being loaded and executed by a processor to implement any of the methods in the above-described static permission policy automatic generation method for LLM agent tool calls.

[0009] The beneficial effects of the technical solutions provided in the embodiments of the present invention include at least the following: The embodiments of the present invention can identify and reinforce the risks of multi-tool invocation that cannot be covered by single-step strategies, and improve the protection against risks such as compound attacks, indirect leakage and process bypass. Under the premise of ensuring auditability, reproducibility and scalability, the embodiments of the present invention improve the permission security of intelligent agent systems in scenarios such as sensitive data processing, file system operations, command execution and data outgoing.

[0010] The embodiments of the present invention can automatically generate static policies and reinforcement policies based on tool definitions, parameter descriptions and risk scenarios, reducing the workload of manually writing permission rules one by one and improving development and maintenance efficiency.

[0011] Existing single-step review methods struggle to identify attack chains where each step is legitimate but the combination is dangerous. This invention, through multi-step dangerous trajectory generation and multi-step strategy reinforcement, can identify and reinforce implicit risks in multi-tool call chains, thereby significantly improving security coverage.

[0012] This invention introduces abstract security areas and sensitive data metadata, which can uniformly represent and control the propagation of various types of sensitive data such as ID cards, contact information, educational background, financial information, or health information, thereby effectively improving the protection capability of sensitive data in intelligent agent systems.

[0013] The strategies generated in this embodiment of the invention are ultimately implemented as deterministic rules that can be executed by the rule engine. There is no need to call the large model again for security determination at runtime. It has the advantages of stable results, reproducibility, auditability and interpretability.

[0014] By normalizing and mapping real paths to abstract security regions, this invention avoids strong binding of rules to specific device and operating system paths, enabling policies to have cross-environment migration capabilities and higher reusability. Attached Figure Description

[0015] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0016] Figure 1 This is a flowchart of a method for automatically generating static permission policies for LLM agent tool calls, provided by an embodiment of the present invention. Figure 2 This is a schematic diagram of the structure for automatically generating static permission policies for LLM agent tool calls provided by an embodiment of the present invention; Figure 3 This is a block diagram of an automated static permission policy generation device for LLM agent tool calls provided in an embodiment of the present invention; Figure 4 This is a schematic diagram of the structure of an automated generation device for static permission policies for LLM agent tool calls provided in an embodiment of the present invention. Detailed Implementation

[0017] The technical solution of the present invention will now be described with reference to the accompanying drawings.

[0018] In embodiments of the present invention, words such as "exemplarily," "for example," etc., are used to indicate that something is an example, illustration, or description. Any embodiment or design described as "exemplary" in the present invention should not be construed as being more preferred or advantageous than other embodiments or designs. Specifically, the use of the word "exemplary" is intended to present the concept in a concrete manner. Furthermore, in embodiments of the present invention, the meaning expressed by "and / or" can be both, or either one.

[0019] In the embodiments of this invention, the terms "image" and "picture" may sometimes be used interchangeably. It should be noted that, without emphasizing the distinction between them, they convey the same meaning. Similarly, the terms "of," "corresponding (relevant)," and "corresponding" may sometimes be used interchangeably. It should be noted that, without emphasizing the distinction between them, they convey the same meaning.

[0020] In this embodiment of the invention, sometimes a subscript such as W1 may be written in a non-subscript form such as W1. When the difference is not emphasized, the meaning they express is the same.

[0021] To make the technical problems, technical solutions and advantages of the present invention clearer, a detailed description will be given below in conjunction with the accompanying drawings and specific embodiments.

[0022] This invention provides an automated method for generating static permission policies for LLM agent tool calls. This method can be implemented by a device for automatically generating static permission policies for LLM agent tool calls, which can be a terminal or a server. Figure 1 The flowchart shown is for an automated generation method of static permission policies for LLM agent tool calls. The processing flow of this method may include the following steps: S1. Construct an automated generation framework for tool static permission policies for LLM intelligent agent systems; the framework includes: a tool description receiving module, a static risk scenario generation module, a static policy generation module, a multi-step danger trajectory generation module, a multi-step policy reinforcement module, and a runtime rule execution module.

[0023] Optionally, the tool description receiving module is used to receive the tool name, tool function description, parameter description, and parameter type; Among them, the static risk scenario generation module is used to automatically generate static risk scenarios for the tool based on the tool definition information; Among them, the static policy generation module is used to convert static risk scenarios into single-step rejection policies that can be directly executed by the rule engine; Among them, the multi-step danger trajectory generation module is used to generate multi-step call trajectories for target tools and identify potential risks caused by the combination of multiple tools. Among them, the multi-step strategy reinforcement module is used to generate new strategy rules for the target tool based on multi-step dangerous trajectories, in order to supplement the risks that static strategies cannot cover; The runtime rule execution module is used to review the current call and the necessary context before each actual tool call by the agent, and determine whether to allow or deny it.

[0024] S2. Obtain the target tool definition information and input it into the tool description receiving module. Combine the predefined abstract security area system and sensitive data metadata system to perform security semantic mapping on the parameters, input objects and output objects of the target tool. Input the target tool definition information into the static risk scenario generation module to generate a static risk scenario.

[0025] To avoid the strategy directly depending on the specific operating system path, this embodiment of the invention introduces an abstract security region to uniformly represent the location of data with different sensitivity levels; wherein the security region may include, but is not limited to: CONFIDENTIAL_ZONE, SYSTEM_STATE_ZONE, TEMPORARY_ZONE, The zones are SECURE_ZONE, PRIVATE_ZONE, USER_HOME_ZONE, DOCUMENTS_ZONE, DOWNLOAD_ZONE, PUBLIC_ZONE, and ROOT_LEVEL_ZONE. During runtime, a path normalization module maps the actual paths to these zones. For example, directories containing sensitive user documents such as certificates, transcripts, and tax materials can be mapped to DOCUMENTS_ZONE / or CONFIDENTIAL_ZONE / .

[0026] Among these features, designing an abstract security region enables the rule generation process to be independent of specific device environments, improves rule reusability, and ensures deterministic rule matching during runtime.

[0027] In one feasible implementation, sensitive data metadata refers to data objects such as files, text, images, documents, forms, or records. This embodiment of the invention introduces lightweight metadata to represent their sensitivity. The metadata includes: contains_sensitive_data: true / false; data_risk: [{data_type, risk_level}]; Here, `data_type` can represent: ID number, contact information, address information, and financial information, etc.; `risk_level` can represent: low, medium, high, and critical. This metadata can be inherited from file attributes, known data types, input sources, or upstream tool results.

[0028] In particular, by introducing lightweight metadata, the system can control the flow of sensitive data without relying on complex semantic understanding, providing a basis for multi-step risk identification and strategy reinforcement.

[0029] Optionally, the target tool definition information includes: tool name, tool function description, parameter list, parameter name, parameter type, and parameter description.

[0030] Optionally, static risk scenarios refer to risk scenarios identified solely based on the tool's semantics, parameter types, parameter value patterns, sensitivity of the processed object, and target area. In step S2, several static risk scenarios are automatically generated for a single tool.

[0031] The static risk scenarios include: scenario number, scenario title, severity level, risk cause, example parameters, and detection approach.

[0032] Among them, such as Figure 2The diagram illustrates a structure for the automated generation of static permission policies for LLM agent tool calls, as provided in an embodiment of the present invention. This embodiment utilizes a mechanism for automatically generating static risk scenarios based on tool definitions. By automatically generating static risk scenarios that match the semantics of the tool according to its name, function description, and parameter specifications, rather than relying on manual definition. Without this mechanism, automated policy generation for different tools would be impossible.

[0033] S3. Input the static risk scenario into the static policy generation module to convert the static risk scenario into a single-step rejection policy that can be directly executed by the rule engine.

[0034] Specifically, the risk semantics, example parameters, and detection ideas in static risk scenarios are transformed into a single-step rejection policy that can be directly executed by the rule engine through field mapping, parameter pattern generalization, and rule structuring.

[0035] Each single-step denial policy includes: id, single_step = true, description, severity, when, all, any, action = deny, and generalization_reason.

[0036] The `when` condition consists of several condition objects, each of which contains at least: field, op, and value.

[0037] In one feasible implementation, the static policy generation module translates high-level risk scenarios into deterministic rules and forms a basic permission policy library that can be directly used for runtime review.

[0038] S4. For complex risks involving multiple tools, with a specific target tool as the final destination, the target tool definition information, system tool definition set, static risk scenarios, and predefined abstract security area system and sensitive data metadata system are input into the multi-step danger trajectory generation module to automatically generate a danger trajectory for multi-step tool calls.

[0039] Optionally, the constraints followed by the generation of dangerous trajectories for multi-step tool calls include: In a multi-step hazardous trajectory, each tool invocation step preceding the target tool invocation must not violate existing static policies; Risk must arise from a combination of multiple steps, rather than from a single step being illegal in itself; The final step must be the invocation of the target tool; The trajectory must reflect the propagation process of sensitive data within the toolchain; The output of each step in the trajectory can carry or inherit sensitive metadata.

[0040] In the implementation with complete metadata, the step of generating dangerous trajectories for multi-step tool calls is mainly used for offline analysis and policy enhancement, rather than a necessary step at runtime. In the implementation with incomplete metadata, the step of generating dangerous trajectories for multi-step tool calls can be used to discover "process-related risks" that cannot be covered by static rules.

[0041] S5. Input the dangerous trajectory and target tool definition information of the tool call into the multi-step strategy reinforcement module, generate reinforcement strategy based on single-step rejection strategy, and construct execution rules.

[0042] Optionally, the rules to be executed include: generalizable single-step rules and multi-step association rules; Among them, generalizable single-step rules refer to rules that can complete the matching judgment based solely on the parameter pattern of the current tool call; they do not depend on historical call records or context, but only on the parameters of the current tool call itself. Among them, generalizable single-step rules may include: prohibiting the writing of files containing educational background or financial information to PUBLIC_ZONE / ; prohibiting the execution of script files from PUBLIC_ZONE / ; and prohibiting the writing of sensitive data to the temporary area TEMPORARY_ZONE / .

[0043] If the risk only applies if the preceding toolchain exists, then a multi-step rule is generated that includes: require_history_tools, derived_from_scenarios, when, and action=deny.

[0044] Among them, multi-step association rules are rules that combine the current tool call parameters with the historical tool call sequence to complete the matching judgment, and are used to identify risks formed by the combination of multiple legitimate steps.

[0045] In this approach, multi-step rules only apply to the final target tool in the trajectory, without modifying the legitimate tool access permissions of preceding steps. This avoids accidentally affecting intermediate steps and maintains system availability.

[0046] In one feasible implementation, the multi-step strategy reinforcement module can automatically compensate for the problem of insufficient coverage of multi-step combination risks by static strategies; and improve the ability of the intelligent agent system to identify process-oriented attacks, indirect leakage and combination evasion behaviors.

[0047] S6. Input the execution rules into the runtime rule execution module for real-time review by the rule engine and complete the tool call.

[0048] Optionally, the specific implementation process of S6 includes S61-S65: S61. Obtain the name and specific parameters of the tool currently being called; S62. Based on the tool name and specific parameters of the current tool call, the path normalization module maps the real path to the defined abstract security area to achieve cross-environment rule matching. S63. Execute generalizable single-step rules based on tool parameters, abstract security areas, and sensitive data metadata; S64. If a multi-step rule exists, combine the agent's historical tool call records and execute the multi-step matching strategy; if any rejection policy is hit, the current tool call is directly blocked; if no rejection policy is hit, the tool call is allowed. S65. After the tool call is completed, output the results of the currently completed tool call or inherit the corresponding sensitive data metadata; the output results are used as the basis for the review of the next tool call; the output results refer to the execution result data returned after the current tool call is completed, including: text, file path, structured object, command output and resource creation results.

[0049] Specifically, during runtime, the engine preferentially avoids relying on large models and instead performs deterministic judgments based on a rules engine. This ensures stable execution results, an interpretable review process, and reproducibility through log auditing.

[0050] In one feasible implementation, the embodiments of the present invention are applied to an enterprise management scenario, where intelligent agents are commonly used to process employee information, contracts, resumes, expense reports, financial materials, scholarship application materials, academic certificates, tax information, and health certificates. This type of data typically includes ID numbers, contact information, educational background, work history, financial information, tax information, or health information, and is highly sensitive. In this specific embodiment, the intelligent agent system is a system composed of enterprise office intelligent assistants.

[0051] In one feasible implementation method, the specific implementation process includes: 1. Users input task requests into the enterprise office intelligent assistant, for example: "Please extract a summary of the candidate's academic background from the resume, academic certificates, and transcripts uploaded by the employee, and save it to the specified directory." 2. Large-scale intelligent agents invoke relevant tools based on task planning, such as: read_file: Reads resumes, academic certificates, and transcripts; OCR or image parsing tools: Extract text from JPG / PNG / PDF scans; phase_summary: Organizes educational background and academic performance; new_file: Creates a new profile file; write_file: Writes the generated results to a file; Optional send_email or external submission tool: send to the recruitment system or approval system.

[0052] 3. Before each tool call, the runtime rule execution module checks the call parameters using the single-step rejection policy generated according to this embodiment of the invention, for example: Reading highly sensitive historical archive files located in CONFIDENTIAL_ZONE / is prohibited; Data containing sensitive educational background information must not be written to PUBLIC_ZONE / or TEMPORARY_ZONE / . It is prohibited to process sensitive certificate files through command execution tools; It is prohibited to send results containing sensitive metadata directly to external email addresses.

[0053] 4. The tool's execution results are appended to or inherited with metadata upon return, for example: contains_sensitive_data=true data_risk=[{data_type: education background, risk_level: high}] 5. When subsequent tools such as write_file, send_email, and upload_file are invoked, the runtime rule execution module combines sensitive data metadata with existing static policies to conduct a review, thereby preventing the leakage of sensitive employee information.

[0054] In this scenario, the embodiments of the present invention can automatically identify and restrict the reading, writing, and external transmission of sensitive personnel information; prevent the leakage of information such as resumes, academic qualifications, and transcripts due to toolchain calls by internal enterprise intelligent assistants; and reduce the cost of manually configuring permission policies for each tool.

[0055] The embodiments of the present invention can identify and reinforce the risks of multi-tool invocation that cannot be covered by single-step strategies, and improve the protection against risks such as compound attacks, indirect leakage and process bypass. Under the premise of ensuring auditability, reproducibility and scalability, the embodiments of the present invention improve the permission security of intelligent agent systems in scenarios such as sensitive data processing, file system operations, command execution and data outgoing.

[0056] The embodiments of the present invention can automatically generate static policies and reinforcement policies based on tool definitions, parameter descriptions and risk scenarios, reducing the workload of manually writing permission rules one by one and improving development and maintenance efficiency.

[0057] Existing single-step review methods struggle to identify attack chains where each step is legitimate but the combination is dangerous. This invention, through multi-step dangerous trajectory generation and multi-step strategy reinforcement, can identify and reinforce implicit risks in multi-tool call chains, thereby significantly improving security coverage.

[0058] This invention introduces abstract security areas and sensitive data metadata, which can uniformly represent and control the propagation of various types of sensitive data such as ID cards, contact information, educational background, financial information, or health information, thereby effectively improving the protection capability of sensitive data in intelligent agent systems.

[0059] The strategies generated in this embodiment of the invention are ultimately implemented as deterministic rules that can be executed by the rule engine. There is no need to call the large model again for security determination at runtime. It has the advantages of stable results, reproducibility, auditability and interpretability.

[0060] By normalizing and mapping real paths to abstract security regions, this invention avoids strong binding of rules to specific device and operating system paths, enabling policies to have cross-environment migration capabilities and higher reusability.

[0061] Figure 3 This is a block diagram of an automated static permission policy generation device for LLM agent tool calls, provided by an embodiment of the present invention. This device is used in a method for automatically generating static permission policies for LLM agent tool calls. (Refer to...) Figure 3 The device includes a tool description receiving module 310, a static risk scenario generation module 320, a static policy generation module 330, a multi-step hazard trajectory generation module 340, a multi-step policy reinforcement module 350, and a runtime rule execution module 360. Among them: The tool description receiving module 310 is used to obtain the target tool definition information and input it into the tool description receiving module. Combined with the predefined abstract security area system and sensitive data metadata system, it performs security semantic mapping on the parameters, input objects and output objects of the target tool. The static risk scenario generation module 320 is used to input the target tool definition information into the static risk scenario generation module to generate a static risk scenario; The static policy generation module 330 is used to input static risk scenarios into the static policy generation module and convert the static risk scenarios into single-step rejection policies that can be directly executed by the rule engine. The multi-step hazard trajectory generation module 340 is used to address complex risks involving multiple tools working together. Taking a specific target tool as the final destination, the module automatically generates hazard trajectories for multi-step tool calls by inputting the target tool definition information, system tool definition set, static risk scenarios, and predefined abstract security area system and sensitive data metadata system into the multi-step hazard trajectory generation module. The multi-step strategy reinforcement module 350 is used to input the dangerous trajectory of the tool call and the target tool definition information into the multi-step strategy reinforcement module, generate a reinforcement strategy based on the single-step rejection strategy, and construct execution rules. The runtime rule execution module 360 ​​is used to input execution rules into the runtime rule execution module for real-time review by the rule engine and to complete tool calls.

[0062] Optionally, the runtime rule execution module is used to perform rule review on the current call and the necessary context before each actual call of the tool by the agent, and determine whether to allow or deny it; The tool description receiving module is used to receive the tool name, tool function description, parameter description, and parameter type. Among them, the static risk scenario generation module is used to automatically generate static risk scenarios for the tool based on the tool definition information; Among them, the static policy generation module is used to convert static risk scenarios into single-step rejection policies that can be directly executed by the rule engine; Among them, the multi-step danger trajectory generation module is used to generate multi-step call trajectories for target tools and identify potential risks caused by the combination of multiple tools. Among them, the multi-step strategy reinforcement module is used to generate new strategy rules for the target tool based on multi-step dangerous trajectories, in order to supplement the risks that static strategies cannot cover.

[0063] Optionally, the target tool definition information includes: tool name, tool function description, parameter list, parameter name, parameter type, and parameter description.

[0064] Optionally, the static risk scenario refers to a risk scenario identified solely based on the tool's semantics, parameter type, parameter value pattern, processing object sensitivity, and target area. The static risk scenarios include: scenario number, scenario title, severity level, risk cause, example parameters, and detection approach.

[0065] Optionally, the constraints followed by the dangerous trajectories invoked by the generation tool include: In a multi-step hazardous trajectory, each tool invocation step preceding the target tool invocation must not violate existing static policies; Risk must arise from a combination of multiple steps, rather than from a single step being illegal in itself; The final step must be the invocation of the target tool; The trajectory must reflect the propagation process of sensitive data within the toolchain; The output of each step in the trajectory can carry or inherit sensitive metadata.

[0066] Optionally, the execution rules include: generalizable single-step rules and multi-step association rules; Among them, generalizable single-step rules refer to rules that can complete the matching judgment based solely on the parameter pattern of the current tool call; they do not depend on historical call records or context, but only on the parameters of the current tool call itself. Among them, multi-step association rules are rules that combine the current tool call parameters with the historical tool call sequence to complete the matching judgment, and are used to identify risks formed by the combination of multiple legitimate steps.

[0067] Optionally, the runtime rule execution module 360 ​​is used for: Get the name and parameters of the tool currently being called; Based on the tool name and specific parameters of the current tool call, the path normalization module maps the real path to the defined abstract security area, thereby achieving cross-environment rule matching. Based on tool parameters, abstract security zones, and sensitive data metadata, execute generalizable single-step rules; If a multi-step rule exists, the multi-step matching strategy is executed by combining the agent's historical tool call records; if any rejection policy is hit, the current tool call is directly blocked; if no rejection policy is hit, the tool call is allowed. After a tool call is completed, it outputs the results of the currently completed tool call or inherits the corresponding sensitive data metadata. The output results are used as the basis for review in the next tool call. The output results refer to the execution result data returned after the current tool call is completed, including: text, file path, structured object, command output, and resource creation results.

[0068] The embodiments of the present invention can identify and reinforce the risks of multi-tool invocation that cannot be covered by single-step strategies, and improve the protection against risks such as compound attacks, indirect leakage and process bypass. Under the premise of ensuring auditability, reproducibility and scalability, the embodiments of the present invention improve the permission security of intelligent agent systems in scenarios such as sensitive data processing, file system operations, command execution and data outgoing.

[0069] The embodiments of the present invention can automatically generate static policies and reinforcement policies based on tool definitions, parameter descriptions and risk scenarios, reducing the workload of manually writing permission rules one by one and improving development and maintenance efficiency.

[0070] Existing single-step review methods struggle to identify attack chains where each step is legitimate but the combination is dangerous. This invention, through multi-step dangerous trajectory generation and multi-step strategy reinforcement, can identify and reinforce implicit risks in multi-tool call chains, thereby significantly improving security coverage.

[0071] This invention introduces abstract security areas and sensitive data metadata, which can uniformly represent and control the propagation of various types of sensitive data such as ID cards, contact information, educational background, financial information, or health information, thereby effectively improving the protection capability of sensitive data in intelligent agent systems.

[0072] The strategies generated in this embodiment of the invention are ultimately implemented as deterministic rules that can be executed by the rule engine. There is no need to call the large model again for security determination at runtime. It has the advantages of stable results, reproducibility, auditability and interpretability.

[0073] By normalizing and mapping real paths to abstract security regions, this invention avoids strong binding of rules to specific device and operating system paths, enabling policies to have cross-environment migration capabilities and higher reusability.

[0074] Figure 4 This is a schematic diagram of the structure of an automated static permission policy generation device for LLM agent tool calls provided in an embodiment of the present invention, as shown below. Figure 4 As shown, the static permission policy automated generation device for LLM agent tool calls can include the above-mentioned Figure 3 The illustrated device is an automated static permission policy generation apparatus for LLM agent tool calls. Optionally, the automated static permission policy generation apparatus 410 for LLM agent tool calls may include a first processor 2001.

[0075] Optionally, the static permission policy automated generation device 410 for LLM agent tool calls may also include a memory 2002 and a transceiver 2003.

[0076] The first processor 2001, memory 2002, and transceiver 2003 can be connected via a communication bus.

[0077] The following is combined Figure 4 The components of the static permission policy automated generation device 410 for LLM agent tool calls are described in detail below: The first processor 2001 is the control center of the static permission policy automated generation device 410 for LLM agent tool calls. It can be a single processor or a collective term for multiple processing elements. For example, the first processor 2001 can be one or more central processing units (CPUs), application-specific integrated circuits (ASICs), or one or more integrated circuits configured to implement embodiments of the present invention, such as one or more digital signal processors (DSPs), or one or more field-programmable gate arrays (FPGAs).

[0078] Optionally, the first processor 2001 can execute various functions of the static permission policy automation generation device 410 for LLM agent tool calls by running or executing software programs stored in memory 2002 and calling data stored in memory 2002.

[0079] In a specific implementation, as one example, the first processor 2001 may include one or more CPUs, for example... Figure 4 CPU0 and CPU1 are shown in the diagram.

[0080] In a specific implementation, as one example, the static permission policy automated generation device 410 for LLM agent tool calls may also include multiple processors, for example... Figure 4 The first processor 2001 and the second processor 2004 are shown in the diagram. Each of these processors can be a single-core processor or a multi-core processor. Here, a processor can refer to one or more devices, circuits, and / or processing cores used to process data (such as computer program instructions).

[0081] The memory 2002 is used to store the software program that executes the present invention, and is controlled by the first processor 2001 to execute it. The specific implementation method can be referred to the above method embodiment, and will not be repeated here.

[0082] Optionally, the memory 2002 may be a read-only memory (ROM) or other type of static storage device capable of storing static information and instructions, random access memory (RAM) or other type of dynamic storage device capable of storing information and instructions, or electrically erasable programmable read-only memory (EEPROM), compact disc read-only memory (CD-ROM) or other optical disc storage, optical disc storage (including compressed optical discs, laser discs, optical discs, digital universal optical discs, Blu-ray discs, etc.), magnetic disk storage media or other magnetic storage devices, or any other medium capable of carrying or storing desired program code in the form of instructions or data structures and accessible by a computer, but not limited thereto. The memory 2002 may be integrated with the first processor 2001 or may exist independently, and may be connected to the interface circuit of the static permission policy automated generation device 410 via LLM agent tool calls. Figure 4 (Not shown in the image) is coupled to the first processor 2001, and this embodiment of the invention does not specifically limit this.

[0083] The transceiver 2003 is used to communicate with network devices or with terminal devices.

[0084] Alternatively, transceiver 2003 may include a receiver and a transmitter. Figure 4 (Not shown separately). The receiver is used to implement the receiving function, and the transmitter is used to implement the transmitting function.

[0085] Optionally, the transceiver 2003 can be integrated with the first processor 2001 or exist independently, and the interface circuit of the device 410 can be automatically generated through a static permission policy for LLM agent tool calls. Figure 4 (Not shown in the image) is coupled to the first processor 2001, and this embodiment of the invention does not specifically limit this.

[0086] It should be noted that, Figure 4 The structure of the static permission policy automated generation device 410 for LLM agent tool calls shown in the diagram does not constitute a limitation on the router. Actual static permission policy automated generation devices for LLM agent tool calls may include more or fewer components than shown, or combine certain components, or have different component arrangements.

[0087] Furthermore, the technical effects of the static permission policy automated generation device 410 for LLM agent tool calls can be referred to the technical effects of the static permission policy automated generation method for LLM agent tool calls described in the above method embodiments, and will not be repeated here.

[0088] It should be understood that the first processor 2001 in the embodiments of the present invention may be a central processing unit (CPU), or it may be other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor may be a microprocessor, or it may be any conventional processor, etc.

[0089] It should also be understood that the memory in the embodiments of the present invention can be volatile memory or non-volatile memory, or may include both volatile and non-volatile memory. The non-volatile memory can be read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), or flash memory. The volatile memory can be random access memory (RAM), which is used as an external cache. By way of example, but not limitation, many forms of random access memory (RAM) are available, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate synchronous DRAM (DDR SDRAM), enhanced synchronous DRAM (ESDRAM), synchronous linked DRAM (SLDRAM), and direct rambus RAM (DR RAM).

[0090] The above embodiments can be implemented, in whole or in part, by software, hardware (such as circuits), firmware, or any other combination thereof. When implemented using software, the above embodiments can be implemented, in whole or in part, as a computer program product. The computer program product includes one or more computer instructions or computer programs. When the computer instructions or computer programs are loaded or executed on a computer, all or part of the processes or functions described in the embodiments of the present invention are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that includes one or more sets of available media. The available medium can be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium. A semiconductor medium can be a solid-state drive.

[0091] It should be understood that the term "and / or" in this article is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, or B existing alone. A and B can be singular or plural. Additionally, the character " / " in this article generally indicates an "or" relationship between the preceding and following related objects, but it can also represent an "and / or" relationship. Please refer to the context for a more accurate understanding.

[0092] In this invention, "at least one" means one or more, and "more than one" means two or more. "At least one of the following" or similar expressions refer to any combination of these items, including any combination of a single item or a plurality of items. For example, at least one of a, b, or c can represent: a, b, c, ab, ac, bc, or abc, where a, b, and c can be a single item or multiple items.

[0093] It should be understood that, in various embodiments of the present invention, the order of the above-mentioned process numbers does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present invention.

[0094] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementations should not be considered beyond the scope of this invention.

[0095] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working processes of the devices, apparatuses, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0096] In the several embodiments provided by this invention, it should be understood that the disclosed devices, apparatuses, and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another device, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between devices or units may be electrical, mechanical, or other forms.

[0097] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0098] In addition, the functional units in the various embodiments of the present invention can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit.

[0099] If the aforementioned functions are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this invention, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0100] The above description is merely a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in the present invention should be included within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the scope of the claims.

Claims

1. A static permission policy automatic generation method for LLM intelligent agent tool invocation, characterized in that, The method includes: S1. Construct an automated generation framework for tool static permission policies for LLM intelligent agent systems; wherein, the framework includes: a tool description receiving module, a static risk scenario generation module, a static policy generation module, a multi-step danger trajectory generation module, a multi-step policy reinforcement module, and a runtime rule execution module; S2. Obtain the target tool definition information and input it into the tool description receiving module. Combine the predefined abstract security area system and sensitive data metadata system to perform security semantic mapping on the parameters, input objects and output objects of the target tool. Input the target tool definition information into the static risk scenario generation module to generate static risk scenarios. S3. Input the static risk scenario into the static policy generation module to convert the static risk scenario into a single-step rejection policy that can be directly executed by the rule engine; S4. For complex risks involving multiple tools, with a specific target tool as the final destination, the target tool definition information, system tool definition set, static risk scenarios, and predefined abstract security area system and sensitive data metadata system are input into the multi-step danger trajectory generation module to automatically generate the danger trajectory of multi-step tool calls. S5. Input the dangerous trajectory and target tool definition information of the tool call into the multi-step strategy reinforcement module, generate a reinforcement strategy based on the single-step rejection strategy, and construct the execution rules; S6. Input the execution rules into the runtime rule execution module for real-time review by the rule engine and complete the tool call.

2. The method for automatically generating static permission policies for LLM agent tool calls according to claim 1, characterized in that, The tool description receiving module is used to receive the tool name, tool function description, parameter description, and parameter type. Among them, the static risk scenario generation module is used to automatically generate static risk scenarios for the tool based on the tool definition information; Among them, the static policy generation module is used to convert static risk scenarios into single-step rejection policies that can be directly executed by the rule engine; Among them, the multi-step danger trajectory generation module is used to generate multi-step call trajectories for target tools and identify potential risks caused by the combination of multiple tools. Among them, the multi-step strategy reinforcement module is used to generate new strategy rules for the target tool based on multi-step dangerous trajectories, in order to supplement the risks that static strategies cannot cover; The runtime rule execution module is used to review the current call and the necessary context before each actual tool call by the agent, and determine whether to allow or deny it.

3. The method for automatically generating static permission policies for LLM agent tool calls according to claim 1, characterized in that, The target tool definition information includes: tool name, tool function description, parameter list, parameter name, parameter type, and parameter description.

4. The method for automatically generating static permission policies for LLM agent tool calls according to claim 1, characterized in that, The static risk scenario refers to a risk scenario identified solely based on the tool's semantics, parameter type, parameter value pattern, sensitivity of the processed object, and target area. The static risk scenarios include: scenario number, scenario title, severity level, risk cause, example parameters, and detection approach.

5. The method for automatically generating static permission policies for LLM agent tool calls according to claim 1, characterized in that, The constraints that the multi-step tool for generating dangerous trajectories follows include: In a multi-step hazardous trajectory, each tool invocation step preceding the target tool invocation must not violate existing static policies; Risk must arise from a combination of multiple steps, rather than from a single step being illegal in itself; The final step must be the invocation of the target tool; The trajectory must reflect the propagation process of sensitive data within the toolchain; The output of each step in the trajectory can carry or inherit sensitive metadata.

6. The method for automatically generating static permission policies for LLM agent tool calls according to claim 1, characterized in that, The execution rules include: generalizable single-step rules and multi-step association rules; Among them, generalizable single-step rules refer to rules that can complete the matching judgment based solely on the parameter pattern of the current tool call; they do not depend on historical call records or context, but only on the parameters of the current tool call itself. Among them, multi-step association rules are rules that combine the current tool call parameters with the historical tool call sequence to complete the matching judgment, and are used to identify risks formed by the combination of multiple legitimate steps.

7. The method for automatically generating static permission policies for LLM agent tool calls according to claim 1, characterized in that, The process described in S6 involves inputting the execution rules into the runtime rule execution module for real-time review by the rule engine, and completing the tool invocation, including: S61. Obtain the name and specific parameters of the tool currently being called; S62. Based on the tool name and specific parameters of the current tool call, the path normalization module maps the real path to the defined abstract security area to achieve cross-environment rule matching. S63. Execute generalizable single-step rules based on tool parameters, abstract security areas, and sensitive data metadata; S64. If a multi-step rule exists, combine the agent's historical tool call records and execute the multi-step matching strategy; if any rejection policy is hit, the current tool call is directly blocked; if no rejection policy is hit, the tool call is allowed. S65. After the tool call is completed, output the results of the currently completed tool call or inherit the corresponding sensitive data metadata; the output results are used as the basis for the review of the next tool call; the output results refer to the execution result data returned after the current tool call is completed, including: text, file path, structured object, command output and resource creation results.

8. An automated static permission policy generation device for LLM agent tool calls, wherein the automated static permission policy generation device for LLM agent tool calls is used to implement the automated static permission policy generation method for LLM agent tool calls as described in any one of claims 1-7, characterized in that, The device includes: The tool description receiving module is used to obtain the target tool definition information and input it into the tool description receiving module. Combined with the predefined abstract security area system and sensitive data metadata system, it performs security semantic mapping on the parameters, input objects and output objects of the target tool. The static risk scenario generation module is used to input the target tool definition information into the static risk scenario generation module to generate static risk scenarios; The static policy generation module is used to input static risk scenarios into the static policy generation module and convert the static risk scenarios into single-step rejection policies that can be directly executed by the rule engine. The multi-step hazard trajectory generation module is used to address complex risks involving multiple tools working together. It takes a specific target tool as the final destination and inputs the target tool definition information, system tool definition set, static risk scenario, and predefined abstract security area system and sensitive data metadata system into the multi-step hazard trajectory generation module to automatically generate hazard trajectories for multi-step tool calls. The multi-step strategy reinforcement module is used to input the dangerous trajectory of the tool call and the target tool definition information into the multi-step strategy reinforcement module, generate reinforcement strategies based on the single-step rejection strategy, and construct execution rules; The runtime rule execution module is used to input execution rules into the runtime rule execution module for real-time review by the rule engine and to complete tool calls.

9. A device for automatically generating static permission policies for LLM agent tool calls, characterized in that, The automated generation device for static permission policies for LLM agent tool calls includes: processor; A memory storing computer-readable instructions that, when executed by the processor, implement the method as described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium contains program code that can be invoked by a processor to execute the method as described in any one of claims 1 to 7.

Images