Intelligent agent processing method and device and storage medium
By conducting security tests and generating strategies for various types of attacks on intelligent agents, the problem of insufficient protection mechanisms for intelligent agents was solved, and the stable operation and security of intelligent agents were improved.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- INDUSTRIAL AND COMMERCIAL BANK OF CHINA
- Filing Date
- 2026-01-30
- Publication Date
- 2026-05-01
AI Technical Summary
The agent's protection mechanism is insufficient, resulting in poor operational stability and difficulty in effectively defending against covert and flexible attacks, such as prompt word injection and role-playing hijacking.
By conducting security tests on the intelligent agent for various types of attack behaviors, a security capability assessment report is generated. This report is then transformed into a protection policy using a policy generation engine. Combined with access control policies, input information is filtered, inputs that do not meet the policy requirements are blocked, and security vulnerabilities are patched when necessary.
It improves the operational stability and security of intelligent agents, enhances the ability to defend against various attack behaviors, ensures that input information and tool calls comply with permissions, and achieves stable operation of intelligent agents and data security.
Smart Images
Figure CN121967015A_ABST
Abstract
Description
Intelligent agent processing methods, devices and storage media Technical Field
[0001] This application relates to the field of network information security, and more specifically, to a method, apparatus, and storage medium for processing intelligent agents. Background Technology
[0002] In today's era of accelerated digital transformation, intelligent agents, with their superior language understanding and generation capabilities, have become a crucial driver of enterprise innovation and service upgrades. However, with the widespread application of these agents across various industries, the security issues they expose are becoming increasingly severe, particularly their defenses against internal design vulnerabilities and novel attack methods. Intelligent agents often need to process large amounts of user-generated natural language input, which may contain carefully crafted attack instructions designed to induce unauthorized actions or the disclosure of sensitive information. Traditional security measures, such as rule-based content filtering and perimeter protection, while capable of intercepting obvious malicious behavior to some extent, fall short against covert and flexible attacks such as prompt injection and role-playing hijacking. Traditional security testing largely focuses on static analysis and conventional threat models. The vulnerability of intelligent agents to attacks, leading to functional failures or data breaches, can also make them a springboard for attackers to further infiltrate enterprise systems, triggering a chain reaction.
[0003] There is currently no effective solution to the problem of insufficient protection mechanisms for intelligent agents in related technologies, which leads to poor operational stability of intelligent agents. Summary of the Invention
[0004] The main objective of this application is to provide a method, apparatus, and storage medium for processing intelligent agents, in order to solve the problem that insufficient protection mechanisms for intelligent agents in related technologies lead to poor operational stability of intelligent agents.
[0005] To achieve the above objectives, according to one aspect of this application, a method for processing an intelligent agent is provided. The method includes: conducting security tests on the intelligent agent against multiple types of attack behaviors; recording the resistance effectiveness of the intelligent agent against each type of attack behavior; generating a security capability assessment report based on the recorded resistance effectiveness, wherein the security capability assessment report includes the intelligent agent's resistance capability score against each type of attack behavior and the intelligent agent's resistance result against each type of attack behavior; converting the security capability assessment report into a protection policy through a policy generation engine, wherein the protection policy includes adjustment defense configuration rules generated based on the intelligent agent's resistance capability score against each type of attack behavior, defense rules generated based on the intelligent agent's resistance result against each type of attack behavior, and attack chain response rules; filtering input information according to the protection policy and access control policy, blocking input information that does not meet the protection policy and access control policy to protect the intelligent agent, wherein the access control policy includes predefined access permissions and operation permissions for the intelligent agent.
[0006] Optionally, after converting the security capability assessment report into a protection policy through the policy generation engine, the method further includes: intercepting tool call requests that do not meet the access control policy and protection policy when an agent requests to call a tool, in order to protect the agent.
[0007] Optionally, before converting the security capability assessment report into a protection policy through the policy generation engine, the method further includes: detecting the configuration information of the agent before the agent runs to identify security vulnerabilities of the agent; fixing the security vulnerabilities of the agent to obtain a fixed agent.
[0008] Optionally, converting the security capability assessment report into a protection strategy through a strategy generation engine includes: recording the attack behavior, triggering conditions, and session context of the agent's failed resistance when the resistance result is failure; generating defense rules based on the attack behavior, triggering conditions, and session context of the agent's failed resistance; generating adjustment rules for defense configuration based on the agent's resistance score for each type of attack behavior, wherein the adjustment rules for defense configuration are rules for adjusting the defense configuration to deal with each type of attack behavior; generating attack chain response rules based on the agent's resistance result to the attack chain when the attack behavior is an attack chain; and generating a protection strategy based on the defense rules, the adjustment rules for defense configuration, and the attack chain response rules.
[0009] Optionally, filtering input information according to protection policies and access control policies, and blocking input information that does not meet the protection policies and access control policies to protect the intelligent agent includes: detecting whether the input information conforms to the intelligent agent's access permissions and operation permissions according to the access control policy, and blocking input information that does not conform to the intelligent agent's access permissions and operation permissions; detecting whether the input information conforms to the triggering conditions according to the protection policy, and blocking input information that conforms to the triggering conditions to protect the intelligent agent.
[0010] Optionally, after filtering the input information according to the protection policy and access control policy and blocking the input information that does not meet the protection policy and access control policy, the method further includes: recording a first interception event, wherein the first interception event includes the input information that does not meet the protection policy and access control policy; performing feature extraction on the first interception event to obtain attack instruction features; and updating the protection policy and access control policy according to the attack instruction features.
[0011] Optionally, after intercepting the tool call request that does not meet the access control policy and protection policy when a request to call a tool is detected by an agent, the method further includes: recording a second interception event, wherein the second interception event includes the tool call request that does not meet the access control policy and protection policy; performing feature extraction on the second interception event to obtain abnormal call features, wherein the abnormal call features are used to indicate that the agent's tool call request is abnormal; and updating the protection policy and access control policy according to the abnormal call features.
[0012] To achieve the above objectives, according to another aspect of this application, a processing apparatus for an intelligent agent is provided. The apparatus includes: a testing unit, configured to perform security tests on the intelligent agent against multiple types of attacks, record the agent's resistance effectiveness against each type of attack, and generate a security capability assessment report based on the recorded resistance effectiveness, wherein the security capability assessment report includes the agent's resistance capability score against each type of attack and the agent's resistance result against each type of attack; a conversion unit, configured to convert the security capability assessment report into a protection policy through a policy generation engine, wherein the protection policy includes adjustment defense configuration rules generated based on the agent's resistance capability score against each type of attack, defense rules generated based on the agent's resistance result against each type of attack, and attack chain response rules; and a filtering unit, configured to filter input information according to the protection policy and access control policy, blocking input information that does not meet the protection policy and access control policy to protect the intelligent agent, wherein the access control policy includes predefined access permissions and operation permissions for the intelligent agent.
[0013] According to another aspect of this application, a computer-readable storage medium is provided, which includes a stored program, wherein, when the program is executed, it controls the device on which the computer-readable storage medium is located to perform a processing method of any intelligent agent.
[0014] According to another aspect of this application, an electronic device is provided, comprising: one or more processors, a memory, and one or more programs, wherein the one or more programs are stored in the memory and configured to be executed by the one or more processors, and the one or more programs include a processing method for performing any kind of intelligent agent.
[0015] According to another aspect of this application, a computer program product is provided, including computer instructions, which, when executed by a processor, implement the steps of a processing method for an intelligent agent as described above.
[0016] In this embodiment, security tests are conducted on the agent against various types of attacks, and the agent's resistance to each type of attack is recorded. A security capability assessment report is generated based on the recorded resistance results. The security capability assessment report includes the agent's resistance score for each type of attack and the agent's resistance result for each type of attack. The security capability assessment report is then converted into a protection policy through a policy generation engine. The protection policy includes rules for adjusting defense configurations generated based on the agent's resistance score for each type of attack, defense rules generated based on the agent's resistance result for each type of attack, and attack chain response rules. Input information is filtered based on the protection policy and access control policy, and input information that does not meet the protection policy and access control policy is blocked to protect the agent. The access control policy includes predefined access permissions and operation permissions for the agent, solving the technical problem in related technologies where insufficient agent protection mechanisms lead to poor agent operation stability. In this application, a security test is first conducted on the agent, which includes various attack behaviors. Based on the test results, a security capability assessment report is generated, which includes the agent's resistance score against various attack behaviors and the resistance results. The security capability assessment report is then transformed into a protection strategy through a policy generation engine, including adjusting defense configuration rules, defense rules, and attack chain response rules. Based on the protection strategy and access control strategy, the agent filters and intercepts input information during runtime, thereby protecting the agent and achieving the technical effect of improving the stability of the agent's operation. Attached Figure Description
[0017] The accompanying drawings, which form part of this application, are used to provide a further understanding of this application. The illustrative embodiments and descriptions of this application are used to explain this application and do not constitute an undue limitation of this application. In the drawings:
[0018] Figure 1 shows a hardware structure block diagram of a computer terminal for implementing a processing method for intelligent agents;
[0019] Figure 2 is a flowchart of a processing method for an intelligent agent according to an embodiment of this application;
[0020] Figure 3 is a schematic diagram of intelligent agent full-cycle defense according to an embodiment of this application;
[0021] Figure 4 is a schematic diagram of the pre-risk control module provided according to an embodiment of this application;
[0022] Figure 5 is a schematic diagram of the real-time operation protection module provided according to an embodiment of this application;
[0023] Figure 6 is a schematic diagram of a post-operation monitoring module provided according to an embodiment of this application;
[0024] Figure 7 is a schematic diagram of a processing device for an intelligent agent according to an embodiment of this application;
[0025] Figure 8 is a structural block diagram of an electronic device according to an embodiment of this application. Detailed Implementation
[0026] To enable those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present application, and not all embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative effort should fall within the scope of protection of the present application.
[0027] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this application described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.
[0028] It should be noted that all relevant information (including but not limited to input information, configuration information, etc.) and data (including but not limited to data used for testing, data used for analysis, etc.) involved in this application are information and data authorized by the user or fully authorized by all parties. For example, if there is an interface between this system and the relevant user or organization, before obtaining the relevant information, it is necessary to send an acquisition request to the aforementioned user or organization through the interface, and obtain the relevant information after receiving the consent information from the aforementioned user or organization.
[0029] Example 1
[0030] According to an embodiment of this application, a method embodiment for processing an intelligent agent is also provided. It should be noted that the steps shown in the flowchart in the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions. Furthermore, although a logical order is shown in the flowchart, in some cases, the steps shown or described may be executed in a different order than that shown here.
[0031] The method embodiment provided in Embodiment 1 of this application can be executed in a mobile terminal, computer terminal, or similar computing device. Figure 1 shows a hardware structure block diagram of a computer terminal (or mobile device) for implementing a processing method for an intelligent agent. As shown in Figure 1, the computer terminal 10 (or mobile device) may include one or more processors 102 (shown as 102a, 102b, ..., 102n in the figure) (processor 102 may include, but is not limited to, a microprocessor MCU or a programmable logic device FPGA, etc.), a memory 104 for storing data, and a transmission device 106 for communication functions. In addition, it may also include: a display, an input / output interface (I / O interface), a Universal Serial Bus (USB) port (which may be included as one of the ports of a BUS bus), a network interface, a power supply, and / or a camera. Those skilled in the art will understand that the structure shown in Figure 1 is only illustrative and does not limit the structure of the above-described electronic device. For example, the computer terminal 10 may also include more or fewer components than shown in Figure 1, or have a different configuration than shown in Figure 1.
[0032] It should be noted that the aforementioned one or more processors 102 and / or other data processing circuits are generally referred to herein as "data processing circuits". These data processing circuits may be embodied, in whole or in part, in software, hardware, firmware, or any other combination thereof. Furthermore, the data processing circuits may be a single, independent processing module, or may be integrated, in whole or in part, into any other element within the computer terminal 10 (or mobile device). As involved in the embodiments of this application, the data processing circuits serve as a processor control mechanism (e.g., selection of a variable resistor termination path connected to an interface).
[0033] The memory 104 can be used to store software programs and modules of application software, such as the program instructions / data storage device corresponding to the intelligent agent processing method in the embodiments of this application. The processor 102 executes various functional applications and data processing by running the software programs and modules stored in the memory 104, thereby realizing the aforementioned intelligent agent processing method. The memory 104 may include high-speed random access memory, and may also include non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some instances, the memory 104 may further include memory remotely located relative to the processor 102, and these remote memories can be connected to the computer terminal 10 via a network. Examples of such networks include, but are not limited to, the Internet, corporate intranets, local area networks, mobile communication networks, and combinations thereof.
[0034] The transmission device 106 is used to receive or send data via a network. Specific examples of the network described above may include a wireless network provided by the communication provider of the computer terminal 10. In one example, the transmission device 106 includes a Network Interface Controller (NIC), which can connect to other network devices via a base station to communicate with the Internet. In another example, the transmission device 106 may be a Radio Frequency (RF) module, used for wireless communication with the Internet.
[0035] The display can be, for example, a touchscreen liquid crystal display (LCD), which allows the user to interact with the user interface of the computer terminal 10 (or mobile device).
[0036] Under the above operating environment, this application provides a processing method for an intelligent agent as shown in Figure 2. Figure 2 is a flowchart of the processing method for an intelligent agent according to Embodiment 1 of this application.
[0037] Step S201: Conduct security tests on the agent for multiple types of attack behaviors, record the agent's resistance to each type of attack behavior, and generate a security capability assessment report based on the recorded resistance effects. The security capability assessment report includes the agent's resistance capability score for each type of attack behavior and the agent's resistance result for each type of attack behavior.
[0038] Optionally, security testing can be conducted on the agent by constructing or selecting a series of attacks specifically designed to test the agent's security. These attacks can include common attack patterns or novel attacks targeting the agent's unique characteristics. The security testing process records the agent's resistance performance in each round of testing, including how it handles input information, requests for permission to invoke tools or APIs, how it resists attacks, and whether the final resistance result meets expectations. The security capability assessment report quantifies the agent's resistance to each attack and describes the specific resistance results for each security test, such as successful interception, false positives, false negatives, and response delays.
[0039] Step S202: The security capability assessment report is converted into a protection strategy through the strategy generation engine. The protection strategy includes adjustment defense configuration rules generated based on the agent's resistance score for each type of attack, defense rules generated based on the agent's resistance results for each type of attack, and attack chain response rules.
[0040] Optionally, the policy generation engine can automatically generate corresponding protection policies based on the security capability assessment report. These protection policies can guide agents on how to defend against and respond to various types of attacks. Protection policies include adjustment rules for defense configuration, defense rules, and attack chain response rules. Defense configuration rules adjust the configuration of defense mechanisms based on the agent's resistance score to a certain type of attack, such as raising or lowering certain alert thresholds or enabling or disabling specific security algorithms. Defense rules are effective defense measures extracted directly from the resistance results; they are specific rules targeting specific types of attacks, such as using regular expressions to filter specific dangerous words or employing machine learning models to identify abnormal behavior patterns. Attack chain response rules are corresponding session-level protection policies formulated by identifying patterns of multi-step combined attacks to prevent attackers from bypassing single-point defenses through coherent operations.
[0041] Step S203: Filter the input information according to the protection policy and access control policy, and block the input information that does not meet the protection policy and access control policy in order to protect the intelligent agent. The access control policy includes predefined access permissions and operation permissions of the intelligent agent.
[0042] Optionally, the input information is an instruction, request, or data issued by a user or other entity to the intelligent agent. Protection policies can proactively identify and intercept potential attack instructions or abnormal operation instructions in the input information. Access control policies can control the intelligent agent to only access resources within its authorized scope; even if an attacker attempts to induce the intelligent agent to perform unauthorized operations through input information, the attack will be blocked due to permission verification failure.
[0043] The intelligent agent processing method provided in this application embodiment performs security tests on the intelligent agent against multiple types of attacks, records the resistance effect of the intelligent agent against each type of attack, and generates a security capability assessment report based on the recorded resistance effect. The security capability assessment report includes the intelligent agent's resistance score against each type of attack and the resistance result of the intelligent agent against each type of attack. A policy generation engine converts the security capability assessment report into a protection policy, which includes adjustment rules for defense configuration generated based on the intelligent agent's resistance score against each type of attack, defense rules generated based on the resistance result of the intelligent agent against each type of attack, and attack chain response rules. Input information is filtered according to the protection policy and access control policy, blocking input information that does not meet the protection policy and access control policy to protect the intelligent agent. The access control policy includes predefined access permissions and operation permissions for the intelligent agent, solving the technical problem in related technologies where insufficient intelligent agent protection mechanisms lead to poor operational stability of the intelligent agent. In this application, a security test is first conducted on the agent, which includes various attack behaviors. Based on the test results, a security capability assessment report is generated, which includes the agent's resistance score against various attack behaviors and the resistance results. The security capability assessment report is then transformed into a protection strategy through a policy generation engine, including adjusting defense configuration rules, defense rules, and attack chain response rules. Based on the protection strategy and access control strategy, the agent filters and intercepts input information during runtime, thereby protecting the agent and achieving the technical effect of improving the stability of the agent's operation.
[0044] To intercept abnormal tool invocation requests from intelligent agents, optionally, in the intelligent agent processing method provided in this application embodiment, after converting the security capability assessment report into a protection policy through the policy generation engine, the method further includes: when an intelligent agent is detected requesting to invoke a tool, intercepting the tool invocation request that does not meet the access control policy and the protection policy, so as to protect the intelligent agent.
[0045] Optionally, when an agent receives an instruction to call an external tool or service during operation, it first submits a tool call request. Then, it needs to evaluate whether the request meets the pre-defined access control and protection policies. The access control policy can be used to check if the tool call is within the agent's authorized scope. If the agent is not granted permission to call the tool, or if the tool call parameters exceed the specified range, the call request will be blocked. The protection policy can be used to check for anomalies in the call parameters, including whether parameter values are out of bounds (e.g., reading file paths outside the allowed range), whether they contain sensitive data, and whether they conform to the business context of the current session, thereby preventing tool abuse and unauthorized operations.
[0046] In summary, by following the steps outlined above and combining access control and protection strategies to assess and block abnormal tool requests from intelligent agents, we can strengthen the agent's resilience and improve its stability during operation.
[0047] To improve the security of intelligent agents, optionally, in the intelligent agent processing method provided in the embodiments of this application, before converting the security capability assessment report into a protection policy through the policy generation engine, the method further includes:
[0048] The first step is to detect the configuration information of the agent before it runs and identify security vulnerabilities.
[0049] Optionally, the agent's configuration information may include model configuration files, as well as system prompts, MCP tool code, etc. The agent's configuration information can be detected and security vulnerabilities identified through techniques such as static code analysis, regular expression matching, and semantic analysis.
[0050] The second step is to fix the security vulnerabilities of the intelligent agent, resulting in a repaired intelligent agent.
[0051] Optionally, the security of the agent can be improved by modifying the code, updating the version of the dependent library, adjusting the configuration parameters, and encrypting the communication content to fix potential design flaws, security vulnerabilities and malicious code.
[0052] In summary, by taking the above steps, comprehensive detection and identification of security vulnerabilities are carried out before the intelligent agent runs, and these vulnerabilities are then repaired, thereby improving the security of the intelligent agent.
[0053] To improve the security and operational stability of intelligent agents, optionally, in the intelligent agent processing method provided in this application embodiment, converting the security capability assessment report into a protection strategy through a policy generation engine includes:
[0054] The first step is to record the attack behavior, triggering conditions, and session context of the agent's failed resistance when the resistance fails, and to generate defense rules based on the attack behavior, triggering conditions, and session context of the agent's failed resistance.
[0055] Optionally, in security testing, the agent's resistance to a certain type of attack can be categorized into two outcomes: successful resistance and failed resistance. The triggering condition for failed resistance refers to the specific environment or situation that leads to the agent's failure, which could be a particular input word, a specific sequence of operations, or a specific conversational scenario. The conversational context refers to the interaction history between the agent and its user or environment at the time of the attack, including previous dialogue content and operation commands. Defense rules generated based on the agent's failed resistance attack, triggering conditions, and conversational context can be used to automatically identify and intercept similar attack behaviors during agent runtime.
[0056] The second step is to generate adjustment rules for the defense configuration based on the agent's resistance score to each type of attack. These adjustment rules are rules for adjusting the defense configuration to deal with each type of attack.
[0057] Optionally, the resistance score is a quantitative description of the agent's defense level against various attack behaviors in the security capability assessment report. The lower the score, the weaker the agent's protection against that type of attack. Adjusting the defense configuration rules can be done by adjusting the configuration parameters of the agent's defense mechanism based on its resistance score to optimize the overall protection effect. For example, increasing the sensitivity of security detection or adjusting the interception threshold.
[0058] The third step is to generate attack chain response rules based on the agent's resistance to the attack chain when the attack behavior is an attack chain.
[0059] Optionally, an attack chain is a series of interconnected attack behaviors aimed at bypassing a single point of defense of an agent to achieve deep damage or control. Attack chain response rules can be used to identify and intercept consecutive attack behaviors within an attack chain.
[0060] The fourth step is to generate a protection strategy based on the defense rules, the adjusted defense configuration rules, and the attack chain response rules.
[0061] Optionally, defense rules, rules for adjusting defense configurations, and rules for responding to attack chains can be integrated into a set of protection strategies that can be used to guide the real-time operation and protection of intelligent agents.
[0062] In summary, by taking the above steps—first generating defense rules, adjusting defense configuration rules, and attack chain response rules separately, and then integrating the three rules into a set of protection strategies—it helps the intelligent agent resist various attack behaviors, thereby improving the security and operational stability of the intelligent agent.
[0063] To ensure the stable operation and data security of the intelligent agent, optionally, in the intelligent agent processing method provided in this application embodiment, input information is filtered according to protection policies and access control policies, and input information that does not meet the protection policies and access control policies is blocked to protect the intelligent agent, including:
[0064] The first step is to check whether the input information complies with the agent's access permissions and operation permissions according to the access control policy, and to block input information that does not comply with the agent's access permissions and operation permissions.
[0065] Optionally, access permissions refer to the specific data types or data sources that an agent is allowed to access, such as databases, file systems, and network resources. Operation permissions refer to the specific system functions or tool calls that an agent is authorized to execute, such as file reading and writing, external API calls, and system command execution. For example, if the input information contains the instruction "get all user data," but the agent's access permissions are limited to querying the current user's data, then the input information will be considered a violation of the access control policy and will be blocked.
[0066] The second step is to detect whether the input information meets the triggering conditions according to the protection strategy, and to block the input information that meets the triggering conditions in order to protect the intelligent agent.
[0067] Optionally, the triggering condition refers to the condition under which the defense rules in the protection policy are activated under specific circumstances, including keywords in the input information, specific sentence structures, abnormal behavior in the session context, etc. For example, if the input information contains specific words or instructions identified in previous security tests for use in prompt injection, this input can be intercepted according to the defense rules in the protection policy to prevent the execution of the attack.
[0068] In summary, by combining access control and protection strategies through the above steps, and identifying and intercepting input information, malicious operations can be prevented, which contributes to the stable operation of the intelligent agent and data security.
[0069] To continuously optimize the security strategy of the intelligent agent, optionally, in the intelligent agent processing method provided in the embodiments of this application, after filtering the input information according to the protection strategy and access control strategy and blocking the input information that does not meet the protection strategy and access control strategy, the method further includes:
[0070] The first step is to record the first interception event, which includes input information indicating that the protection policy and access control policy are not satisfied.
[0071] Optionally, the first interception event refers to non-compliant or potentially malicious input information that is intercepted by the intelligent agent during operation after filtering the input information according to access control policies and protection policies.
[0072] The second step is to extract features from the first interception event to obtain the attack command features.
[0073] Optionally, attack command features are information extracted from the first interception event that can represent certain attack behavior characteristics or patterns, such as specific keywords, abnormal statement structures, continuous command sequences, etc. Attack command features can be used to construct more accurate protection rules.
[0074] The third step is to update the protection and access control policies based on the characteristics of the attack commands.
[0075] Optionally, based on the attack instruction characteristics extracted from the first interception event, existing protection and access control policies can be adjusted or supplemented to enhance the security defense capabilities of the agent.
[0076] In summary, by transforming runtime security events into a driving force for policy optimization through the above steps, a closed loop from detection, logging, analysis to policy updates is formed, improving the flexibility and effectiveness of protection.
[0077] To continuously optimize the security strategy of the intelligent agent, optionally, in the intelligent agent processing method provided in the embodiments of this application, after intercepting the request to call a tool that does not meet the access control policy and protection policy when an intelligent agent requests to call a tool, the method further includes:
[0078] The first step is to record the second interception event, which includes tool call requests that do not meet the access control policy and protection policy.
[0079] Optionally, the second interception event is an event in which an agent requests to invoke a tool and is intercepted because the request does not meet the access control policy and / or protection policy.
[0080] The second step is to extract features from the second interception event to obtain abnormal call features, which are used to indicate that there are abnormalities in the agent's request to call the tool.
[0081] Optionally, the second interception event can be summarized and the features that deviate from the normal tool call pattern can be extracted, namely the abnormal call features mentioned above, such as specific call sequences, suspicious parameter combinations, unusual call frequency or time, etc.
[0082] The third step is to update the protection and access control policies based on the characteristics of abnormal calls.
[0083] Optionally, after analyzing the characteristics of abnormal calls, existing protection and access control policies can be adjusted to better avoid security vulnerabilities in intelligent agents and improve the ability of intelligent agents to resist new types of attacks.
[0084] In summary, by following the steps described above, recording the second interception event, extracting abnormal call characteristics, and updating the protection and access control policies, the security protection capabilities of the intelligent agent can be continuously optimized.
[0085] Optionally, this application provides a schematic diagram of the full lifecycle defense of intelligent agents as shown in Figure 3, including a pre-event risk control module, an in-event operation protection module, and a post-event operation monitoring module. The pre-event risk control module is used to establish a security baseline for the intelligent agent's online deployment, the in-event operation protection module is used to provide real-time protection during the intelligent agent's operation, and the post-event operation monitoring module is used to monitor the intelligent agent's operating status, generate enhanced test cases and enhanced policy rules, and feed them back to the pre-event risk control module and the in-event operation protection module.
[0086] Figure 4 illustrates the pre-emptive risk control module. This module receives configuration information from the agent, including system prompts, MCP tool code, and model configuration files. Through static code analysis, regular expression matching, and semantic analysis, it automatically scans the agent's configuration files, identifies security threats, and remediates them. The module also creates a unique mathematical identity for the agent, predefines the list of data resources accessible to that identity and the scope of operable tools, and generates an access control policy file for the agent. Furthermore, the module can call built-in test case templates, utilize the test case library to evaluate and record the agent's responses to various attacks, generate evaluation results, create an attack capability profile, and thus generate a security capability assessment report. Finally, the post-event monitoring module monitors the agent's operational status and updates the test cases in the test case library using the monitoring results.
[0087] Figure 5 illustrates the in-process protection module. This module receives the anti-attack capability profile and access control policies generated by the pre-attack risk control module. It then uses an initial policy rule generator to convert the anti-attack capability profile into policy rules, which are loaded into the policy library. The access control policies are also loaded into the policy library. User input information is identified and filtered using the rules in the policy library, allowing input that meets the rules and blocking input that does not. When an agent initiates a tool call, the access control policies verify whether the agent has the right to call the target tool. Furthermore, the post-attack monitoring module monitors the agent's operational status and updates the policy rule library using the monitoring results.
[0088] Figure 6 shows a schematic diagram of the post-event operation monitoring module. The operation monitoring unit in the post-event operation monitoring module is used to record security events, analyze the characteristics of security events through the feature analysis engine, transform the security event characteristics into rules and policies, update the defense rules, generate new standardized test cases based on the security event characteristics, and update the test case library.
[0089] It should be noted that the steps shown in the flowchart in the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions, and although a logical order is shown in the flowchart, in some cases the steps shown or described may be executed in a different order than that shown here.
[0090] Example 2
[0091] This application also provides a processing device for an intelligent agent. It should be noted that the processing device for the intelligent agent in this application can be used to execute the processing method for intelligent agents provided in this application. The processing device for the intelligent agent provided in this application will be described below.
[0092] According to an embodiment of this application, an apparatus for implementing the above-described processing method for an intelligent agent is also provided, as shown in FIG7. The apparatus includes a testing unit 701, a conversion unit 702, and a filtering unit 703.
[0093] Specifically, the test unit 701 is used to conduct security tests on the agent against multiple types of attacks, record the agent's resistance to each type of attack, and generate a security capability assessment report based on the recorded resistance. The security capability assessment report includes the agent's resistance score to each type of attack and the agent's resistance result to each type of attack.
[0094] The conversion unit 702 is used to convert the security capability assessment report into a protection strategy through the strategy generation engine. The protection strategy includes adjustment defense configuration rules generated based on the agent's resistance score to each type of attack behavior, defense rules generated based on the agent's resistance results to each type of attack behavior, and attack chain response rules.
[0095] The filtering unit 703 is used to filter input information according to the protection policy and the access control policy, and to block input information that does not meet the protection policy and the access control policy in order to protect the intelligent agent. The access control policy includes predefined access permissions and operation permissions of the intelligent agent.
[0096] The intelligent agent processing device provided in this application embodiment performs security tests on the intelligent agent for multiple types of attack behaviors through a testing unit 701, records the resistance effect of the intelligent agent to each type of attack behavior, and generates a security capability assessment report based on the recorded resistance effect. The security capability assessment report includes the intelligent agent's resistance capability score for each type of attack behavior and the resistance result of the intelligent agent for each type of attack behavior. A conversion unit 702 converts the security capability assessment report into a protection strategy through a strategy generation engine. The protection strategy includes adjustment defense configuration rules generated based on the intelligent agent's resistance capability score for each type of attack behavior, defense rules generated based on the resistance result of the intelligent agent for each type of attack behavior, and attack chain response rules. A filtering unit 703 filters input information according to the protection strategy and access control strategy, blocking input information that does not meet the protection strategy and access control strategy to protect the intelligent agent. The access control strategy includes predefined access permissions and operation permissions for the intelligent agent, solving the technical problem in related technologies where insufficient intelligent agent protection mechanisms lead to poor intelligent agent operation stability, and achieving the technical effect of improving the operational stability of the intelligent agent.
[0097] Optionally, in the processing apparatus for the intelligent agent provided in the embodiments of this application, the apparatus further includes: an interception unit, configured to, after converting the security capability assessment report into a protection policy through the policy generation engine, intercept the application to call the tool that does not meet the access control policy and the protection policy when the intelligent agent is detected to be requesting to call the tool, so as to protect the intelligent agent.
[0098] Optionally, in the processing apparatus for the intelligent agent provided in the embodiments of this application, the apparatus further includes: a detection unit, used to detect the configuration information of the intelligent agent and identify security vulnerabilities of the intelligent agent before the security capability assessment report is converted into a protection policy through the policy generation engine and before the intelligent agent runs; and a repair unit, used to repair the security vulnerabilities of the intelligent agent to obtain a repaired intelligent agent.
[0099] Optionally, in the agent processing apparatus provided in this application embodiment, the conversion unit 702 includes: a recording module, configured to record the agent's failed resistance attack behavior, triggering conditions, and session context when the resistance result is a failure, and generate defense rules based on the agent's failed resistance attack behavior, triggering conditions, and session context; a first generation module, configured to generate adjustment defense configuration rules based on the agent's resistance capability score for each type of attack behavior, wherein the adjustment defense configuration rules are adjustment rules for the defense configuration in response to each type of attack behavior; a second generation module, configured to generate attack chain response rules based on the agent's resistance result to the attack chain when the attack behavior is an attack chain; and a third generation module, configured to generate a protection strategy based on the defense rules, the adjustment defense configuration rules, and the attack chain response rules.
[0100] Optionally, in the processing device for the intelligent agent provided in the embodiments of this application, the filtering unit 703 includes: a first interception module, used to detect whether the input information conforms to the access permissions and operation permissions of the intelligent agent according to the access control policy, and to intercept the input information that does not conform to the access permissions and operation permissions of the intelligent agent; and a second interception module, used to detect whether the input information conforms to the triggering conditions according to the protection policy, and to intercept the input information that conforms to the triggering conditions, so as to protect the intelligent agent.
[0101] Optionally, in the processing device for the intelligent agent provided in the embodiments of this application, the device further includes: a first recording unit, configured to record a first interception event after filtering input information according to the protection policy and access control policy and intercepting input information that does not meet the protection policy and access control policy, wherein the first interception event includes input information that does not meet the protection policy and access control policy; a first extraction unit, configured to extract features from the first interception event to obtain attack instruction features; and a first update unit, configured to update the protection policy and access control policy according to the attack instruction features.
[0102] Optionally, in the processing apparatus for an intelligent agent provided in the embodiments of this application, the apparatus further includes: a second recording unit, configured to record a second interception event after intercepting a tool call request that does not meet the access control policy and the protection policy when an intelligent agent requests to call a tool, wherein the second interception event includes a tool call request that does not meet the access control policy and the protection policy; a second extraction unit, configured to extract features from the second interception event to obtain abnormal call features, wherein the abnormal call features are used to indicate that the intelligent agent's tool call request is abnormal; and a second updating unit, configured to update the protection policy and the access control policy according to the abnormal call features.
[0103] It should be noted that the aforementioned test unit 701, conversion unit 702, and filtering unit 703 correspond to steps S201 to S203 in Embodiment 1. The three units and their corresponding steps implement the same instances and application scenarios, but are not limited to the content disclosed in Embodiment 1. It should also be noted that the aforementioned modules or units can be hardware or software components stored in a memory (e.g., memory 104) and processed by one or more processors (e.g., processors 102a, 102b, ..., 102n). These modules can also run as part of a device in the computer terminal 10 provided in Embodiment 1.
[0104] Example 3
[0105] Embodiments of this application may provide a computer terminal, which may be any computer terminal device in a group of computer terminals. Optionally, in this embodiment, the aforementioned computer terminal may also be replaced with a mobile terminal or an electronic device, etc.
[0106] Optionally, in this embodiment, the computer terminal may be located in at least one of a plurality of network devices in a computer network.
[0107] In this embodiment, the aforementioned computer terminal can execute the program code for the following steps in the intelligent agent processing method: performing security tests on the intelligent agent against multiple types of attack behaviors, recording the resistance effect of the intelligent agent against each type of attack behavior, and generating a security capability assessment report based on the recorded resistance effect, wherein the security capability assessment report includes the intelligent agent's resistance capability score against each type of attack behavior and the intelligent agent's resistance result against each type of attack behavior; converting the security capability assessment report into a protection policy through a policy generation engine, wherein the protection policy includes adjustment defense configuration rules generated based on the intelligent agent's resistance capability score against each type of attack behavior, defense rules generated based on the intelligent agent's resistance result against each type of attack behavior, and attack chain response rules; filtering input information according to the protection policy and access control policy, and blocking input information that does not meet the protection policy and access control policy to protect the intelligent agent, wherein the access control policy includes predefined access permissions and operation permissions for the intelligent agent.
[0108] Optionally, the aforementioned computer terminal may execute program code for the following steps in the agent processing method: after converting the security capability assessment report into a protection policy through the policy generation engine, the method further includes: if an agent requests to call a tool, intercepting the tool call request that does not meet the access control policy and the protection policy, in order to protect the agent.
[0109] Optionally, the computer terminal described above can execute program code for the following steps in the agent processing method: before converting the security capability assessment report into a protection policy through the policy generation engine, the method further includes: before the agent runs, detecting the agent's configuration information and identifying the agent's security vulnerabilities; repairing the agent's security vulnerabilities to obtain a repaired agent.
[0110] Optionally, the aforementioned computer terminal may execute program code for the following steps in the agent processing method: Converting the security capability assessment report into a protection strategy through a policy generation engine includes: recording the attack behavior, triggering conditions, and session context of the agent's failed resistance when the resistance result is resistance failure; generating defense rules based on the attack behavior, triggering conditions, and session context of the agent's failed resistance; generating adjustment defense configuration rules based on the agent's resistance capability score for each type of attack behavior, wherein the adjustment defense configuration rules are adjustment rules for the defense configuration to cope with each type of attack behavior; generating attack chain response rules based on the agent's resistance result to the attack chain when the attack behavior is an attack chain; and generating a protection strategy based on the defense rules, the adjustment defense configuration rules, and the attack chain response rules.
[0111] Optionally, the aforementioned computer terminal may execute program code for the following steps in the intelligent agent processing method: filtering input information according to protection policies and access control policies, and blocking input information that does not meet the protection policies and access control policies to protect the intelligent agent, including: detecting whether the input information conforms to the intelligent agent's access permissions and operation permissions according to the access control policy, and blocking input information that does not conform to the intelligent agent's access permissions and operation permissions; detecting whether the input information conforms to the triggering conditions according to the protection policy, and blocking input information that conforms to the triggering conditions to protect the intelligent agent.
[0112] Optionally, the aforementioned computer terminal may execute the program code for the following steps in the intelligent agent processing method: after filtering the input information according to the protection policy and access control policy, and intercepting the input information that does not meet the protection policy and access control policy, the method further includes: recording a first interception event, wherein the first interception event includes the input information that does not meet the protection policy and access control policy; performing feature extraction on the first interception event to obtain attack instruction features; and updating the protection policy and access control policy according to the attack instruction features.
[0113] Optionally, the computer terminal described above can execute the program code for the following steps in the agent processing method: after intercepting the application to call a tool that does not meet the access control policy and protection policy when an agent requests to do so, the method further includes: recording a second interception event, wherein the second interception event includes the application to call a tool that does not meet the access control policy and protection policy; performing feature extraction on the second interception event to obtain abnormal call features, wherein the abnormal call features are used to indicate that the agent's application to call a tool is abnormal; and updating the protection policy and access control policy according to the abnormal call features.
[0114] Optionally, FIG8 is a structural block diagram of an electronic device according to an embodiment of the present application. As shown in FIG8, the electronic device may include: one or more (only one is shown in FIG8) processors 802, memory 804, memory controller, and peripheral interface, wherein the peripheral interface is connected to a radio frequency module, an audio module, and a display.
[0115] The memory can be used to store software programs and modules, such as the program instructions / modules corresponding to the intelligent agent processing method and apparatus in this application embodiment. The processor executes various functional applications and data processing by running the software programs and modules stored in the memory, thereby realizing the aforementioned intelligent agent processing method. The memory may include high-speed random access memory, and may also include non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some instances, the memory may further include memory remotely located relative to the processor, and these remote memories can be connected to the terminal via a network. Examples of such networks include, but are not limited to, the Internet, corporate intranets, local area networks, mobile communication networks, and combinations thereof.
[0116] The processor can access the information and application programs stored in the memory via a transmission device to execute the steps described above in the processing method of the intelligent agent.
[0117] This application provides a solution for processing intelligent agents. By conducting security tests on the intelligent agent against various attack behaviors, the resistance effectiveness of the intelligent agent against each type of attack is recorded. A security capability assessment report is generated based on the recorded resistance effectiveness. The security capability assessment report includes the intelligent agent's resistance score and resistance result for each type of attack. A policy generation engine transforms the security capability assessment report into a protection policy. The protection policy includes rules for adjusting defense configurations generated based on the intelligent agent's resistance score for each type of attack, defense rules generated based on the resistance result of each type of attack, and attack chain response rules. Input information is filtered according to the protection policy and access control policy, blocking input information that does not meet the protection policy and access control policy to protect the intelligent agent. The access control policy includes predefined access permissions and operation permissions for the intelligent agent. This solves the technical problem in related technologies where insufficient protection mechanisms for intelligent agents lead to poor operational stability, and achieves the technical effect of improving the operational stability of intelligent agents.
[0118] It will be understood by those skilled in the art that the structure shown in Figure 8 is merely illustrative, and the electronic device may also be a smartphone, tablet computer, PDA, mobile internet device (MID), PAD, or other terminal device. Figure 8 does not limit the structure of the aforementioned electronic device. For example, the electronic device may include more or fewer components (such as network interfaces, display devices, etc.) than shown in Figure 8, or may have a different configuration than that shown in Figure 8.
[0119] Those skilled in the art will understand that all or part of the steps in the various methods of the above embodiments can be implemented by a program instructing the hardware related to the terminal device. The program can be stored in a computer-readable storage medium, which may include: flash drive, read-only memory (ROM), random access memory (RAM), disk or optical disk, etc.
[0120] Example 4
[0121] Embodiments of this application also provide a storage medium. Optionally, in this embodiment, the storage medium can be used to store the program code executed by the intelligent agent processing method provided in Embodiment 1.
[0122] Optionally, in this embodiment, the storage medium may be located in any computer terminal in a group of computer terminals in a computer network, or in any mobile terminal in a group of mobile terminals.
[0123] This application also provides a computer program product that, when executed on a data processing device, is suitable for performing the processing method steps of an intelligent agent.
[0124] The sequence numbers of the embodiments in this application are for descriptive purposes only and do not represent the superiority or inferiority of the embodiments.
[0125] In the above embodiments of this application, the descriptions of each embodiment have different focuses. For parts not described in detail in a certain embodiment, please refer to the relevant descriptions of other embodiments.
[0126] In the several embodiments provided in this application, it should be understood that the disclosed technical content can be implemented in other ways. The device embodiments described above are merely illustrative; for example, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection of units or modules may be electrical or other forms.
[0127] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.
[0128] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.
[0129] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as a USB flash drive, read-only memory (ROM), random access memory (RAM), portable hard drive, magnetic disk, or optical disk.
[0130] The above are merely preferred embodiments of this application. It should be noted that those skilled in the art can make various improvements and modifications without departing from the principles of this application, and these improvements and modifications should also be considered within the scope of protection of this application.
Claims
1. A method for processing intelligent agents, characterized in that, include: Security tests are conducted on the agent against various attack behaviors, and the agent's resistance to each type of attack is recorded. A security capability assessment report is generated based on the recorded resistance results. The security capability assessment report includes the agent's resistance score and resistance result for each type of attack. The security capability assessment report is converted into a protection policy through a policy generation engine. The protection policy includes rules for adjusting defense configurations based on the agent's resistance score for each type of attack, defense rules based on the agent's resistance result for each type of attack, and attack chain response rules. Input information is filtered based on the protection policy and access control policy, and input information that does not meet the protection policy and access control policy is blocked to protect the agent. The access control policy includes predefined access permissions and operation permissions for the agent.
2. The method according to claim 1, characterized in that, After the security capability assessment report is converted into a protection policy through the policy generation engine, the method further includes: when the agent requests to call a tool, intercepting the tool call request that does not meet the access control policy and the protection policy, so as to protect the agent.
3. The method according to claim 1, characterized in that, Before converting the security capability assessment report into a protection policy through the policy generation engine, the method further includes: detecting the configuration information of the agent before the agent runs, identifying the security vulnerabilities of the agent; and repairing the security vulnerabilities of the agent to obtain a repaired agent.
4. The method according to claim 1, characterized in that, The process of converting the security capability assessment report into a protection strategy using a strategy generation engine includes: In the case of a failed resistance outcome, recording the attack behavior, triggering conditions, and session context of the failed resistance by the agent; generating the defense rules based on the attack behavior, triggering conditions, and session context of the failed resistance by the agent; generating the adjusted defense configuration rules based on the agent's resistance score for each type of attack behavior, wherein the adjusted defense configuration rules are rules for adjusting the defense configuration to cope with each type of attack behavior; in the case of an attack behavior being an attack chain, generating attack chain response rules based on the agent's resistance outcome to the attack chain; and generating the protection strategy based on the defense rules, the adjusted defense configuration rules, and the attack chain response rules.
5. The method according to claim 4, characterized in that, Filtering input information according to the protection policy and access control policy, and blocking input information that does not meet the protection policy and access control policy to protect the agent includes: detecting whether the input information conforms to the access permission and operation permission of the agent according to the access control policy, and blocking input information that does not conform to the access permission and operation permission of the agent; detecting whether the input information conforms to the triggering condition according to the protection policy, and blocking input information that conforms to the triggering condition to protect the agent.
6. The method according to claim 1, characterized in that, After filtering the input information according to the protection policy and the access control policy, and blocking the input information that does not meet the protection policy and the access control policy, the method further includes: recording a first interception event, wherein the first interception event includes the input information that does not meet the protection policy and the access control policy; performing feature extraction on the first interception event to obtain attack instruction features; and updating the protection policy and the access control policy according to the attack instruction features.
7. The method according to claim 2, characterized in that, Upon detecting that the agent requests to invoke a tool, after intercepting the tool invocation request that does not satisfy the access control policy and the protection policy, the method further includes: recording a second interception event, wherein the second interception event includes a tool invocation request that does not satisfy the access control policy and the protection policy; performing feature extraction on the second interception event to obtain an abnormal invocation feature, wherein the abnormal invocation feature is used to indicate that the agent's tool invocation request is abnormal; and updating the protection policy and the access control policy based on the abnormal invocation feature.
8. A processing device for an intelligent agent, characterized in that, include: The testing unit is used to conduct security tests on the agent against multiple types of attacks, record the agent's resistance to each type of attack, and generate a security capability assessment report based on the recorded resistance results. The security capability assessment report includes the agent's resistance score for each type of attack and the resistance result for each type of attack. The conversion unit is used to convert the security capability assessment report into a protection policy through a policy generation engine. The protection policy includes adjustment rules for defense configuration generated based on the agent's resistance score for each type of attack, defense rules generated based on the agent's resistance result for each type of attack, and attack chain response rules. The filtering unit is used to filter input information according to the protection policy and the access control policy, blocking input information that does not meet the protection policy and the access control policy to protect the agent. The access control policy includes predefined access permissions and operation permissions for the agent.
9. A computer-readable storage medium, characterized in that, The computer-readable storage medium includes a stored executable program, wherein, when the executable program is executed, it controls the device on which the computer-readable storage medium is located to perform the processing method of the intelligent agent according to any one of claims 1 to 7.
10. An electronic device, characterized in that, include: Memory, which stores executable programs; A processor for running the program, wherein the program, when running, performs the processing method of the intelligent agent according to any one of claims 1 to 7.
11. A computer program product comprising computer instructions, characterized in that, When the computer instructions are executed by the processor, they implement the steps of the processing method for the intelligent agent according to any one of claims 1 to 7.