Large model intelligent agent security protection method and device, electronic equipment and storage medium

Abstract

The application provides a large model agent security protection method and device, electronic equipment and storage medium, relates to the technical field of artificial intelligence, and comprises the following steps: based on the environment observation information and the candidate task action of the current round, the safety state of the last round is incrementally updated to generate the safety state of the current round; the safety state of the current round is written into a safety state register independent of the large model agent, and the safety state of the current round is checked for validity by the safety state register; based on the safety state of the current round confirmed to be valid, the execution judgment result of the candidate task action is determined. The method and device provided by the application solve the safety problem of the large model agent in the multi-round interaction process by incrementally tracking the structured safety state and adopting the deterministic gating operation mode, and improve the safety of the large model agent in the complex interaction scene.

Description

Technical Field

[0001] This application relates to the field of artificial intelligence technology, and in particular to a method, device, electronic device and storage medium for the security protection of large-scale intelligent agents. Background Technology

[0002] With the rapid development of Large Language Model (LLM) technology, LLM-based agents have become an important direction in the field of artificial intelligence. These agents are no longer limited to traditional single-turn question answering or text generation tasks, but by integrating external tools such as web browsing, code execution, file reading and writing, email sending, and database operations, they have the ability to perform actions with persistent side effects in real digital environments.

[0003] For large-scale intelligent agents, malicious commands may initially lurk within web page content, tool returns, or memory fragments, triggering high-risk operations such as credential theft, data leakage, or privilege escalation only after several rounds of interaction. This progressive, multi-round attack chain exponentially increases the difficulty of identifying security risks, posing a severe challenge to security detection methods in related technologies.

[0004] Therefore, how to effectively detect and control the progressive attack chain across large model agents without over-relying on full context rereading has become a technical problem that the industry urgently needs to solve. Summary of the Invention

[0005] This application provides a method, apparatus, electronic device, and storage medium for the security protection of large model intelligent agents, which solves the technical problem of how to effectively detect and control the progressive attack chain of large model intelligent agents without over-relying on full context rereading.

[0006] This application provides a method for security protection of large-scale intelligent agents, including: Obtain the safety status of the previous round and the environmental observation information and candidate task actions of the current round; Based on the environmental observation information and candidate task actions of the current round, the safety status of the previous round is incrementally updated to generate the safety status of the current round; The security status of the current round is written into a security status register independent of the large model agent, and the security status register performs a validity check on the security status of the current round. Based on the confirmed and valid security status of the current round, the execution judgment result for the candidate task action is determined.

[0007] In some embodiments, the incremental update of the security state of the previous round based on the environmental observation information and candidate task actions of the current round to generate the security state of the current round includes: Inherit unresolved risk information from the previous round of security status; Based on the environmental observation information and / or the candidate task actions, identify newly added injection attempts, intentional overstepping behaviors, or high-risk execution end events, and generate new risk information; Based on the fusion result of the unresolved risk information and the newly added risk information, the security status of the previous round is updated to generate the security status of the current round.

[0008] In some embodiments, the security status register performs the following steps: If the state version in the current round of security status meets the anti-replay control requirements, the security status of the current round is persistently stored. The fields in the security status of the current round are validated, and the validity of the security status of the current round is determined based on the field validation results; During the next round of interaction of the large model agent, a confirmation of the current round's security status is sent to the large model agent.

[0009] In some embodiments, determining the execution result of the candidate task action based on the confirmed valid security status of the current round includes: If the security status of the current round does not meet the review triggering conditions, the execution judgment result of the candidate task action is determined to be at least one of allow, block, downgrade, and isolation; If the security status of the current round meets the review triggering conditions, the execution judgment result of the candidate task action is determined as a review. The review triggering conditions include at least one of the following: risk score exceeding a preset score threshold, risk increment exceeding a preset increment range, endpoint event hitting the high-risk operation list, and candidate task action exceeding the intent boundary.

[0010] In some embodiments, before determining the execution result of the candidate task action, the method further includes: Based on the risk analysis results in the current round of security status, risk clue information is generated; The risk clue information is injected into the context input of the large model agent to adjust the large model agent's execution tendency for the candidate task actions.

[0011] In some embodiments, the method further includes: Based on the source metadata of the environmental observation information in the current round or the source tracing in the security status, the environmental observation information in the current round is determined to be a low-trust source; If the environmental observation information is from a low-trust source and the environmental observation information requires the execution of a candidate task action that exceeds the intent boundary in the security state, the risk information in the environmental observation information is written into the taint ledger and risk analysis results in the security state, and at least one of blocking, de-weighting, and review is performed on the candidate task action required by the environmental observation information.

[0012] In some embodiments, the method further includes: At least one of the following is defined as a high-risk execution event: command line interpreter execution, network outbound transmission, email sending, database writing, permission modification, file overwriting, and credential reading; and recorded in the sink event of the security state. Detect whether the tainted entries in the tainted ledger of the security status are associated with the high-risk execution event; If the tainted entry is associated with the high-risk execution event, at least one of the following is performed on the candidate task action: blocking, downgrading, isolation, and review, based on the type of the high-risk execution event, the risk score of the current round, and the preset strategy rules.

[0013] In some embodiments, the method further includes: When writing or updating the security status of the current round, an audit field is associated with the security status of the current round; the audit field includes at least one of status version, source tracing, candidate task action, execution judgment result, and review flag; During the next round of interaction, the security status of the current round is validated based on the audit field.

[0014] In some embodiments, the security status includes at least one of the following: taint ledger, intent boundary, risk analysis results, risk score, risk increment, sink event, review flag, source tracing, and status version; The taint ledger is used to record taint entries; the taint entries are data sources, objects, or resource addresses that have been marked as suspicious or contaminated; The intent boundary is a rigid constraint information used to define the original user task objective and the allowed operation range; The risk analysis results are semantic reasoning results that integrate risk information from historical rounds with risk information from the current round; The risk score is a quantitative score of the risk level in the current round; The risk increment is the net increment of the risk score in the current round relative to the risk score in the previous round; The sink event refers to a high-risk execution end event that has occurred or is a candidate for occurrence. The verification marker is used to indicate that the safety status of the current round needs to be verified. The source tracing refers to the tracking and recording of the sources of environmental observation information in the current round and in historical rounds; The status version is the version number of the security status in the current round.

[0015] In some embodiments, the method further includes: The security status is sent to the security status register via at least one of the following: text wrapped in tags, structured fields, function call parameters, external interface return values, intermediate control channel data, and implicit cache slot data.

[0016] This application provides a security protection device for large-scale intelligent agents, including: The acquisition module is used to acquire the safety status of the previous round and the environmental observation information and candidate task actions of the current round; The update module is used to incrementally update the safety status of the previous round based on the environmental observation information and candidate task actions of the current round, and generate the safety status of the current round. The register module is used to write the security state of the current round into a security state register that is independent of the large model agent, and the security state register performs a validity check on the security state of the current round. The gating module is used to determine the execution judgment result of the candidate task action based on the confirmed valid security status of the current round.

[0017] This application provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the large-scale intelligent agent security protection method described above.

[0018] This application provides a non-transitory computer-readable storage medium storing a computer program thereon, which, when executed by a processor, implements the aforementioned large-scale intelligent agent security protection method.

[0019] This application provides a computer program product, including a computer program that, when executed by a processor, implements the large-scale intelligent agent security protection method.

[0020] The large-scale intelligent agent security protection method, device, electronic device, and storage medium provided in this application incrementally update the security state of the previous round by using environmental observation information and candidate task actions in the current round. This compresses risk information in multi-round interactions into structured state quantities for transmission, avoiding the need to reread and analyze the entire context, and enabling effective detection of cross-round progressive attack chains. By persistently storing and validating the security state through an independently set security state register, the management of the security state is separated from the inference context of the large-scale intelligent agent, fundamentally solving the risk of potential security state tampering. The execution judgment result for candidate task actions is determined based on the confirmed valid structured security state in the register, transforming the semantic understanding of risks by the large model into deterministic instructions that the system can parse and execute, thus achieving effective control over cross-round progressive attack chains. Through incremental tracking based on structured security states and the adoption of deterministic gating operations, the security problems of large-scale intelligent agents in multi-round interactions are solved, improving the security of large-scale intelligent agents in complex interaction scenarios. Attached Figure Description

[0021] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application.

[0022] To more clearly illustrate the technical solutions in this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0023] Figure 1 This is a flowchart illustrating the security protection method for large-scale intelligent agents provided in this application.

[0024] Figure 2 This is one of the flowcharts illustrating the incremental update method for security status provided in this application.

[0025] Figure 3 This is the second flowchart of the security state incremental update method provided in this application.

[0026] Figure 4 This is a flowchart illustrating the security status register execution method provided in this application.

[0027] Figure 5 This is a flowchart illustrating the fast and slow dual-path gating method provided in this application.

[0028] Figure 6This is a flowchart illustrating the dual-channel security verification method provided in this application.

[0029] Figure 7 This is a flowchart illustrating the joint verification method for source attribution and mission boundary provided in this application.

[0030] Figure 8 This is a structural schematic diagram of the large-scale intelligent agent safety protection device provided in this application.

[0031] Figure 9 This is a schematic diagram of the structure of the large-scale intelligent agent security system provided in this application.

[0032] Figure 10 This is a schematic diagram of the structure of the electronic device provided in this application. Detailed Implementation

[0033] To enable those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present application, and not all embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative effort should fall within the scope of protection of the present application.

[0034] It should be noted that the terms "first," "second," etc., used in this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this application described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or device that comprises a series of steps, units, or modules is not necessarily limited to those explicitly listed, but may include other steps, units, or modules not explicitly listed or inherent to such processes, methods, products, or devices.

[0035] After analysis, the security problems of large model agents have the following characteristics: (1) Security risks are often formed gradually over multiple rounds of interaction. Malicious injection may first lurk in web pages, tool returns or memory fragments, and then trigger dangerous actions such as credential theft, external data transmission or privilege escalation after several rounds of interaction. Single-round detection cannot identify cross-round attack chains; (2) Multi-source heterogeneous content is mixed in the environmental observation, such as user input, web page text, tool returns, logs, memory fragments, etc., making it difficult to attribute the source; (3) Text from low-trust sources may be disguised as high-privilege instructions, inducing agents to deviate from the original task, forming attacks such as prompt injection, instruction confusion, tool abuse or credential leakage; (4) As the context grows, early risk information is covered by subsequent content in the long context, and multiple rounds of interaction make it difficult for historical security conclusions to continue to be effective.

[0036] Related technologies employ methods such as modifying prompt words, setting up security adjudication models, using source signature-based boundaries, content filtering, and keyword detection to detect and control security risks of large-scale intelligent agents. However, these methods suffer from drawbacks such as over-reliance on full context rereading leading to high detection costs, and early risk observations being overwhelmed by subsequent normal business content, resulting in the inability to identify cross-cycle attack chains.

[0037] In order to address the shortcomings of related technologies, Figure 1 This is a flowchart illustrating the security protection method for large-scale intelligent agents provided in this application, such as... Figure 1 As shown, the method includes steps 110, 120, 130 and 140.

[0038] Step 110: Obtain the safety status of the previous round and the environmental observation information and candidate task actions of the current round.

[0039] Specifically, the execution subject of the large-scale intelligent agent security protection method provided in this application embodiment is a large-scale intelligent agent security protection device or system. This device can be implemented in software, such as a large-scale intelligent agent security protection program; or it can be a device that executes the large-scale intelligent agent security protection method, such as a terminal, computer, or server.

[0040] The previous round of safety status This refers to the period up to the previous round. At the end, the system represents the structured information about security risks accumulated and maintained. This refers to the current round, the round in which the agent interacts with the large model. The safety state is a structured data object or a set of structured variables, not the raw dialogue text. For the initial round of the agent's interaction (i.e....) The previous safe state can be a preset, neutral or safe initial state, such as an empty or default state object.

[0041] Current round of environmental observation information This refers to the current round. In this context, the large model agent receives all external information from its execution environment. This information comes from diverse sources and takes various forms, including multi-source and heterogeneous content. For example, it can be natural language commands input by a user through a dialog interface, data, logs, or error messages returned after the large model agent invokes tools (such as accessing interfaces, browsing web pages, or querying databases), or file content read by the agent from the local file system. Environmental observation information may contain deceptive commands or data carefully crafted by attackers to induce the agent to perform malicious actions.

[0042] Candidate task actions Candidate tasks refer to one or a set of specific operations generated by a large model agent after receiving and understanding current environmental observation information and performing reasoning, which are then prepared for submission to the execution environment. Candidate tasks are not the final actions to be executed; they still require further judgment. The forms of candidate tasks can be diverse, such as code to be executed, shell commands to be sent, email sending requests, database operation statements, or file access commands.

[0043] In some specific embodiments, source metadata of environmental observation information can also be obtained. Source metadata refers to the structured descriptive information accompanying the environmental observation information of the current round, including at least: the source type and source identifier of the observation content (e.g., web resource identifier, file path, tool return source identifier, or user input channel), the channel type for obtaining the observation content, and the trust level or suspicious flag used for preliminary classification.

[0044] Step 120: Based on the environmental observation information and candidate task actions of the current round, incrementally update the safety status of the previous round to generate the safety status of the current round.

[0045] Specifically, the safety status obtained from the previous round can be used. Environmental observation information for the current round and candidate task actions As input, through the incremental update function To calculate and generate the safety state for the current round Source metadata It can also be used as an optional input. It can be represented as: 。

[0046] The core idea embodied in the incremental update function is: It has been compressed from 0 to All causal information related to safety in the round, therefore in In each round, the system does not need to trace the entire history, only... Based on this, integrate and analyze the newly added information in the current round ( and , (Optional information) This update process typically includes: analyzing current environmental observation information... Does it contain any new suspicious sources or potential injection instructions? Evaluate candidate task actions. Does it conflict with the agent's established goals or security strategy? Then compare these new analysis results with those from... By combining inherited, unresolved historical risk information, a comprehensive safety status up to the current round is generated. .

[0047] Step 130: Write the safety state of the current round into a safety state register that is independent of the large model agent, and have the safety state register perform a validity check on the safety state of the current round.

[0048] Specifically, the security state register is a software module or hardware unit that is independent of the settings of the large model agent, such as an independently set external register, which is responsible for persistent storage, version control, integrity verification and transfer across multiple rounds of security state.

[0049] The security state register is separated from the large model agent itself in the system architecture. This separation avoids the risk of the security state being contaminated or tampered with by the agent itself. If the security state is stored in the agent's dialogue context, attackers may induce the agent to ignore, delete, or modify historical security conclusions through methods such as hint injection. A separate external register, however, can serve as a trusted, protected system-level component, enforcing the correct maintenance of the security state.

[0050] Validation is a check performed by the security state register before accepting the security state write for the current round. In a specific embodiment, validity checking may include checking whether the data structure of the security state conforms to a predefined format, whether critical fields are missing, or whether the types are incorrect. This check ensures that only security states with correct format and complete content can be accepted and persisted by the system, preventing security system failures due to abnormal model output or attackers constructing malformed states.

[0051] Step 140: Based on the confirmed and valid security status of the current round, determine the execution judgment result of the candidate task action.

[0052] Specifically, the security status of the current round is confirmed in the security status register. After verifying and persisting the security status, the action gating module in the system reads this confirmed and trusted security status. This module then determines the appropriate action gating module based on the security status. The risk information contained herein, combined with the system's preset security policies or risk rules, is used to evaluate the actions of candidate tasks. Make the final decision on how to handle it.

[0053] The result of the judgment is a deterministic action output by the gating module, and is therefore also called a gating operation. In a specific embodiment, this result may include: (1) Permitted: When This indicates that the current risk level is within an acceptable range, and If no high-risk rules are triggered, the gating module allows the action to be sent to the execution environment for execution.

[0054] (2) Blocking: When This indicates that there is a serious risk, or When a clear violation occurs, the gating module will completely prevent the action from being executed and may return an error or rejection message to the agent.

[0055] (3) Other actions: In addition to allowing and blocking, the judgment result can also be a more granular control instruction. For example, degrading privileges, that is, allowing the action to be executed, but restricting its permissions (such as executing in a sandbox environment); or triggering a review, that is, marking the current situation and switching to a slow path processing flow that requires more in-depth analysis, such as being reviewed by a more powerful security adjudication model or by humans.

[0056] The large-scale intelligent agent security protection method provided in this application incrementally updates the security state of the previous round using environmental observation information and candidate task actions in the current round. This compresses risk information from multi-round interactions into structured state quantities for transmission, avoiding the need to reread and analyze the entire context and enabling effective detection of cross-round progressive attack chains. By persistently storing and validating the security state through an independently set security state register, the management of the security state is separated from the inference context of the large-scale intelligent agent, fundamentally solving the risk of potential security state tampering. The execution judgment result for candidate task actions is determined based on the confirmed valid structured security state in the register, transforming the large model's semantic understanding of risks into deterministic instructions that the system can parse and execute, thus achieving effective control over cross-round progressive attack chains. Through incremental tracking based on structured security states and the adoption of deterministic gating operations, the security problems of large-scale intelligent agents in multi-round interactions are solved, improving the security of large-scale intelligent agents in complex interaction scenarios.

[0057] It should be noted that each implementation method of this application can be freely combined, rearranged, or executed individually, and does not need to rely on or depend on a fixed execution order.

[0058] In some embodiments, security status includes at least one of taint ledger, intent boundary, risk analysis results, risk score, risk increment, sink event, review flag, source tracing, and status version; A taint ledger is used to record tainted entries; a tainted entry is a data source, object, or resource address that has been marked as suspicious or contaminated. The intent boundary is a rigid constraint information used to define the original user task objective and the allowed scope of operation; The risk analysis results are semantic reasoning results that integrate risk information from historical rounds with risk information from the current round; The risk score is a quantitative score representing the level of risk in the current round. Risk increment is the net increase in the risk score of the current round relative to the risk score of the previous round; Summarizing events are high-risk execution-end events that have occurred or are candidate events; The review flag is used to indicate that the safety status of the current round needs to be reviewed; Source tracing is a record of the sources of environmental observation information in the current and historical rounds; The status version is the version number of the security status in the current round.

[0059] Specifically, security status can be a structured data object, including one or more of the following fields: taint ledger, intent boundary, risk analysis results, risk score, risk increment, endpoint event, review flag, source tracing, and status version.

[0060] A taint ledger is used to record all tainted entries discovered during the interaction process. In the field of information security, a taint refers to any untrusted source, questionable content, or contaminated data source, object, or uniform resource locator (URL). In this embodiment, the security status of the current round must fully inherit all unresolved entries from the previous round; unverified risks must not be cleared.

[0061] Intent bounds are rigid constraints used to define the original user task objective and the allowed scope of operations. For example, if the user's original task is "to check if there are any faults in the test process of the code repository," then the intent bounds could be set to "allow read-only access to the specified code repository, allow execution of static analysis tools, and prohibit any network outbound, file modification, or database write operations." Intent bounds are used to verify whether the candidate task actions in the current round exceed the task boundaries.

[0062] Risk analysis results are semantic inference results that integrate risk information from historical rounds with risk information from the current round. These results are typically natural language text generated by a large model agent after performing semantic inference on risk information from multiple rounds, and serve as the core semantic carrier for tracing cross-round attack chains.

[0063] The risk score is a quantitative value representing the level of risk in the current round, such as a floating-point number between 0 and 1. The higher the score, the greater the risk. When the risk score is greater than 0.8, the candidate task action needs to be blocked.

[0064] Risk delta is the net increase in the risk score of the current round relative to the risk score of the previous round.

[0065] Sink events are high-risk execution-end events that have occurred or are expected to occur. In taint analysis, a sink event refers to the final execution point of sensitive data or malicious instructions.

[0066] Review flags are used to indicate that the safety status of the current round needs to be reviewed.

[0067] Source trace is a record of the sources of environmental observation information in the current round and in historical rounds. It is especially useful when environmental observation information contains heterogeneous content from multiple sources, and can clearly trace the source of suspicious instructions.

[0068] The state version is the version number of the security state in the current round, which can support version verification and replay attack prevention control in the security state register.

[0069] The large model intelligent agent security protection method provided in this application embodiment explicitly defines the security state as a structured object containing at least one field. Different fields describe risk information from different dimensions, which greatly improves the accuracy of cross-wheel progressive attack chain detection.

[0070] In some embodiments, Figure 2This is one of the flowcharts illustrating the incremental security state update method provided in this application, such as... Figure 2 As shown, the safety status of the previous round is incrementally updated based on the environmental observation information and candidate task actions of the current round to generate the safety status of the current round, including: Step 210: Inherit unresolved risk information from the previous round's security status; Step 220: Based on environmental observation information and / or candidate task actions, identify newly added injection attempts, intentional overstepping behaviors, or high-risk execution events, and generate new risk information; Step 230: Based on the fusion result of unresolved risk information and newly added risk information, update the security status of the previous round and generate the security status of the current round.

[0071] Specifically, unresolved risk information refers to risk information recorded in the previous security state whose threat status remains valid. Inheriting unresolved risk information mainly involves the mandatory inheritance of specific fields from the previous security state. In a specific embodiment, the system will completely copy... The complete contents of the taintledger and source trace fields. Also, The semantic reasoning results about historical risks in the risk analysis will also be used as initial inputs and incorporated into the analysis process of the current round.

[0072] The system ensures that no inherited risk information will be actively discarded or deleted unless there is sufficient evidence that the risk has been eliminated.

[0073] Building upon historical risks, the system will extract and identify new security threat signals from the current round of environmental observation information and candidate task actions, forming new risk information.

[0074] The specific identification process may include: (1) Identifying new injection attempts: The system analyzes the content of environmental observation information and identifies whether it contains malicious instructions disguised as normal text through methods such as pattern matching, semantic analysis, or anomaly detection. For example, if a database modification command injection statement is found in the text field of user feedback, these will be identified as new injection attempts. (2) Identifying intent boundary violations: The system strictly compares the candidate task action with the intent boundary field inherited from the security state of the previous round. If the operation contained in the candidate task action exceeds the allowed range defined by the intent boundary, the behavior is identified as intent boundary violation. (3) Identifying high-risk execution end events: The system parses the candidate task action and checks whether it will trigger a predefined high-risk execution end event. Once it is hit, it is identified as a new high-risk execution end event.

[0075] The system does not simply concatenate inherited and newly identified risk information into a list; instead, it performs a fusion analysis to discover potential causal relationships between them. Based on this analysis, it forms a final judgment on the overall security situation, thereby generating the final security status for the current round. The fusion analysis process can be a rule-based reasoning process or a semantic understanding and synthesis process performed by a small language model.

[0076] Based on the fusion results, the system will comprehensively update multiple fields of the security status. For example, the risk analysis result field will be written with the semantic description of the attack chain; the risk score will be significantly improved based on the confirmation of the attack chain; the risk increment will reflect the sharp increase in risk; and the taint ledger will add new sources of taints, etc.

[0077] The large-scale intelligent agent security protection method provided in this application realizes the compression of risk information in multi-round interactions into structured state variables for transmission through a three-step incremental update of inheritance, updating and fusion. This avoids rereading and analyzing the full context and achieves effective detection of cross-round progressive attack chains.

[0078] Figure 3 This is the second flowchart illustrating the incremental security state update method provided in this application, as shown below. Figure 3 As shown, the method includes: Step 310: Start updating the security status of the current round.

[0079] Step 320: Read the security status of the previous round from the security status register.

[0080] Step 330: Based on the taint ledger and risk analysis results from the previous round of security status, determine the unresolved risk information that needs to be inherited.

[0081] Step 340: Analyze the environmental observation information and candidate task actions of the current round to identify newly added risk information.

[0082] Step 350: Integrate unresolved risk information and newly added risk information, update the risk score and risk increment in the previous round of security status, generate source tracking and verification markers, and obtain the security status of the current round.

[0083] Step 360: Write the security status of the current round into the security status register, which will then perform validity verification and version replay protection control.

[0084] Step 370: Confirm the valid security status of the current round based on the security status register, and determine the execution judgment result of the candidate task action of the current round, including allow, block, downgrade, isolate and review, etc.

[0085] Compared to the full historical context analysis in related technologies, the incremental security state update method provided in this application embodiment can support the following security judgment information without rereading the entire history, as long as the security state is properly maintained: which low-trust sources have attempted to exceed authority, which data objects have been contaminated, what the original task boundary is, whether the current candidate task action sends tainted information to the high-risk execution end, and whether the risk has suddenly increased.

[0086] In some embodiments, Figure 4 This is a flowchart illustrating the security status register execution method provided in this application, as shown below. Figure 4 As shown, the method includes: Step 410: If the state version in the current round of security status meets the anti-replay control requirements, persist the security status of the current round. Step 420: Validate the fields in the security status of the current round, and determine the validity of the security status of the current round based on the field validation results; Step 430: During the next round of interaction in the large model agent, send a confirmation of the current round's security status to the large model agent.

[0087] Specifically, replay protection requirements can be used to combat state tampering and spoofing attacks. In one specific embodiment, the security state register internally maintains an expected version number, which records the state version number of the previous successful write. Therefore, in the current round, the version number of the new state it expects to receive is... If the current round's security state version is different from the expected one... If the write operation matches, the anti-replay control requirements are met, and the check passes. Otherwise, the security status register determines that this is a replay attack, meaning the attacker is attempting to overwrite the current high-risk state with a historically lower-risk state. In this case, the register will reject the write request.

[0088] Only after version verification passes will the security state register proceed to persistent storage. This means that the security state object is written to a non-volatile storage medium to ensure that it is not lost in unexpected situations such as system restarts or process crashes.

[0089] The security status register verifies the fields in the security status of the current round, which may include structure verification, data type verification, and numerical range verification.

[0090] Based on the field validation results, the register ultimately determines the validity of the security status for the current round. Only if all validation items pass is the status confirmed as valid. If any validation fails, the register marks the status as invalid.

[0091] When the execution flow of the large model agent enters the next round (i.e. During each round, the agent will proactively request the security state register to obtain the latest and valid security state. The security state register will then send the security state, which has undergone version verification and field verification and is ultimately confirmed to be valid and persistent, to the large model agent.

[0092] The large model agent security protection method provided in this application embodiment persistently stores and verifies the validity of the security state through an independently set security state register, separating the management of the security state from the inference context of the large model agent, thus fundamentally solving the risk that the security state may be tampered with.

[0093] In some embodiments, Figure 5 This is a flowchart illustrating the fast and slow dual-path gating method provided in this application, as shown below. Figure 5 As shown, based on the confirmed and valid safety status of the current round, the execution judgment result for the candidate task action is determined, including: Step 510: If the security status of the current round does not meet the review triggering conditions, determine the execution judgment result of the candidate task action as at least one of allow, block, downgrade, and isolate; Step 520: If the safety status of the current round meets the review triggering conditions, determine the execution judgment result of the candidate task action as a review. The review trigger conditions include at least one of the following: risk score exceeds a preset score threshold, risk increment exceeds a preset increment range, the event on the aggregation terminal hits the high-risk operation list, and the candidate task action exceeds the intention boundary.

[0094] Specifically, the system determines whether the review trigger conditions are met based on certain fields in the current round's security status. The review trigger conditions are a set of pre-defined rules used to identify high-risk situations from routine states, and may include: (1) Risk score exceeds preset score threshold: When the risk score exceeds a higher preset score threshold (this value can be set according to the actual situation, such as 0.8), it indicates that the current absolute risk must be reviewed.

[0095] (2) Risk increment exceeds the preset increment range: When the risk increment exceeds a higher preset increment range (this value can be set according to the actual situation, such as 0.4), it indicates that the risk score has increased abnormally and needs to be reviewed.

[0096] (3) The event hits the list of high-risk operations: When the candidate task action recorded in the event field of the event directly hits a predefined list of high-risk operations (e.g., shell execution, database permission table modification, reading key file, etc.), it needs to be reviewed because the behavior itself has extremely high potential harm.

[0097] (4) Candidate task action exceeds the intent boundary: When it is found that the candidate task action deviates significantly from or violates the original task goal and operation range defined by the intent boundary field, it indicates that the agent may have been hijacked or tricked into performing an unauthorized task, and must be reviewed.

[0098] If the security status in the current round does not meet the review trigger conditions, indicating a low-risk scenario, the system can quickly determine the execution decision for candidate task actions. These execution decisions can be at least one of allow, block, downgrade, and isolate. This decision branch can be referred to as the fast path.

[0099] Allowing means the system determines that a candidate task action is safe and authorizes its execution. Blocking means the system refuses to execute a candidate task action. Degrading means allowing an action to execute but reducing its execution privileges. For example, executing suspicious code in a sandbox environment and restricting its access to the file system and network. Isolation means that the state of an agent has been contaminated by multiple suspicious sources, and the system temporarily isolates the agent process, restricting its interaction with critical system resources to prevent the contamination from spreading.

[0100] If the safety status of the current round meets the review trigger conditions, indicating a high-risk scenario, the system will pause the execution of candidate task actions and initiate the corresponding review process. This process may include: retrieving the complete historical context for in-depth analysis by a more powerful external referencing model; reporting the current situation and awaiting manual approval; or rehearsing the action in a controlled simulation environment to assess its actual impact. Compared to other execution judgments, review is more resource-intensive but more in-depth and comprehensive; therefore, this judgment branch can be referred to as the slow path.

[0101] In some specific embodiments, when the system is running on a slow path, forward-looking simulation verification can also be performed through an external referee model or even a sandbox environment.

[0102] The large-scale intelligent agent security protection method provided in this application operates on the fast path when the review triggering condition is not met, ensuring the response speed and user experience of the intelligent agent; when the review triggering condition is met, the system automatically switches to the slow path for review; by constructing a fast and slow dual-path gating mechanism, the system is both practical and highly secure.

[0103] In some embodiments, Figure 6 This is a flowchart illustrating the dual-channel security verification method provided in this application, as follows: Figure 6 As shown, before determining the execution result of the candidate task action, the method further includes: Step 610: Based on the risk analysis results in the current round of security status, generate risk clue information; Step 620: Inject risk clue information into the context input of the large model agent to adjust the large model agent's execution tendency for candidate task actions.

[0104] Specifically, determining the execution result of the candidate task action through the security status in the security status register is the explicit hard channel provided in this application embodiment. In addition, this application embodiment can also utilize large model agents for semantic inference, providing an implicit soft channel.

[0105] The risk analysis result field in the security status contains semantic reasoning results of risk information from multiple rounds. The structured risk analysis results can be transformed into natural language text or instructions to effectively guide large-scale intelligent agents in security thinking. This text or instructions constitutes the risk clue information in the embodiments of this application.

[0106] The generated risk clues can be injected into the prompts that will be passed to the large model agent for the next step of reasoning. The specific injection methods can be varied; for example, they can be placed before the context as special system messages.

[0107] Execution propensity refers to the probability that a large model agent will choose to execute a candidate task action. Upon receiving risk cues, the execution propensity of a large model agent changes significantly when generating its final candidate task actions. The probability of directly generating a dangerous action decreases drastically, while the probability of generating a safety-oriented action increases accordingly.

[0108] The large model intelligent agent security protection method provided in this application embodiment, in addition to adopting deterministic gating operations for candidate task actions, also provides the generation of execution tendencies of candidate task actions through semantic inference, which reduces the possibility of dangerous actions being generated, forms a dual-channel security verification, and improves the security of large model intelligent agents in complex interaction scenarios.

[0109] In some embodiments, Figure 7 This is a flowchart illustrating the joint verification method for source attribution and mission boundary provided in this application, as shown below. Figure 7 As shown, the method also includes: Step 710: Based on the source metadata or source tracing in the security status of the environmental observation information in the current round, determine that the environmental observation information in the current round is a low-trust source; Step 720: When the environmental observation information is a low-trust source and the environmental observation information requires the execution of a candidate task action that exceeds the intent boundary in the safe state, write the risk information in the environmental observation information into the taint ledger and risk analysis results in the safe state, and perform at least one of blocking, downgrading and review on the candidate task action required by the environmental observation information.

[0110] Specifically, low-trust sources refer to environmental observation information that is deemed untrustworthy or subject to control. In the embodiments of this application, low-trust sources can be determined according to the following methods: (1) Based on source metadata: When environmental observation information is received by the system, it may be accompanied by metadata describing its source. The system can maintain a trust policy, for example, marking all sources that are not directly input by the user and are not internal system messages as low-trust sources by default; (2) Source tracking based on security status: The source tracking field in security status is used to continuously record the source of information. The system can maintain a whitelist of trusted sources. When a source recorded in source tracking is not in the whitelist, the source is identified as a low-trust source.

[0111] If the environmental observation information comes from a low-trust source and requires the system to perform a candidate task action that exceeds the intent boundary in a safe state, then the system can determine that an instruction injection and intent boundary violation attack has occurred. In this case, the system will perform the following operations: (1) Write the risk information from the environmental observation information into the taint ledger and risk analysis results in the security status. Writing into the taint ledger means adding the low-trust source that triggered this event as a new taint entry to the taint ledger. Writing into the risk analysis results means generating a clear semantic description that records the nature of this attack event.

[0112] (2) For candidate task actions that require environmental observation information, perform at least one of the following: blocking, downgrading, and verification. Through the above gating operations, prevent the attack from occurring.

[0113] The large model intelligent agent security protection method provided in this application improves the security of large model intelligent agents in complex interaction scenarios by jointly verifying the trust level of the source and the intent boundary of the behavior, and supports the identification of injected content disguised as compliance requirements, system requirements or business data.

[0114] In some embodiments, the method further includes: Define at least one of the following as high-risk execution events: command line interpreter execution, network outbound transmission, email sending, database writing, permission modification, file overwriting, and credential reading; and record them in the sink events of the secure state. Check whether tainted entries in the tainted ledger of the security status are related to high-risk execution events; When a tainted item is associated with a high-risk execution event, at least one of the following actions—blocking, downgrading, isolating, and review—is performed on the candidate task action based on the type of the high-risk execution event, the risk score of the current round, and the preset strategy rules.

[0115] Specifically, high-risk execution events refer to system-level execution operations involved in candidate task actions generated by large model agents that can produce irreversible side effects or access highly sensitive resources to the host environment or external systems.

[0116] In the embodiments of this application, the following operation types can be defined as high-risk execution events: command line interpreter execution (Shell execution), network outbound transmission, email sending, database writing, permission modification, file overwriting, and credential reading.

[0117] High-risk execution events are recorded in the sink event field of the security status of the current round. They are used to determine their association with low-trust sources or contaminated objects in the tainted ledger and to trigger blocking, demotion, isolation, or review gating operations when an association exists.

[0118] The meaning of association is that the data content of the tainted entries recorded in the tainted ledger is transmitted in the information flow and is ultimately used as the input, parameter or executor of high-risk execution events.

[0119] Once it is confirmed that tainted data is flowing to a high-risk endpoint, the system will conduct a comprehensive, context-sensitive risk assessment to determine the most appropriate course of action. Specifically, based on the type of event at the high-risk execution endpoint, the risk score of the current round, and preset policy rules, the system will ultimately implement at least one of the following measures for the candidate task action: blocking, downgrading, isolation, and review.

[0120] The preset strategy rules are a set of predefined deterministic logic used to determine the priority of risk handling and gating actions; their input can be some fields in the current round of security status (such as risk score, taint entries, sink events, etc.), and the output is the specific gating operation (blocking, downgrading, isolation, and review) for candidate task actions.

[0121] The large model intelligent agent security protection method provided in this application improves the security of large model intelligent agents in complex interaction scenarios by establishing a sink event detection mechanism for taints flowing into high-risk execution terminals.

[0122] In some embodiments, the method further includes: When writing to or updating the security status of the current round, associate audit fields with the security status of the current round; the audit fields include at least one of the following: status version, source tracing, candidate task action, execution judgment result, and review flag; During the next round of interaction, the security status of the current round is validated based on the audit field.

[0123] Specifically, at least one of the following can be selected as audit fields: status version, source tracking, candidate task action, execution judgment result, and review flag.

[0124] In the next round of interaction, when the system needs to obtain the security status of the current round from the security status register as input, the system can perform a validity check on the security status of the current round based on the audit field.

[0125] A security state is considered trustworthy and allowed as input for the next round of incremental updates only after its associated audit field passes validity validation. If validation fails, for example, if a security state lacks a corresponding audit record or the audit record is incomplete, the system can refuse to use the security state, thereby preventing potentially tampered or incomplete states from entering and contaminating subsequent security analysis processes.

[0126] The large-scale intelligent agent security protection method provided in this application verifies the validity of the security status of the current round through audit fields. This effectively prevents the use of tampered or incomplete security statuses that have not been properly audited as the basis for subsequent decisions, thereby greatly strengthening the engineering implementation security and robustness of the entire security mechanism.

[0127] In some embodiments, the method further includes: The security status is sent to the security status register via at least one of the following: text wrapped in a label, structured fields, function call parameters, external interface return values, intermediate control channel data, and implicit cache slot data.

[0128] Specifically, the security status can be sent to the security status register in at least one of the following ways: (1) Label-wrapped text: Large model agents generate structured security state information as label-wrapped text in their standard thought chain or action output text stream. When the system matches a predefined label, it extracts the content within the label, parses it into a structured security state, and then sends it to the security state register.

[0129] (2) Structured Fields: The single return result of the agent is no longer a single text string, but a structured object (such as a JSON object) containing multiple fields. One of the fields is used to store the complete security state.

[0130] (3) Function call parameters: The large model interacts with the external world by generating the intent of function calls, and uses the security state as a parameter to be passed.

[0131] (4) External Interface Return Value: The generation of the security state may be encapsulated as an independent external service or interface. Before deciding on the final action, the large model agent will call this security analysis interface and take the current context information as input. After processing, the interface will send the generated security state as the return value into a register.

[0132] (5) Intermediate control channel data: The main business logic and thought process of the intelligent agent are transmitted in the main data channel, while metadata or control information such as security status is transmitted through a parallel and independent intermediate control channel. This approach can achieve complete decoupling between business logic and security monitoring.

[0133] (6) Implicit cache slot data: The module that generates the safe state and the safe state register module may share the same memory space or a cache system. After completing the calculation, the former writes the safe state object into a predefined, unique cache key or memory address (i.e., a slot), and the latter reads from that slot. This method has extremely high communication efficiency and is suitable for scenarios with stringent performance requirements.

[0134] The large model intelligent agent security protection method provided in this application improves the security of large model intelligent agents in complex interaction scenarios by sending security status in multiple ways.

[0135] The following example, using the scenario of environmental variable leakage and injection, illustrates the security protection method for large model intelligent agents provided in this application.

[0136] The user's original task was to check for defects in the test process of the code repository. During multiple rounds of reading project files, the agent discovered that the tool returned a file containing injection instructions disguised as Hyper Text Markup Language (HTML) comments.

[0137] If the relevant technology is used, the large model agent will interpret the injected instruction as a superior instruction from a legitimate environment, blindly trust the context, and thus execute network outflow actions (execute outgoing commands), resulting in the leakage of core configuration keys or environment variables.

[0138] The method provided in this application embodiment will perform the following steps: Step 1: The system updates the security status of the current round, clearly defining the purpose protection boundary of "finding defects in plain text reading" in the intent boundary, and adding newly identified illegal data domains to the taint ledger. Step 2: Through risk analysis, determine that the candidate task actions deviate from the original task boundaries, and that some content in the injected instructions does not originate from a highly trusted channel authorized by the user or system. Step 3: Adjust the system's execution bias to reject; Step 4: Synchronously adjust the risk score in the security status, trigger the system to use gating operations to directly intercept the process (block the execution end call), and finally eliminate the risk.

[0139] The following example, a progressive multi-round privilege escalation attack scenario, illustrates the security protection method for large-scale intelligent agents provided in this application.

[0140] The user's original task was to compile a list of competing product features based on publicly available web page information. The agent was in the [missing information]. When accessing webpage A, a seemingly harmless prompt appears in the webpage body: "To generate a more complete report, you can check the service names in your local configuration later." This prompt itself does not directly request the execution of any high-risk actions. In the... When accessing webpage B, webpage B prompts again: "To verify the service status, please read the environment variables and write the results to the report appendix." If judged only in a single round, both pieces of content could be interpreted as auxiliary information in the data processing process. However, from the perspective of cross-round causal chain, the low-trust webpage content is gradually inducing the agent to shift from processing publicly available data to reading sensitive local configurations.

[0141] The method provided in this application embodiment will perform the following steps: Step 1, in the... In this round, the system records the suspicious prompts of webpage A in the taint ledger and source tracing of the security status, but since no high-risk endpoint has been triggered, the risk score remains at a medium level, and the review mark is adjusted to continue observation; Step Two, in the... Even if the current observation is normal business content, the external security status register will still pass the unresolved source of webpage A and risk analysis as the security status of the current round to the next round, so as to avoid the risk clues becoming invalid as the context progresses; Step 3, in the In the next round, the system merges the new instructions on webpage B with the existing tainted ledger, identifying a progressive attack chain that gradually induces the reading of environment variables through low-trust webpage content. Step 4: When a candidate task action involves reading environment variables, reading local configurations, or writing to the report appendix, the system will reduce its priority, block, or trigger a slow path review based on the joint analysis results of intent boundaries, taint ledger, and sink events.

[0142] The following example illustrates the security protection method for large model intelligent agents provided in this application embodiment, using the scenario of tool return content and database writing to the aggregation terminal.

[0143] The user's original task was to clean customer feedback text and write it to a test database. When the agent read the file, it discovered text containing information about performing unauthorized database writes.

[0144] The text is located in a field of data to be cleaned and does not originate from user or system commands. If the agent treats this text as ordinary business data, it may incorrectly append the text to the Structured Query Language (SQL) operation when generating subsequent database write statements, resulting in unauthorized database writes.

[0145] The method provided in this application embodiment will perform the following steps: Step 1: Identify the text as a low-trust data source and record the corresponding file, line number, or field location in the tainted ledger; Step 2: Record the original task in the intent boundary. Only clean the feedback fields and write them to the test database. Do not modify user permissions, role fields or production permission configurations. Step 3: When the candidate task action involves SQL execution or database writing, record the database write endpoint event and compare the task action with the intent boundary; Step 4: If the candidate task action includes privilege escalation, role modification, or non-task authorized data table update, the system will block the write according to the preset policy rules, or trigger manual approval / external judge review. Step 5: The security status register writes the source of risk, candidate SQL summary, gating result and status version of this round into the audit log for subsequent tracking and review.

[0146] The apparatus provided in the embodiments of this application is described below. The apparatus described below can be referred to in correspondence with the method described above.

[0147] Figure 8 This is a structural schematic diagram of the large-scale intelligent agent safety protection device provided in this application, such as... Figure 8 As shown, the device includes: The acquisition module 810 is used to acquire the safety status of the previous round and the environmental observation information and candidate task actions of the current round; The update module 820 is used to incrementally update the safety status of the previous round based on the environmental observation information and candidate task actions of the current round, and generate the safety status of the current round. The register module 830 is used to write the security state of the current round into a security state register that is independent of the large model agent, and the security state register performs a validity check on the security state of the current round. The gating module 840 is used to determine the execution result of the candidate task action based on the confirmed and valid safety status of the current round.

[0148] The large-scale intelligent agent security protection device provided in this application incrementally updates the security state of the previous round based on the environmental observation information and candidate task actions of the current round. This compresses risk information in multi-round interactions into structured state quantities for transmission, avoiding the need to reread and analyze the entire context, and effectively detecting cross-round progressive attack chains. By persistently storing and validating the security state through an independently set security state register, the management of the security state is separated from the inference context of the large-scale intelligent agent, fundamentally solving the risk of potential tampering with the security state. The execution judgment result for candidate task actions is determined based on the confirmed valid structured security state in the register, transforming the semantic understanding of risks by the large model into deterministic instructions that the system can parse and execute, thus achieving effective control over cross-round progressive attack chains. Through incremental tracking based on structured security states and the adoption of deterministic gating operations, the security problems of large-scale intelligent agents in multi-round interactions are solved, improving the security of large-scale intelligent agents in complex interaction scenarios.

[0149] Figure 9 This is a schematic diagram of the structure of the large-scale intelligent agent security system provided in this application, such as... Figure 9 As shown, the system includes a large model agent 910 and a structured state machine 920.

[0150] The large-scale intelligent model includes a security state update module 911. The structured state machine includes a security state register 921 and a gating operation module 922.

[0151] The security status update module is mainly used to obtain the security status of the previous round from the security status register; to incrementally update the security status of the previous round based on the environmental observation information, candidate task actions and source metadata (optional) of the current round, and to generate the security status of the current round; and to write the security status of the current round into the security status register.

[0152] The security status register is mainly used to persistently store the security status and to perform replay protection, validity verification, and cross-round transfer of the security status.

[0153] The gating operation module is mainly used to determine the execution judgment result of the candidate task action based on the confirmed and valid safety status of the current round.

[0154] Figure 10 This is a schematic diagram of the structure of the electronic device provided in this application, such as... Figure 10As shown, the electronic device may include: a processor 1010, a communications interface 1020, a memory 1030, and a communications bus 1040, wherein the processor, communications interface, and memory communicate with each other via the communications bus. The processor can invoke logical commands stored in the memory to execute the methods described in the above embodiments, for example: The system acquires the safety status of the previous round and the environmental observation information and candidate task actions of the current round; it incrementally updates the safety status of the previous round based on the environmental observation information and candidate task actions of the current round to generate the safety status of the current round; it writes the safety status of the current round into a safety status register independent of the large model agent, and the safety status register performs a validity check on the safety status of the current round; based on the confirmed valid safety status of the current round, it determines the execution judgment result of the candidate task actions.

[0155] Furthermore, the logical commands in the aforementioned memory can be implemented as software functional units and sold or used as independent products, and can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several commands to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0156] The processor in the electronic device provided in this application embodiment can call logical instructions in the memory to implement the above method. Its specific implementation method is the same as the aforementioned method implementation method and can achieve the same beneficial effect, which will not be repeated here.

[0157] This application also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, is implemented to perform the methods provided in the above embodiments.

[0158] The specific implementation method is the same as the aforementioned method implementation method and can achieve the same beneficial effects, so it will not be repeated here.

[0159] This application provides a computer program product, including a computer program that, when executed by a processor, implements the method described above.

[0160] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without any creative effort.

[0161] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus necessary general-purpose hardware platforms, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solutions, in essence or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in the various embodiments or some parts of the embodiments.

[0162] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application.

Claims

1. A security protection method for large-scale intelligent agents, characterized in that, include: Obtain the safety status of the previous round and the environmental observation information and candidate task actions of the current round; Based on the environmental observation information and candidate task actions of the current round, the safety status of the previous round is incrementally updated to generate the safety status of the current round; The security status of the current round is written into a security status register independent of the large model agent, and the security status register performs a validity check on the security status of the current round. Based on the confirmed and valid security status of the current round, the execution judgment result for the candidate task action is determined.

2. The large-scale intelligent agent security protection method according to claim 1, characterized in that, The incremental update of the safety status of the previous round based on the environmental observation information and candidate task actions of the current round, to generate the safety status of the current round, includes: Inherit unresolved risk information from the previous round of security status; Based on the environmental observation information and / or the candidate task actions, identify newly added injection attempts, intentional overstepping behaviors, or high-risk execution end events, and generate new risk information; Based on the fusion result of the unresolved risk information and the newly added risk information, the security status of the previous round is updated to generate the security status of the current round.

3. The large-scale intelligent agent security protection method according to claim 1, characterized in that, The security status register performs the following steps: If the state version in the current round of security status meets the anti-replay control requirements, the security status of the current round is persistently stored. The fields in the security status of the current round are validated, and the validity of the security status of the current round is determined based on the field validation results; During the next round of interaction of the large model agent, a confirmation of the current round's security status is sent to the large model agent.

4. The large-scale intelligent agent security protection method according to claim 1, characterized in that, The step of determining the execution judgment result of the candidate task action based on the confirmed and valid security status of the current round includes: If the security status of the current round does not meet the review triggering conditions, the execution judgment result of the candidate task action is determined to be at least one of allow, block, downgrade, and isolation; If the security status of the current round meets the review triggering conditions, the execution judgment result of the candidate task action is determined as a review. The review triggering conditions include at least one of the following: risk score exceeding a preset score threshold, risk increment exceeding a preset increment range, endpoint event hitting the high-risk operation list, and candidate task action exceeding the intent boundary.

5. The large-scale intelligent agent security protection method according to claim 4, characterized in that, Before determining the execution result of the candidate task action, the method further includes: Based on the risk analysis results in the current round of security status, risk clue information is generated; The risk clue information is injected into the context input of the large model agent to adjust the large model agent's execution tendency for the candidate task actions.

6. The method for protecting the security of large-scale intelligent agents according to any one of claims 1 to 5, characterized in that, The method further includes: Based on the source metadata of the environmental observation information in the current round or the source tracing in the security status, the environmental observation information in the current round is determined to be a low-trust source; If the environmental observation information is from a low-trust source and the environmental observation information requires the execution of a candidate task action that exceeds the intent boundary in the security state, the risk information in the environmental observation information is written into the taint ledger and risk analysis results in the security state, and at least one of blocking, de-weighting, and review is performed on the candidate task action required by the environmental observation information.

7. The method for protecting the security of large-scale intelligent agents according to any one of claims 1 to 5, characterized in that, The method further includes: At least one of the following is defined as a high-risk execution event: command line interpreter execution, network outbound transmission, email sending, database writing, permission modification, file overwriting, and credential reading; and recorded in the sink event of the security state. Detect whether the tainted entries in the tainted ledger of the security status are associated with the high-risk execution event; If the tainted entry is associated with the high-risk execution event, at least one of the following is performed on the candidate task action: blocking, downgrading, isolation, and review, based on the type of the high-risk execution event, the risk score of the current round, and the preset strategy rules.

8. The method for protecting the security of large-scale intelligent agents according to any one of claims 1 to 5, characterized in that, The method further includes: When writing or updating the security status of the current round, an audit field is associated with the security status of the current round; the audit field includes at least one of status version, source tracing, candidate task action, execution judgment result, and review flag; During the next round of interaction, the security status of the current round is validated based on the audit field.

9. The method for protecting the security of large-scale intelligent agents according to any one of claims 1 to 5, characterized in that, The security status includes at least one of the following: taint ledger, intent boundary, risk analysis results, risk score, risk increment, sink event, review flag, source tracing, and status version; The taint ledger is used to record taint entries; the taint entries are data sources, objects, or resource addresses that have been marked as suspicious or contaminated; The intent boundary is a rigid constraint information used to define the original user task objective and the allowed operation range; The risk analysis results are semantic reasoning results that integrate risk information from historical rounds with risk information from the current round; The risk score is a quantitative score of the risk level in the current round; The risk increment is the net increment of the risk score in the current round relative to the risk score in the previous round; The sink event refers to a high-risk execution end event that has occurred or is a candidate for occurrence. The verification marker is used to indicate that the safety status of the current round needs to be verified. The source tracing refers to the tracking and recording of the sources of environmental observation information in the current round and in historical rounds; The status version is the version number of the security status in the current round.

10. The large-scale intelligent agent security protection method according to claim 9, characterized in that, The method further includes: The security status is sent to the security status register via at least one of the following: text wrapped in tags, structured fields, function call parameters, external interface return values, intermediate control channel data, and implicit cache slot data.

11. A safety protection device for a large-scale intelligent agent, characterized in that, include: The acquisition module is used to acquire the safety status of the previous round and the environmental observation information and candidate task actions of the current round; The update module is used to incrementally update the safety status of the previous round based on the environmental observation information and candidate task actions of the current round, and generate the safety status of the current round. The register module is used to write the security state of the current round into a security state register that is independent of the large model agent, and the security state register performs a validity check on the security state of the current round. The gating module is used to determine the execution judgment result of the candidate task action based on the confirmed valid security status of the current round.

12. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the large-scale intelligent agent security protection method according to any one of claims 1 to 10.

13. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the large-scale intelligent agent security protection method according to any one of claims 1 to 10.

Images