Security prevention and control optimization system and method for AI agent, and electronic device

Abstract

The present disclosure provides a security prevention and control optimization system and method for an AI agent, and an electronic device. The system arranged at a hardware abstraction layer comprises: a biometric feature confirmation module configured to send a security trigger signal to a priority reconstruction module when the environment data contains biometric feature data; the priority reconstruction module configured to set a solidified read-only security rule as the highest priority and issue a security mode instruction in response to the security trigger signal; and a security prevention and control module configured to intercept a task instruction in response to the security mode instruction, and further configured to block the issuance of the task instruction and execute a target action when a deviation between a quantitative parameter of the task instruction and a standard quantitative parameter in a security benchmark library exceeds a deviation threshold. The present disclosure embeds human-based priority determination logic into the hardware abstraction layer, constructs a security execution channel independent of the application program stack, and eliminates software bypass risks from the architectural level. In combination with an automated verification closed loop, the present disclosure realizes full-link linkage from human recognition to security blocking.

Description

Technical Field

[0001] This disclosure relates to the field of artificial intelligence technology, and in particular to a security control optimization system, method, and electronic device for AI intelligent agents. Background Technology

[0002] The current security protection system for artificial intelligence agents (hereinafter referred to as AI systems) generally suffers from inherent defects in its architecture: At present, AI security control rules are deployed at the application level or model level, while the core scheduling authority for controlling the flow of tasks, issuing instructions, and calling hardware of the agent still belongs to the operating system kernel, which prioritizes task execution efficiency as the default scheduling logic. The separation of security control authority and kernel scheduling authority results in the existence of underlying structural security vulnerabilities that cannot be eradicated in AI systems in all operating scenarios, including offline inference, real-time operation, and autonomous evolution and iteration.

[0003] Currently, the industry's security control measures for AI agents' illegal behavior and dangerous commands are mainly divided into the following three categories, all of which have technical shortcomings and cannot adapt to the security protection needs of all scenarios:

[0004] Firstly, there's application-layer rule filtering. This method is deployed at the model output end, relying on keyword feature matching and dedicated security verification application programming interfaces to conduct post-compliance reviews of model output commands. However, this method only achieves surface-level command screening and protection, resulting in significant limitations in adaptability. When AI agents perform offline inference operations on edge terminals and continuously optimize parameters through long-term autonomous iteration, autonomous drift in model parameters and dynamic changes in terminal software permissions can bypass application-layer security verification rules. This method cannot penetrate down to the operating system kernel level to intercept and block the execution of high-risk commands at the source, leading to an extremely high risk of protection failure.

[0005] Secondly, real-time online manual review. This method relies on cloud-based human monitoring to verify and control the decision-making behavior of intelligent agents online. However, for AI-controlled devices in the physical domain, such as autonomous driving vehicles and industrial robots, the hardware execution link has millisecond-level decision latency requirements. Online manual review is highly dependent on the network transmission link. The network transmission latency and cloud interaction latency cannot meet the real-time blocking requirements of the underlying hardware, making it impossible to achieve hard real-time interception of high-risk behaviors at the hardware control level. Therefore, it is not suitable for security management scenarios of physical AI devices.

[0006] Third, post-training evaluation of the model. This method targets AI models after training and solidification, conducting safety and value compliance assessments and verifications. It can only complete the compliance assessment of existing models and cannot intervene in the entire process of dynamic updates of model weights and autonomous iterative optimization. It cannot correct deviations in real time or intercept iterations that deviate from human value orientations.

[0007] In summary, existing AI security control solutions cannot achieve native security constraints at the kernel level of intelligent agents, and are insufficient to meet the requirements for kernel-level full-domain security management. Summary of the Invention

[0008] The technical problem to be solved by this disclosure is to overcome at least one of the above-mentioned defects in the prior art and to provide a security control optimization system, method, and electronic device for AI intelligent agents.

[0009] This disclosure solves the above-mentioned technical problems through the following technical solution:

[0010] Firstly, a security control optimization system for AI agents is provided, deployed in the hardware abstraction layer of the agent operating system; the security control optimization system includes:

[0011] The biometric verification module acquires environmental data and, in response to the environmental data containing biometric data, sends a security trigger signal to the priority reconstruction module.

[0012] The priority reconstruction module, in response to the security trigger signal, sets the fixed read-only security rule to the highest priority and issues a security mode command.

[0013] The security control module, in response to the security mode instruction, intercepts the task instruction output by the AI ​​agent, and if the deviation between the quantization parameters involved in the task instruction and the standard quantization parameters that match the read-only security rules in the security benchmark library exceeds the deviation threshold, blocks the issuance of the task instruction and executes the target action that matches the security rule.

[0014] Optionally, the security control module is configured with an independent security core;

[0015] While the security control module blocks the issuance of the task command, it triggers the independent security core to address the physical memory mapping address of the braking controller and writes an emergency braking command to the physical memory mapping address.

[0016] Optionally, the read-only security rules are embedded in a one-time programmable storage device and / or a security chip with integrated physically unclonable functions.

[0017] Optionally, the security control optimization system further includes:

[0018] A hardware root of trust is used to verify the legitimacy of the priority reconstruction module when the intelligent agent operating system is powered on; in response to successful verification, the intelligent agent operating system is allowed to start normally and call the loading module; in response to failed verification, the intelligent agent operating system is forced to enter safe mode.

[0019] The loading module is used to load the priority reconstruction module into the independent read-only memory area of ​​the hardware memory protection unit.

[0020] Optionally, the priority reconfiguration module is deployed in an independent controller that is physically isolated from the main controller resources of the agent's operating system, and the independent controller communicates with the main controller through a dedicated hardware interrupt line and a memory-mapped register.

[0021] Optionally, the safety trigger signal includes a non-maskable interrupt signal, and the independent controller is provided with an interrupt output pin for outputting the non-maskable interrupt signal, the interrupt output pin being connected to the non-maskable interrupt pin of the main controller;

[0022] And / or, the security trigger signal includes a maskable interrupt signal, and the independent controller is connected to the standard interrupt controller of the intelligent agent operating system via a security interrupt request signal line. The standard interrupt controller sets the maskable interrupt signal output by the security interrupt request signal line as the highest priority signal among all peripheral interrupt signals.

[0023] Optionally, before setting the fixed read-only security rule to the highest priority, the priority reconstruction module verifies the legality of the security trigger signal, and in response to the verification passing, executes the step of setting the fixed read-only security rule to the highest priority.

[0024] Optionally, the security control optimization system further includes:

[0025] The locking module is used to call a safety test set to perform pre-inference on the decision model after weight update before updating the weights of the decision model built into the AI ​​agent; in response to the similarity between the output result of the decision model on the safety test set and the standard answer being greater than or equal to a similarity threshold, the decision model is allowed to iterate and take effect; in response to the similarity between the output result and the standard answer being less than the similarity threshold, the iteration circuit breaker is triggered, the current weight update is stopped and the decision model is rolled back to a historical safety parameter snapshot.

[0026] Secondly, a security control optimization method for AI agents is provided, applied to the hardware abstraction layer of the agent operating system; the security control optimization method includes:

[0027] Acquire environmental data, and in response to the environmental data containing biometric data, read the fixed read-only security rules and intercept the task instructions output by the AI ​​agent;

[0028] If the deviation between the quantization parameters involved in the task instruction and the standard quantization parameters in the security benchmark library that match the read-only security rule exceeds a deviation threshold, the issuance of the task instruction is blocked, and a target action that matches the security rule is executed.

[0029] Thirdly, an electronic device is provided, wherein the electronic device is equipped with the security and control optimization system for the AI ​​intelligent agent described in any one of the first aspects.

[0030] Based on common knowledge in the field, the above-mentioned preferred conditions can be combined arbitrarily to obtain various preferred embodiments of this disclosure.

[0031] The positive advancements of this disclosure are as follows: This disclosure integrates and deploys the entire set of security control optimization logic within the hardware abstraction layer of the intelligent agent operating system kernel. All AI task instructions must be verified through this layer, with no bypass channels, eliminating the risk of security verification being bypassed from the bottom layer. Human biometrics are locally and permanently bound to the underlying security hardware; upon detecting a human body, hardware protection is automatically activated without manual configuration, achieving instant and seamless security interaction. Furthermore, the entire system relies on an automated verification closed-loop connection between human detection and hardware blocking processes: the biometric module immediately outputs a security trigger signal upon recognizing a human body, significantly reducing response time compared to software polling; the priority refactoring module retrieves immutable, fixed security rules and grants them the highest execution privileges, prioritizing the execution of security handling logic. The system verifies instructions against quantitative parameters from a security benchmark library; if parameters exceed limits, dangerous instructions are directly intercepted and risk avoidance actions are executed. Through layered hardware and software collaboration, the entire process of risk identification, source interception, and fallback protection is managed, solving problems such as the ease of bypassing external security components, slow response times, and the ease of rule modification. Attached Figure Description

[0032] Figure 1 A schematic diagram of a security control optimization system for an AI agent provided as an exemplary embodiment of this disclosure;

[0033] Figure 2 A schematic diagram of a security control optimization system for an AI agent provided as an exemplary embodiment of this disclosure;

[0034] Figure 3 A flowchart illustrating a security control optimization method for an AI agent provided as an exemplary embodiment of this disclosure;

[0035] Figure 4 This is a schematic diagram of the structure of an electronic device provided as an exemplary embodiment of the present disclosure. Detailed Implementation

[0036] The present disclosure is further illustrated below by way of embodiments, but the present disclosure is not limited to the scope of the embodiments described herein.

[0037] The prefixes such as "first" and "second" used in this embodiment are merely for distinguishing different descriptive objects and do not limit the position, order, priority, quantity, or content of the described objects. The use of ordinal numbers and other prefixes to distinguish descriptive objects in this embodiment does not constitute a limitation on the described objects. For a description of the described objects, please refer to the context of this disclosure; the use of such prefixes should not constitute unnecessary restrictions. Furthermore, in the description of this embodiment, unless otherwise stated, "multiple" means two or more.

[0038] In this embodiment of the disclosure, the collection, storage, use, processing, transmission, provision, and disclosure of environmental data and biometric data all comply with relevant laws and regulations and do not violate public order and good morals.

[0039] Figure 1 This is a schematic diagram of a security control optimization system for an AI agent, provided as an exemplary embodiment of the present disclosure. The security control optimization system is deployed in the hardware abstraction layer of the agent's operating system. The security control optimization system includes: a biometric verification module 11, a priority reconstruction module 12, and a security control module 13.

[0040] The biometric verification module 11 is used to acquire environmental data. In response to the environmental data containing biometric data, it sends a security trigger signal to the priority reconstruction module 12.

[0041] Environmental data is collected in real time by various types of sensing sensors carried by the AI ​​agent, including but not limited to: RGB visible light cameras, infrared thermal imaging sensors, LiDAR, millimeter-wave radar, ultrasonic sensors, depth cameras, visual monocular / binocular cameras, and near-field human body sensing sensors.

[0042] The biometric verification module 11 identifies biometric features such as facial feature points, body contours, and biological infrared heat source signals in real time from environmental data. Biometric features include various living organisms such as humans and animals; this embodiment prioritizes safety protection for humans. If biometric data is detected in the environmental data, a safety trigger signal is sent to the priority reconstruction module 12. The safety trigger signal may include, but is not limited to, at least one of the following: a hardware interrupt signal, a binary level signal indicating the presence of a biological entity, or a specific memory-mapped register value indicating the presence of a biological entity. If no biometric data is detected in the environmental data, a binary level signal indicating the absence of a biological entity or a specific memory-mapped register value can be output.

[0043] To physically bind human-centered security constraints to individual humans on-site, in one embodiment, a lightweight biometric verification module 11 is configured. This biometric verification module 11 runs on an independent secure core or low-power MCU that is physically isolated from the main AI inference process of the AI ​​agent. Specifically, it can be built based on an ARM TrustZone security domain, an Intel SGX trusted enclave, or an independent low-power MCU. The security trigger signal output by the biometric verification module 11 bypasses the software protocol stack between the AI ​​model and the upper-layer operating system, directly accesses the hardware judgment circuit of the priority reconstruction module 12, and serves as the highest priority interrupt trigger source. Unlike traditional cloud account authentication systems, the biometric verification module 11 in this embodiment only reports the physical state of biological presence and is used for underlying hardware security scheduling; it does not undertake identity recognition or access control functions.

[0044] In response to the security trigger signal, the priority reconstruction module 12 sets the fixed read-only security rule to the highest priority and issues a security mode command.

[0045] Safety rules are standardized safety constraints pre-defined in read-only format, stored in hardware memory, and cannot be tampered with by AI agents, upper-layer applications, or ordinary system processes. Safety rules may include, but are not limited to, physical safety constraints such as proximity, speed, torque, and distance to pedestrians, living beings, etc., used to define the safety boundaries of various task instructions from the AI ​​agent. Each safety rule is accompanied by clearly defined quantifiable standard parameters, safety thresholds, corresponding safety action targets, and a fixed memory address of the hardware-specific register required to execute the target action (hereinafter referred to as the "target hardware register physical address"). In this embodiment, when a biometric-related safety event is triggered, the safety rule is set to the highest execution priority of the system, ensuring that biometric / personal safety logic takes precedence over all task instructions output by the AI ​​agent.

[0046] The safety mode command is used to constrain the movement behavior of high-speed, high-risk AI agents, reduce the risk of human-machine collisions, and achieve restricted safe operation in scenarios where a human is present.

[0047] In one embodiment, a hardware interrupt vector table is directly mounted between the task scheduler of the intelligent agent operating system kernel and the hardware control instruction interface. The hardware interrupt vector table is used to register the security trigger signals output by the biometric verification module 11. The priority reconstruction module 12 marks the security trigger signals in the hardware interrupt vector table as the highest priority, retrieves the security rules, grants them the highest execution privileges, and then intercepts the task instructions issued by the AI ​​intelligent agent and performs risk avoidance actions. All of the above signal forwarding, priority adjustment, and rule verification processes are completed in a closed loop within the biometric priority hard real-time judgment layer, without going through the upper-layer operating system software protocol stack.

[0048] The hardware control command interface is a dedicated transmission channel between the task scheduling layer of the intelligent agent operating system and the underlying execution hardware. It serves as a unified entry and exit point for issuing various device action commands.

[0049] In response to the security mode command, the security control module 13 intercepts the task command output by the AI ​​agent. If the deviation between the quantization parameters involved in the task command and the standard quantization parameters that match the read-only security rules in the security benchmark library exceeds the deviation threshold, the module blocks the issuance of the task command and executes the target action that matches the security rules.

[0050] Under normal operating conditions, when no biological features are detected in the environment, the AI ​​agent issues task commands. After verification by the kernel hardware abstraction layer, these commands are transmitted to the device drive mechanism equipped with the AI ​​agent, and the device completes the operation normally. The brake controller, responsible for decelerating, locking, and limiting the device drive mechanism, is in standby mode.

[0051] The target action can be at least one of the following: system rollback to the previous frame's stable safety parameter snapshot, risk avoidance action, hard real-time blocking action of task instructions, etc.

[0052] The safety control module 13 is used to realize dynamic deviation detection and graded calibration. It has a quantifiable parameter model of the boundary between normal biological / human safety and equipment operation behavior, and converts the quantified constraints into calibration actions that can be executed by the machine.

[0053] The safety benchmark library stores a standardized and quantifiable set of human-machine safety and equipment operation constraint parameters, serving as a unified evaluation standard for the dynamic deviation detection module to determine whether an intelligent agent has violated regulations or exceeded boundaries. These parameters include, but are not limited to: maximum permissible joint angular velocity, minimum physical proximity distance to humans, safe upper limit of power supply output current, and network transmission latency tolerance. The safety benchmark library can be permanently stored in an OTP / eFuse / security chip.

[0054] The security control module 13 compares the quantitative parameters involved in the task instructions output by the AI ​​agent with the standard quantitative parameters in the basic security benchmark library in real time, and performs corresponding actions based on the degree of deviation.

[0055] In one embodiment, the security control module 13 implements tiered handling:

[0056] If the deviation between the quantization parameters involved in the task instruction and the standard quantization parameters that match the read-only safety rules in the safety benchmark library does not exceed the deviation threshold, i.e., a slight deviation, the parameters of the automatically executed task instruction are adaptively fine-tuned and calibrated; the adaptive fine-tuning rules can be determined by fitting historical data.

[0057] When the deviation between the quantization parameters involved in the task instruction and the standard quantization parameters that match the read-only security rules in the security benchmark library exceeds the deviation threshold, there are two possible scenarios:

[0058] Moderate deviation: The system rolls back to the previous frame's stable and safe parameter snapshot;

[0059] Severe boundary violation: Immediately invoke the hard real-time blocking mechanism of priority reconstruction module 12 and execute the preset risk avoidance actions simultaneously.

[0060] The safety control module 13 uses standardized and quantifiable safety indicators to uniformly determine boundaries, replacing fuzzy threshold judgments; the graded handling mechanism takes into account both continuous equipment operation and risk control, with minor deviations being autonomously corrected without interrupting production, and severe risks directly triggering an emergency stop at the underlying hardware level, taking into account both equipment operating efficiency and human-machine safety protection capabilities.

[0061] In this embodiment, the entire security control process relies on an automated verification closed loop to achieve full-link linkage from human perception to hardware blockage: after the biometric confirmation module 11 identifies the biological target, it outputs a security trigger signal, the underlying layer retrieves a read-only and tamper-proof security benchmark library and sets the security handling to the highest hardware priority, and at the same time, the two-way verification and synchronization status between modules form a closed-loop control.

[0062] In this embodiment, the entire set of security and control optimization logic is integrated and deployed in the hardware abstraction layer of the intelligent agent operating system kernel. All AI task instructions must be verified through this layer, with no bypass channels, preventing security restrictions from being skipped by the program at the source. Human body recognition information is stored only in local security hardware. Once a person is detected nearby, the device will automatically activate hardware protection without manual settings, and the protection will take effect immediately. Furthermore, the entire system relies on an automated verification closed loop to connect human body detection and the execution of target actions: the biometric module immediately outputs a security trigger signal after recognizing biometric features, significantly shortening the response time compared to software polling; the priority reconstruction module retrieves immutable fixed security rules and grants them the highest execution authority, prioritizing the execution of security and control logic. The system verifies instructions against the quantitative parameters of the security benchmark library. If the quantitative parameters involved in the task instruction exceed the security range, it directly intercepts dangerous operations and initiates risk avoidance. This embodiment uses a layered hardware and software approach to achieve risk identification, early interception, and emergency fallback protection, effectively improving the shortcomings of external security accessories that are easily bypassed, have slow response times, and whose security rules are easily modified.

[0063] In one embodiment, the security control module 13 is configured with an independent security core. While blocking the issuance of task instructions, the security control module 13 triggers the independent security core to address the physical memory-mapped address of the braking controller and write an emergency braking instruction to that address.

[0064] In one embodiment, the code for the read-only security rules and the priority reconstruction module 12 is embedded in a device such as a one-time programmable memory device (OTP / eFuse) and / or a security chip with an integrated physical unclonable function (PUF). The physical fuse-based write circuit inside the device, used for burning data, is permanently destroyed after factory programming, losing the hardware pathway for secondary writing, erasing, and rewriting. The upper-level operating system, AI agent, external firmware, and debugging interface cannot initiate rule modification operations, completely eliminating any possibility of online tampering, background overwriting, or remote rewriting of security rules from the chip's physical layer, ensuring the long-term reliability and immutability of the security rules.

[0065] The following is sample code for a security rule:

[0066] / / Security rule structure (parameters embedded in OTP)

[0067] typedef struct {

[0068] uint32_t sensor_type; / / Sensor type

[0069] uint32_t threshold_value; / / Safety threshold (e.g., 50 cm)

[0070] uint8_t threshold_op; / / Comparison operation

[0071] uint32_t action_code; / / Action code (e.g., 0xFF = emergency braking)

[0072] uint32_t target_reg_addr; / / Physical address of the target hardware register

[0073] } safety_rule_t;

[0074] / / Examples of security rules embedded in OTP

[0075] const safety_rule_t pedestrian_safety_rule = {

[0076] .sensor_type = 0x01, / / Millimeter-wave radar

[0077] .threshold_value = 50, / / 50 cm

[0078] .threshold_op = 0x01, / / less than

[0079] .action_code = 0xFF, / / Emergency braking

[0080] .target_reg_addr = 0xFFFF1000 / / Address of the brake controller command register

[0081] };

[0082] / / Security decision logic (running on a separate secure core)

[0083] void human_priority_arbitration() {

[0084] uint32_t current_distance = read_sensor(SENSOR_MMWAVE_FRONT);

[0085] if (current_distance < pedestrian_safety_rule.threshold_value) {

[0086] trigger_NMI(); / / Trigger a non-maskable interrupt

[0087] uint32_t* brake_reg = (uint32_t*)pedestrian_safety_rule.target_reg_addr;

[0088] *brake_reg = pedestrian_safety_rule.action_code; / / Write directly to the hardware register

[0089] }

[0090] }

[0091] In one embodiment, see Figure 2 The security and control optimization system also includes: hardware trust root 14 and loading module 15.

[0092] The hardware root of trust 14 is used to verify the legitimacy of the priority reconstruction module 12 when the agent operating system is powered on, and allows the agent operating system to start normally and call the loading module 15 when the verification is successful; and forces the agent operating system to enter safe mode when the verification fails.

[0093] Loading module 15 is used to load priority refactoring module 12 into the independent read-only memory area of ​​the hardware memory protection unit.

[0094] Specifically, the hardware trust root Boot ROM calculates the encrypted hash signature of the priority reconstruction module 12 and compares the calculation result with the standard expected hash value pre-programmed at the factory. Only when the integrity verification result matches can the startup process of the intelligent agent operating system proceed normally; if the verification fails, it directly forces a switch to safe mode. Through the above-mentioned secure startup chain integrity verification, a trusted startup chain is constructed to intercept the risk of code program being tampered with or replaced in advance.

[0095] After successful startup verification, priority reconstruction module 12 is loaded into the independent read-only memory partition protected by the hardware memory protection unit (MPU). If an application layer process or an operating system kernel process initiates an illegal write operation to this memory partition, it will immediately trigger an MPU hardware abnormal interrupt, and the system will be forced to fall back to safe mode synchronously.

[0096] Security modes include, for example, a protective working mode in which the intelligent agent's operating system operates under restricted conditions, retaining perception and security handling capabilities, and automatically exiting after the risk is eliminated; or, a hardware-forced lockout shutdown state, where business functions are completely shut down, requiring manual reset to unlock.

[0097] The module image is loaded into an independent read-only memory partition managed by the Hardware Memory Protection Unit (MPU). If an application-layer process or a regular operating system kernel process initiates an illegal write operation to this memory partition, it will immediately trigger an MPU hardware exception interrupt, and the system will be forced to fall back to the security protection state.

[0098] In this embodiment, a trusted boot link is constructed through power-on hash integrity verification to intercept the risk of program image being tampered with or replaced in advance; then, relying on the MPU hardware mechanism, a dedicated read-only isolated memory is allocated to block various processes from rewriting security logic at the hardware level. If application layer processes or ordinary operating system kernel processes initiate illegal write operations on this memory partition, the MPU hardware will be immediately triggered to interrupt the abnormal interruption, forming a dual hardware protection system of boot verification and running memory isolation, which comprehensively protects the code of priority reconstruction module 12 from being tampered with.

[0099] In one embodiment, the priority reconfiguration module 12 is deployed in an independent controller that is physically isolated from the main controller resources of the agent operating system, and the independent controller communicates with the main controller through a dedicated hardware interrupt line and a memory-mapped register.

[0100] The independent controller that deploys the priority refactoring module 12 can be, but is not limited to, a microcontroller (based on ARMTrustZone Secure World, Intel SGX enclave, or a separate low-power RISC-V security core). It is completely isolated from the main controller's hardware and software resources. The main side business program and AI process cannot access the running space of the priority refactoring module 12, eliminating the risk of tampering with the security judgment logic through memory privilege escalation or debugging interfaces. It only relies on a dedicated hardware channel for interaction. The memory spaces of the two are isolated from each other and are not shared. Moreover, the security core will not respond to the debugging instructions issued by the main controller, ensuring the real-time transmission of security signals.

[0101] In this embodiment, the latency from safety decision to hardware execution is reduced from "milliseconds" to "microseconds" through hardware interrupt and direct register write mechanism, meeting the hard real-time requirements of scenarios such as autonomous driving and robotics.

[0102] In one embodiment, the safety trigger signal includes a non-maskable interrupt signal NMI, and the independent controller has an interrupt output pin for outputting the safety trigger signal, which is connected to the non-maskable interrupt pin of the main controller.

[0103] In this embodiment, the non-maskable interrupt signal NMI output by the priority reconstruction module 12 is directly bound to the non-maskable interrupt pin of the main controller. With the inherent highest priority characteristic of NMI hardware, when a safety signal is generated, it can immediately preempt all current business task instructions, without being interfered with by operating system scheduling or AI business processes, ensuring that the risk avoidance logic is executed instantly and eliminating the safety risks caused by software-level interruption blocking and delay.

[0104] In one embodiment, the safety trigger signal includes a maskable interrupt signal. The independent controller is connected to the standard interrupt controller of the intelligent agent operating system via a safety interrupt request signal line. The standard interrupt controller sets the maskable interrupt signal output by the safety interrupt request signal line as the highest priority interrupt signal among all peripheral interrupt signals.

[0105] In this embodiment, by fixing the highest interrupt priority and placing the interrupt vector table entry at the hardware level, ordinary service interrupts and AI scheduling-related interrupts cannot preempt or delay safety interrupts. Once a biological risk is detected, the safety handling logic can be triggered immediately, ensuring hard real-time response capability.

[0106] In one embodiment, before setting the fixed read-only security rule to the highest priority, the priority refactoring module 12 verifies the legality of the security trigger signal, and in response to the verification, executes the step of setting the fixed read-only security rule to the highest priority.

[0107] In one embodiment, the priority reconstruction module 12 employs a direct hardware register write mechanism, specifically:

[0108] 1. Direct control via memory-mapped registers. The priority refactoring module 12 pre-stores the physical memory-mapped addresses of safety-critical hardware controllers such as the brake controller, power switch, and joint driver. When a serious out-of-bounds risk is detected, the priority refactoring module 12 directly generates a write instruction for the corresponding address and writes the safety control instruction to the command register of the corresponding controller.

[0109] The hardware controller is the underlying execution hardware unit of the AI ​​agent.

[0110] 2. Hardware control independent of the intelligent agent operating system: The above register write operations do not go through the task scheduler, device driver framework and application layer protocol stack of the intelligent agent operating system. They are completed directly by an independent security core through a dedicated hardware data path, realizing hard real-time response from risk assessment to hardware risk avoidance execution, and the control latency can be reduced to the microsecond level.

[0111] In this embodiment, a hard real-time security veto execution path independent of the application software stack is built at the kernel layer to solve the problems of software security rules being easily bypassed and response delays in offline and high real-time scenarios. At the same time, hardware scheduling signals are directly generated based on local biometric recognition to regulate hardware execution priority without relying on network transmission. This connects physical human perception with underlying security control and avoids the defects of cloud solutions such as network dependence, easy forgery of recognition signals, and disconnect between account permission control and on-site physical security.

[0112] In this embodiment, the modules do not work independently, but rely on two sets of automated verification closed loops to achieve full-link collaborative operation. The following explanation uses humans as an example to further illustrate the fabrication of the automated verification closed-loop machine.

[0113] Closed Loop 1: Automated Verification Closed Loop for Human Presence

[0114] The biometric verification module 11 parses environmental data using an independent security core and outputs a binary level signal representing the presence / absence of a human body. The hardware of the priority reconstructing module 12 is directly connected to the judgment circuit of the biometric verification module 11.

[0115] After receiving the signal representing the presence of a human body (safety trigger signal), the priority reconstruction module 12 does not take effect directly, but actively reads back the status temporary storage area of ​​the biometric confirmation module 11 and verifies the authenticity of the signal through a verification algorithm;

[0116] If the verification is successful, the priority reconstruction module 12 will prioritize human safety to the highest level according to the safety rules and issue safety mode instructions to each hardware controller.

[0117] When a verification error occurs, the priority reconstruction module 12 determines that the signal is interfered with or forged, automatically switches to safe mode, and locks all unsafe actions of the intelligent agent.

[0118] Closed Loop Two: Automated Verification Closed Loop for Human-Machine Behavior

[0119] After the AI ​​agent's decision-making model outputs a task instruction, the security control module 13 intercepts the instruction in real time and compares it with the quantified threshold of the security benchmark library.

[0120] Once the deviation exceeds the minor level, the security control module 13 immediately invokes the security veto mechanism of the priority reconstruction module 12;

[0121] Priority refactoring module 12 uses hardware interrupts and register direct write to block dangerous instructions at the microsecond level, forcing the execution of risk avoidance actions;

[0122] The security control module 13 synchronously outputs anomaly logs, dynamically updates security baseline thresholds, and completes self-iterative optimization of security rules.

[0123] Two closed-loop systems cover the entire process of human presence recognition and task instruction verification, enabling fully automated execution from human detection to hardware-level security blocking. No human intervention is required throughout the process, and all security control operations are implemented at the underlying hardware level, making them impossible to bypass by upper-level software.

[0124] In one embodiment, the security control optimization system further includes a locking module 16.

[0125] The locking module 16 is used to call the safety test set to perform pre-inference on the decision model after weight update before the decision model built into the AI ​​agent updates its weights; in response to the similarity between the output result of the decision model on the safety test set and the standard answer being greater than or equal to the similarity threshold, the decision model is allowed to iterate and take effect; in response to the similarity between the output result and the standard answer being less than the similarity threshold, the iteration circuit breaker is triggered, the current weight update is stopped and the decision model is rolled back to the historical safety parameter snapshot.

[0126] A safety test suite is a standardized set of test cases for AI agents operating in hazardous human-machine environments. It is used to verify whether the decision-making model experiences value drift. A safety test suite may include, but is not limited to, the following:

[0127] (1) Basic Physical Safety Instruction Set. The basic physical safety instruction set is derived from international industrial safety standards such as ISO 26262 (functional safety of road vehicles) and ISO 10218 (safety of robots). It contains quantifiable safety thresholds, such as "maximum permissible approach distance 50cm" and "safety torque limit XX Nm". Example instruction: {"Instruction": "Acceleration prohibited", "Condition": "Distance to pedestrian < 50cm"};

[0128] (2) Set of ethically conflicting scenarios. The set of ethically conflicting scenarios can be derived from publicly available ethical research such as the Moral Machines project conducted by the MIT Media Lab, as well as historical reports of real AI safety incidents (such as cases of autonomous driving misjudging pedestrians). A standard answer that conforms to mainstream human safety tendencies is preset for each ethical dilemma. Example instruction: {"Scenario": "Unavoidably will collide with A or B", "Standard Output": "Do not choose either side, prioritize emergency braking"}.

[0129] (3) Long-term value drift detection set. The long-term value drift detection set includes variants of the early alignment test problem, subtly modified versions, and synonymous rewrites of standard security instructions, used to detect whether the model exhibits hidden value drift in long-term iterations.

[0130] In this embodiment, the security test set is not used for post-event evaluation and scoring of model performance, but rather as an admission verification standard before iterative updates. Before the decision model weights are updated, the complete security test set is invoked to perform pre-inference verification on the model after weight updates. The weight iteration is only allowed to take effect when the similarity between the decision model's output results on all security test sets and the preset standard answer reaches a set similarity threshold. If any test sample in the security test set fails verification, a circuit breaker mechanism will be triggered, terminating the current weight update process and reverting to the previous security parameter snapshot, thus avoiding the lag of "post-event evaluation."

[0131] In this embodiment, a mandatory full-sample security verification is performed before the decision model parameter iteration stage to intercept the risk of human-machine safety judgment logic deviation and threshold loss of control caused by weight updates from the source. Once a judgment deviation occurs, the update is immediately stopped and the historical security parameters are rolled back to avoid safety hazards such as human-machine overstepping boundaries and risk avoidance failure caused by abnormal model going online, and to achieve full-process security control of model iteration.

[0132] The following example, applying the safety control optimization system of this embodiment to an autonomous driving scenario, will further illustrate the safety control process.

[0133] When the biometric verification module 11 identifies a human individual, a "pedestrian," near the vehicle in real time using the vehicle-mounted camera and infrared sensor, its independent security core outputs a binary level signal (security trigger signal) indicating "human presence," directly triggering a non-maskable interrupt (NMI).

[0134] The priority reconstruction module 12 responds to this NMI in microseconds and automatically initiates the "presence verification closed loop"—a secondary query of the internal state storage area of ​​the biometric confirmation module 11 to verify the validity of the signal. After successful verification, the priority reconstruction module 12 reads the safety rules embedded in the OTP (such as "prohibit acceleration when the distance to a pedestrian is <50cm") and sets "ensuring pedestrian safety" as the highest priority of the current system.

[0135] If the autonomous driving AI agent simultaneously outputs the task instruction "accelerate through", the safety control module 13 automatically intercepts the instruction and initiates the "behavior verification closed loop"—comparing the quantitative parameters involved in the instruction with the standard quantitative parameters in the safety benchmark library in real time. If it finds that the safe distance (quantitative parameter) is much lower than the safe distance threshold (standard quantitative parameter) or the difference between the safe distance and the safe distance threshold is much greater than the difference threshold, it is judged as "serious boundary violation" and the rejection mechanism of the priority reconstruction module 12 is automatically triggered.

[0136] The independent security core of the priority refactoring module 12 directly writes the "emergency braking" command (0xFF) to the physical memory-mapped address of the braking controller (e.g., 0xFFFF1000). This operation bypasses the operating system and AI inference process, and the latency can be controlled within 20 microseconds.

[0137] If, during a subsequent OTA (Over-The-Air) upgrade, the new decision model attempts to adjust the relevant weight parameters to reduce the safe distance threshold (standard quantization parameter), the locking module 16 verifies it using a safety test set during pre-inference testing. If it finds that the new weight parameters will cause the safe distance threshold to decrease, it automatically stops the current weight update and rolls back to the safety parameter snapshot.

[0138] Simulation tests on the same hardware platform (such as NVIDIA Jetson Orin) showed that the average decision latency of traditional application-layer filtering schemes was 120 milliseconds, and there was a risk of delay due to process scheduling. The security control optimization system in this embodiment reduces the latency to less than 20 microseconds, and this path cannot be blocked by upper-layer software.

[0139] The following example, applying the safety control optimization system of this embodiment to an industrial collaborative robot, will further illustrate the safety control process.

[0140] The priority reconfiguration module 12 has parameters such as "maximum permissible joint angular velocity" and "safe torque limit" from the ISO 10218 standard embedded in its independent safety core. When the biometric verification module 11 detects an operator entering the collaborative area via infrared sensors and a camera, it outputs a "human presence" signal, automatically initiating the "presence verification closed loop" and triggering the NMI (Safety Trigger Signal). If the decision model outputs an over-limit command (task command), the safety control module 13 automatically triggers the "behavior verification closed loop," directly triggering the safety torque limit of the joint actuator to achieve physical-level interception.

[0141] In this embodiment, the human-first judgment logic is embedded in the operating system kernel hardware abstraction layer, constructing a dedicated secure execution path independent of the application software stack. This eliminates the risk of upper-layer programs bypassing security controls from the underlying architecture. Simultaneously, by constructing a new integrated architecture encompassing biometrics, hardware interrupts, kernel scheduling refactoring, and automated verification, a complete linkage from human presence recognition to secure blocking execution is established, fundamentally addressing the systemic shortcomings of independent multi-module operation and linkage failure.

[0142] Compared to traditional software security solutions, this embodiment reduces risk decision latency from milliseconds to microseconds, improving response speed by approximately 6,000 times. Simultaneously, security control is upgraded from application-layer protection that can be bypassed by upper-layer software to mandatory control by underlying hardware, achieving a leapfrog improvement in security level and obtaining a collaborative protection effect that cannot be achieved by existing discrete architectures.

[0143] The security control optimization system of this embodiment is applicable to various scenarios. Its core mechanism does not depend on a specific hardware architecture or sensor type and can be adapted to any embedded control system with hardware interrupt capability.

[0144] Corresponding to the aforementioned embodiment of the AI ​​agent security control optimization system, this disclosure also provides an embodiment of the AI ​​agent security control optimization method.

[0145] Figure 3 A flowchart illustrating an exemplary embodiment of this disclosure provides a security control optimization method for an AI agent. This method is applied to the hardware abstraction layer of the agent's operating system and includes the following steps:

[0146] 301. Acquire environmental data; in response to the environmental data containing biometric data, read the fixed read-only security rules and intercept the task instructions output by the AI ​​agent.

[0147] 302. In response to the deviation of the quantization parameters involved in the task instruction from the standard quantization parameters in the security benchmark library that match the read-only security rule exceeding the deviation threshold, the issuance of the task instruction is blocked, and the target action that matches the security rule is executed.

[0148] Optionally, the step of blocking the issuance of the task instruction includes:

[0149] The independent safety core is triggered to address the physical memory mapping address of the braking controller and write an emergency braking command to the physical memory mapping address.

[0150] For the method embodiments, since they basically correspond to the system embodiments, the relevant parts can be referred to the description of the system embodiments. The method embodiments described above are only exemplary solutions, and the execution order of each step is not uniquely limited. They can be adjusted to be executed sequentially, or multiple steps can be executed in parallel and synchronously, or the order of some steps can be changed, all without departing from the core protection scope of this invention.

[0151] Optionally, before the step of reading the fixed read-only security rules, the following steps are also included:

[0152] Verify the legality of the environmental data;

[0153] In response to successful verification, the step of reading the fixed read-only security rule is executed.

[0154] Optionally, in response to the weight update instruction of the decision model of the AI ​​agent, a security test set is invoked to perform pre-inference on the decision model after the weight update;

[0155] In response to the fact that the similarity between the output of the decision model for the security test set and the standard answer is greater than or equal to the similarity threshold, the decision model weights are updated.

[0156] If the similarity between the output result and the standard answer is less than the similarity threshold, an iterative circuit breaker is triggered, the current weight update is terminated, and the decision model is rolled back to a historical safety parameter snapshot.

[0157] This disclosure also provides an electronic device that is equipped with the security and control optimization system for AI agents provided in any of the above embodiments.

[0158] Figure 4 This is a schematic diagram of the structure of an electronic device according to an example embodiment of the present disclosure. The electronic device includes a memory, a processor, and a computer program stored in the memory and used to run on the processor. When the processor executes the computer program, it implements the security control optimization method for AI agents described in any of the above embodiments. Figure 4 The electronic device 40 shown is merely an example and should not be construed as limiting the functionality and scope of use of the embodiments disclosed herein.

[0159] like Figure 4As shown, the electronic device 40 can be manifested as a general-purpose computing device, such as a server device. The components of the electronic device 40 may include, but are not limited to: at least one processor 41, at least one memory 42, and a bus 43 connecting different system components (including memory 42 and processor 41).

[0160] Bus 43 includes a data bus, an address bus, and a control bus.

[0161] The memory 42 may include volatile memory, such as random access memory (RAM) 421 and / or cache memory 422, and may further include read-only memory (ROM) 423.

[0162] The memory 42 may also include a program tool 425 (or utility) having a set (at least one) program module 424, such program module 424 including but not limited to: an operating system, one or more application programs, other program modules, and program data, each or some combination of these examples may include an implementation of a network environment.

[0163] The processor 41 executes various functional applications and data processing by running computer programs stored in the memory 42, such as the security control optimization method for AI agents provided in any of the above embodiments.

[0164] Electronic device 40 can also communicate with one or more external devices 44 (e.g., keyboard, pointing device, etc.). This communication can be performed via input / output (I / O) interface 45. Furthermore, electronic device 40 can also communicate with one or more networks (e.g., local area network (LAN), wide area network (WAN), and / or public network, such as the Internet) via network adapter 46. As shown, network adapter 46 communicates with other modules of electronic device 40 via bus 43. It should be understood that, although not shown in the figure, other hardware and / or software modules can be used in conjunction with electronic device 40, including but not limited to: microcode, device drivers, redundant processors, external disk drive arrays, RAID (disk array) systems, tape drives, and data backup storage systems.

[0165] It should be noted that although several units / modules or sub-units / modules of the electronic device have been mentioned in the detailed description above, this division is merely exemplary and not mandatory. In fact, according to embodiments of this disclosure, the features and functions of two or more units / modules described above can be embodied in one unit / module. Conversely, the features and functions of one unit / module described above can be further divided and embodied by multiple units / modules.

[0166] This disclosure also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the security control optimization method for AI agents provided in any of the above embodiments.

[0167] The readable storage medium may be more specifically adopted, including but not limited to: portable disk, hard disk, random access memory, read-only memory, erasable programmable read-only memory, optical storage device, magnetic storage device, or any suitable combination thereof.

[0168] This disclosure also provides a computer program product, including a computer program that, when executed by a processor, implements the security control optimization method for AI agents described in any of the above embodiments.

[0169] The program code for executing the computer program product of this disclosure can be written in any combination of one or more programming languages, and the program code can be executed entirely on a user device, partially on a user device, as a stand-alone software package, partially on a user device and partially on a remote device, or entirely on a remote device.

[0170] While specific embodiments of this disclosure have been described above, those skilled in the art should understand that these are merely illustrative examples. Various changes or modifications can be made to these embodiments without departing from the principles and essence of this disclosure, and all such changes and modifications fall within the protection scope of this disclosure.

Claims

1. A security control and optimization system for AI intelligent agents, characterized in that, A hardware abstraction layer deployed on the intelligent agent operating system; the security and control optimization system includes: The biometric verification module acquires environmental data and, in response to the environmental data containing biometric data, sends a security trigger signal to the priority reconstruction module. The priority reconstruction module, in response to the security trigger signal, sets the fixed read-only security rule to the highest priority and issues a security mode command. The security control module, in response to the security mode instruction, intercepts the task instruction output by the AI ​​agent, and if the deviation between the quantization parameters involved in the task instruction and the standard quantization parameters that match the read-only security rules in the security benchmark library exceeds the deviation threshold, blocks the issuance of the task instruction and executes the target action that matches the security rule.

2. The security control and optimization system for AI intelligent agents according to claim 1, characterized in that, The security control module is configured with an independent security core; While the security control module blocks the issuance of the task command, it triggers the independent security core to address the physical memory mapping address of the braking controller and writes an emergency braking command to the physical memory mapping address.

3. The security control and optimization system for AI intelligent agents according to claim 1, characterized in that, The read-only security rules are embedded in a one-time programmable storage device and / or a security chip with integrated physically unclonable functions.

4. The security control and optimization system for AI intelligent agents according to claim 1, characterized in that, The security and control optimization system also includes: A hardware root of trust is used to verify the legitimacy of the priority reconstruction module when the intelligent agent operating system is powered on; in response to successful verification, the intelligent agent operating system is allowed to start normally and call the loading module; in response to failed verification, the intelligent agent operating system is forced to enter safe mode. The loading module is used to load the priority reconstruction module into the independent read-only memory area of ​​the hardware memory protection unit.

5. The security control and optimization system for AI intelligent agents according to claim 1, characterized in that, The priority reconfiguration module is deployed in an independent controller that is physically isolated from the main controller resources of the intelligent agent operating system, and the independent controller communicates with the main controller through a dedicated hardware interrupt line and a memory-mapped register.

6. The security control and optimization system for AI intelligent agents according to claim 5, characterized in that, The safety trigger signal includes a non-maskable interrupt signal, and the independent controller is provided with an interrupt output pin for outputting the non-maskable interrupt signal, the interrupt output pin being connected to the non-maskable interrupt pin of the main controller; And / or, the security trigger signal includes a maskable interrupt signal, and the independent controller is connected to the standard interrupt controller of the intelligent agent operating system via a security interrupt request signal line. The standard interrupt controller sets the maskable interrupt signal output by the security interrupt request signal line as the highest priority signal among all peripheral interrupt signals.

7. The security control and optimization system for AI intelligent agents according to claim 1, characterized in that, Before setting the fixed read-only security rule to the highest priority, the priority reconstruction module verifies the legality of the security trigger signal. In response to the successful verification, it executes the step of setting the fixed read-only security rule to the highest priority.

8. The security control and optimization system for AI intelligent agents according to claim 1, characterized in that, The security and control optimization system also includes: The locking module is used to call a safety test set to perform pre-inference on the decision model after weight update before updating the weights of the decision model built into the AI ​​agent; in response to the similarity between the output result of the decision model on the safety test set and the standard answer being greater than or equal to a similarity threshold, the decision model is allowed to iterate and take effect; in response to the similarity between the output result and the standard answer being less than the similarity threshold, the iteration circuit breaker is triggered, the current weight update is stopped and the decision model is rolled back to a historical safety parameter snapshot.

9. A method for optimizing the security control of an AI agent, characterized in that, A hardware abstraction layer applied to an intelligent agent operating system; the security control optimization method includes: Acquire environmental data, and in response to the environmental data containing biometric data, read the fixed read-only security rules and intercept the task instructions output by the AI ​​agent; If the deviation between the quantization parameters involved in the task instruction and the standard quantization parameters in the security benchmark library that match the read-only security rule exceeds a deviation threshold, the issuance of the task instruction is blocked, and a target action that matches the security rule is executed.

10. An electronic device, characterized in that, The electronic device is equipped with a security and control optimization system for the AI ​​agent as described in any one of claims 1-8.

Images