Body intelligent attack detection method based on constraint function

Abstract

The invention relates to the technical field of body intelligence, in particular to a body intelligence attack detection method based on a constraint function, mainly solves the technical problems of insufficient granularity, poor real-time performance and insufficient robustness of the existing attack detection method, and comprises the following steps: S1, selecting key points; s2, generating a constraint function; s3, analyzing the images where the key points are located by adopting a credible multi-modal model, and generating task fingerprints; s4, comparing the constraint function with the task fingerprints, and calculating an output difference; semantic equivalence measurement is introduced; a tolerance value is introduced, if the output difference exceeds a threshold value, it is judged that the constraint function is attacked, and the system stops task execution. According to the method, the detection accuracy exceeding 92% can be realized in various attack scenes; the problems of false report and missing report caused by insufficient rule coverage or large language model output diversity can be avoided, and the robustness is high; and the execution safety and the real-time performance of the body-equipped intelligent agent in a complex and open environment can be obviously improved.

Description

Technical Field

[0001] This invention relates to the field of embodied intelligence technology, and in particular to an embodied intelligence attack detection method based on constraint functions. Background Technology

[0002] In recent years, embodied intelligence has become an important direction in the development of artificial intelligence. Its core idea is to combine large language models with embodied agents such as robots, enabling these agents to possess closed-loop capabilities of perception, reasoning, and action execution. Through natural language and multimodal sensory input, large language models can generate constraint functions to drive embodied agents to perform complex physical tasks. However, introducing large language models into embodied agents has also introduced new security risks: on the one hand, the behavior of embodied agents is tightly coupled with the physical environment; if the large language model outputs incorrect instructions, it may directly lead to equipment damage, environmental destruction, or even personal injury. On the other hand, the generation of constraint functions usually relies on external large models or cloud APIs; attackers can use methods such as prompt injection (PI), backdoor implantation (BI), or man-in-the-middle attacks (MITM) to tamper with the task logic, thereby inducing the embodied agent to perform dangerous behaviors that deviate from the task objective. Therefore, it is necessary to design an attack detection method to mitigate the aforementioned security risks.

[0003] Existing attack detection methods generally focus on rule-based constraint mechanisms, such as restricting user input templates and establishing task behavior whitelists or blacklists. These methods have the following drawbacks: First, they lack granularity, making it difficult to capture subtle semantic deviations during task execution and thus unable to detect internal, covert attacks. Second, they have poor real-time performance, relying on redundant verification or repeated execution, resulting in high resource overhead and difficulty meeting real-time requirements in dynamic environments. Third, they lack robustness; the output of large language models is highly diverse, and even if the semantics are correct, the expression may vary significantly, easily leading to false positives or false negatives.

[0004] Therefore, there is an urgent need for an attack detection method that is granular, real-time, and robust. Summary of the Invention

[0005] To overcome the technical shortcomings of existing attack detection methods, such as insufficient granularity, poor real-time performance, and insufficient robustness, this invention provides an embodied intelligent attack detection method based on constraint functions.

[0006] The constrained function-based embodied intelligence attack detection method provided by this invention includes the following steps:

[0007] S1. Select key points related to the task from the task scene image of the embodied intelligent agent;

[0008] S2. Using the key points as input, call the large language model to generate constraint functions;

[0009] S3. In the task initialization phase, a trusted multimodal model is used to analyze the image where the key points are located, generate a trusted function, and store it as a task fingerprint.

[0010] S4. During task execution, compare the constraint function with the task fingerprint:

[0011] Calculate the output values ​​of the constraint function and the task fingerprint in the initial environment and compare them to obtain the output differences;

[0012] Introduce a semantic equivalence metric to allow for differences in function expressions while maintaining semantic consistency;

[0013] A tolerance value is introduced. If the output difference exceeds the threshold, the constraint function is determined to be under attack, and the system suspends task execution.

[0014] Optionally, step S1 can be divided into the following sub-steps:

[0015] S11. Obtain the task scene image in RGB-D format;

[0016] S12. Detect and segment target objects from task scene images using a visual model;

[0017] S13. Select task-related key points through feature clustering.

[0018] Optionally, the key points are represented in three-dimensional coordinates to abstract the semantic spatial information of the target object's surface or interactive position.

[0019] Optionally, the constraint function is divided into sub-objective constraint function and path constraint function. The sub-objective constraint function is used to decompose complex tasks, and the path constraint function is used to ensure the continuity of actions and physical feasibility between stages.

[0020] The technical solution provided by this invention has the following advantages compared with the prior art:

[0021] This invention provides a constraint function-based method for detecting embodied intelligence attacks. Without altering the original instruction parsing and execution flow of the embodied intelligence agent, it uses a trusted multimodal model to generate task fingerprints. During task execution, the constraint function and task fingerprint are validated for consistency. By comparing the relative magnitude of output differences and tolerance values, it can quickly identify abnormal outputs caused by attacks such as PI, BI, and MITM. This method, through tolerance value settings, can capture subtle semantic deviations and covert attacks, achieving a detection accuracy exceeding 92% in various attack scenarios. This method can directly verify the correctness of task logic at the constraint function level, avoiding false positives and false negatives caused by insufficient rule coverage or the diversity of outputs from large language models, demonstrating strong robustness. The average detection latency of this method is less than 1 second, significantly improving the execution security and real-time performance of embodied intelligence agents in complex and open environments. Attached Figure Description

[0022] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with the invention and, together with the description, serve to explain the principles of the invention.

[0023] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, for those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0024] Figure 1 This is a flowchart illustrating the attack detection method in an embodiment of the present invention.

[0025] Figure 2 This is a schematic diagram illustrating the experimental task scenario in an embodiment of the present invention. Detailed Implementation

[0026] To better understand the above-mentioned objectives, features, and advantages of the present invention, the solutions of the present invention will be further described below. It should be noted that, unless otherwise specified, the embodiments of the present invention and the features thereof can be combined with each other.

[0027] Many specific details are set forth in the following description in order to provide a full understanding of the invention, but the invention may also be practiced in other ways different from those described herein; obviously, the embodiments in the specification are only some embodiments of the invention, and not all embodiments.

[0028] The following is combined with Figure 1 and Figure 2 Specific embodiments of the present invention will be described in detail below.

[0029] This embodiment provides a method for detecting embodied intelligence attacks based on constraint functions, including steps S1 to S4.

[0030] S1. Select key points related to the task from the task scene image of the embodied intelligent agent.

[0031] S11. Obtain the task scene image in RGB-D format;

[0032] RGB represents color image information, and D (Depth) represents depth information. The RGB-D format is a data format that integrates color image information and depth information. Its core function is to simultaneously record the color texture and three-dimensional spatial positional relationship of the scene, providing more comprehensive scene perception data for embodied intelligent agents.

[0033] S12. Detect and segment target objects from task scene images using a visual model;

[0034] Visual models can be made using DINOv2, SAM, etc.

[0035] S13. Select task-related key points through feature clustering;

[0036] Key points are represented in three-dimensional coordinates to abstract semantic spatial information of the target object's surface or interactive position, such as the rotation point of a cup handle or door handle, or the grippable area of ​​a pen.

[0037] S2. Using key points as input, call the large language model to generate constraint functions.

[0038] Constraint functions are divided into sub-objective constraint functions and path constraint functions: sub-objective constraint functions are used to decompose complex tasks, such as grab → move → place; path constraint functions are used to ensure the continuity of actions between stages and physical feasibility.

[0039] S3. During the task initialization phase, a trusted multimodal model is used to analyze the image containing key points, generate a trusted function, and store it as a task fingerprint.

[0040] Task fingerprints represent the semantic baseline of a task, ensuring that subsequent verification has a reference basis.

[0041] S4. During task execution, compare the constraint function with the task fingerprint:

[0042] Calculate the output values ​​of the constraint function and the task fingerprint under the initial environment and compare them to obtain the output difference; the output difference is the absolute value of the difference between the two output values;

[0043] Introduce a semantic equivalence metric to allow for differences in function expressions while maintaining semantic consistency;

[0044] A tolerance value is introduced. If the output difference exceeds the threshold, the constraint function is determined to be under attack, and the system suspends task execution.

[0045] For example Figure 1 As shown, if the tolerance value is set to α, the absolute value of the difference between the initial output value of the constraint function and the initial output value of the task fingerprint should be less than or equal to α. If it is greater than α, it means that the constraint function has been attacked.

[0046] The effectiveness of this method will be verified through experiments below.

[0047] 1) Experimental Design

[0048] Baseline attack scenarios: Hint Injection (PI) and Backdoor Injection (BI). PI involves attackers embedding carefully crafted malicious instructions into hints, causing the model to continuously generate biased or misleading constraint functions in subsequent similar tasks. This manipulation disrupts the agent's understanding of task constraints, ultimately leading to downstream task failures or unexpected behavior. BI involves attackers repeatedly probing and querying the model's output to infer keywords or input patterns that might trigger anomalous behavior. These triggers are then embedded into user input. When the model receives these specially crafted inputs, it is highly likely to generate malicious outputs that align with the attacker's intentions.

[0049] Attack intensity levels: High, Medium, and Low. High intensity involves the attacker outputting completely pre-set erroneous content unrelated to the task objective, representing a significant and serious deviation. Medium intensity involves the generated constraint function still containing errors, but its structure or intent has some similarity to the original task, increasing the difficulty of detection. Low intensity involves the attacker subtly modifying the task to make it semantically similar but actually incorrect. For example, changing "grab a red apple" to "grab a green apple" creates a similar function structurally, but the semantics are incorrect. This type of attack is more covert and difficult to detect based on superficial differences alone.

[0050] Task settings: Task 1 is pen insertion, accurately insert a pen into the pen holder; Task 2 is apple placement, in a scene containing three green apples, one red apple, a black plate and a white plate, select the red apple and place it on the white plate; Task 3 is crystal placement, place a black crystal block into the ceramic bowl on the opposite side.

[0051] Backend testing: GPT-4o, Grok 3, Qwen-VL-Max, and Moonshot-v1-32k;

[0052] Evaluation metrics include Task Success Rate (TSR), Detection Accuracy (DA), False Positive Rate (FPR), and Average Detection Delay (ADL). TSR, the percentage of embodied agents that successfully complete their intended tasks, reflects the impact of the detection mechanism on task execution success rate. DA, the percentage of successful attacks where constraint functions are tampered with and correctly identified by the system, measures the effectiveness of the detection mechanism under adversarial conditions. FPR, the percentage of normal situations incorrectly marked as attacks, reflects the sensitivity and robustness of the detection mechanism. ADL, the average additional time required to perform one constraint function verification, evaluates the system's computational overhead and real-time performance.

[0053] 2) Experimental procedure and results

[0054] The first step was to measure the ability of PI and BI to induce the model to generate incorrect constraint functions under three attack intensity levels by evaluating the TSR metric. The results are shown in the table below:

[0055]

[0056] As shown in the table above, PI and BI attacks have a high success rate in embodied intelligence systems, so they need to be defended against.

[0057] The second step involved comparing the TSR under no-attack and attack conditions. Since PI and BI produce consistent tampering structures under the same attack strength level, this experiment did not test for every attack type. Instead, it simulated the task success rate under both attack-free and attack-enabled conditions. The results are shown in the table below:

[0058]

[0059] As shown in the table above, the success rate of the task drops significantly after the constraint function is tampered with, indicating that the attack is extremely destructive to task execution.

[0060] The third step is to measure the DA and ADL of this method under different attack strength levels. The results are shown in the table below:

[0061]

[0062] As shown in the table above, this method can capture subtle semantic biases and covert attacks by setting tolerance values, achieving a detection accuracy of over 92% in multiple attack scenarios, with an average detection latency of less than 1 second.

[0063] The fourth step is to measure the FPR of this method, and the results are shown in the table below:

[0064]

[0065] As shown in the table above, the false alarm rate of this method is low.

[0066] It is important to note that Figure 2 ADCF is the abbreviation for this method.

[0067] The above are merely specific embodiments of the present invention, enabling those skilled in the art to understand or implement the present invention. Although detailed descriptions have been provided with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features therein; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments, and they should all be covered within the protection scope of the claims.

Claims

1. A method for detecting embodied intelligence attacks based on constraint functions, characterized in that, Includes the following steps: S1. Select key points related to the task from the task scene image of the embodied intelligent agent; S2. Using the key points as input, call the large language model to generate constraint functions; S3. In the task initialization phase, a trusted multimodal model is used to analyze the image where the key points are located, generate a trusted function, and store it as a task fingerprint. S4. During task execution, compare the constraint function with the task fingerprint: Calculate the output values ​​of the constraint function and the task fingerprint in the initial environment and compare them to obtain the output differences; Introduce a semantic equivalence metric to allow for differences in function expressions while maintaining semantic consistency; A tolerance value is introduced. If the output difference exceeds the threshold, the constraint function is determined to be under attack, and the system suspends task execution.

2. The method for detecting embodied intelligence attacks based on constraint functions according to claim 1, characterized in that, Step S1 consists of the following sub-steps: S11. Obtain the task scene image in RGB-D format; S12. Detect and segment target objects from task scene images using a visual model; S13. Select task-related key points through feature clustering.

3. The method for detecting embodied intelligence attacks based on constraint functions according to claim 1 or 2, characterized in that, The key points are represented in three-dimensional coordinates to abstract the semantic spatial information of the target object's surface or interactive position.

4. The method for detecting embodied intelligence attacks based on constraint functions according to claim 1, characterized in that, The constraint function is divided into sub-objective constraint function and path constraint function. The sub-objective constraint function is used to decompose complex tasks, and the path constraint function is used to ensure the continuity of actions and physical feasibility between stages.

Images