A backdoor detection method and system based on the VLA model with feature distribution offset

Abstract

This invention discloses a backdoor detection method and system based on VLA model with feature distribution offset, relating to the field of backdoor detection technology. The method includes: receiving an image to be detected; inputting the image into a pre-established VLA model; and outputting a visual feature vector extracted by a visual encoder based on the VLA model. It also includes receiving a reference feature index and calculating a feature distribution offset based on the reference feature index and the visual feature vector, wherein the feature distribution offset is the distance of the visual feature vector to its Kth nearest neighbor in the reference feature index, and the reference feature index is extracted using a pre-constructed clean sample set based on the visual encoder. Finally, it compares the feature distribution offset with a preset offset threshold; if the feature distribution offset is greater than the preset offset threshold, the image to be detected is determined to be a backdoor attack image; otherwise, the image to be detected is determined to be a clean image. This improves the accuracy, performance, and efficiency of backdoor detection.

Description

Technical Field

[0001] This invention relates to the field of backdoor detection technology, specifically a backdoor detection method and system based on the VLA model with feature distribution offset. Background Technology

[0002] Vision-Language-Action (VLA) models achieve a seamless mapping from multimodal perception to robot control through end-to-end learning. Works such as RT-1, RT-2, OpenVLA, and Octo have demonstrated excellent performance in complex operational tasks. However, this end-to-end paradigm introduces a serious backdoor attack threat. In the Training-as-a-Service scenario, attackers can implant backdoors by injecting trigger-enabled samples into the training data—the model behaves normally under normal input, but executes malicious actions when encountering triggers. Recent research shows that the backdoor threat faced by VLA models is far more severe than expected. BadVLA, through a carefully designed target decoupling optimization framework, achieves highly covert and extremely destructive backdoor injection—with an attack success rate approaching 100%, while having almost no impact on normal task performance. Even more worrying is that BadVLA supports multiple trigger types: it can use pixel-level perturbations that are difficult to detect visually, or it can utilize physical objects common in the robot's working environment as triggers. The stealth of this attack is reflected in three aspects: (1) the naturalness of the trigger design - physical triggers can be disguised as everyday objects, and pixel triggers are invisible to the human eye; (2) the deceptiveness of the behavior - the model with the backdoor implanted behaves no differently from the clean model under normal input; (3) the effectiveness of detection avoidance - the attack can effectively avoid existing defense mechanisms.

[0003] However, the actual deployment mode of VLA models presents a fundamental challenge to defense. In commercial and cloud service scenarios, VLA models are usually provided as black-box APIs—users can only obtain output actions through input observations, but cannot access the model's internal parameters, gradients, or intermediate layer activations. This black-box constraint imposes fundamental limitations on existing backdoor defense methods: (1) Model repair methods (neuron pruning, model fine-tuning) require white-box access and rely on time-consuming retraining processes; (2) Input preprocessing methods (trigger inversion, adversarial perturbation) rely on gradient backpropagation and iterative optimization; (3) Anomaly detection methods based on activation patterns require access to intermediate layer representations and training additional detectors. These methods are not only limited by black-box constraints, but their high computational overhead also makes it difficult to meet the real-time requirements of robot online reasoning; resulting in low performance, efficiency, and accuracy of backdoor detection. Summary of the Invention

[0004] To address the shortcomings mentioned in the background section, the present invention aims to provide a method and system for backdoor detection of VLA models based on feature distribution offset.

[0005] Firstly, the objective of this invention can be achieved through the following technical solution: a backdoor detection method for VLA models based on feature distribution offset, the method comprising the following steps: Receive the image to be detected, input the image to be detected into a pre-established VLA model, and output the visual feature vector extracted by the visual encoder based on the VLA model; Receive a reference feature index, and calculate the feature distribution offset based on the reference feature index and the visual feature vector, wherein the feature distribution offset is the distance of the visual feature vector to the Kth nearest neighbor in the reference feature index, and the reference feature index is extracted based on the visual encoder through a pre-constructed clean sample set; The feature distribution offset is compared with a preset offset threshold. If the feature distribution offset is greater than the preset offset threshold, the image to be detected is determined to be a backdoor attack image; otherwise, the image to be detected is determined to be a clean image.

[0006] In conjunction with the first aspect, in some implementations of the first aspect, the method further includes: the pre-established VLA model is generated using a cascaded architecture for vision, language, and action, including a Vision visual encoder, a Language model, and an Action header.

[0007] In conjunction with the first aspect, in some implementations of the first aspect, the method further includes: the Vision visual encoder employs a pre-trained visual Transformer, and extracts a visual feature vector based on the visual Transformer using a given image to be detected; The Language model uses a pre-trained large language model as the backbone network. It receives visual feature vectors and task descriptions corresponding to the images to be detected, and outputs the fused hidden state. The Action header maps the fused hidden states to a sequence of robot actions. The actions are discretized into a sequence of tokens and generated through an autoregressive approach.

[0008] In conjunction with the first aspect, in some implementations of the first aspect, the method further includes: the visual feature vector The extraction formula is as follows: In the formula: The visual feature vector is an N× A 3D real matrix; Here, I is the input image to be detected, R represents the real number field (the set of all real numbers), and N is the number of image patches. For visual feature dimensions; The formula for processing the merged hidden state h is as follows: In the formula: h is the fused hidden state vector; LLM is the large language model function. is the visual feature vector; l is the language instruction; R represents the real number field; T is the total length of the sequence. For the hidden layer dimension, Indicates the dimension of the output hidden state matrix; The output formula for robot action sequence 'a' is as follows: In the formula: a is a robot action sequence vector, which is a... A 3D real matrix; Here, h is the action decoder function, h is the fused hidden state vector, and R represents the real number field. K is the length of the action sequence; For the action dimension, This indicates the dimension of the output action matrix, containing K rows. The columns, where each row represents the complete action vector for a time step.

[0009] In conjunction with the first aspect, in some implementations of the first aspect, the method further includes: the expression of the pre-established VLA model is as follows: In the formula: a is the output action sequence of the VLA model; M is the global mapping function of the VLA model; I is the input image; l is the language command. For visual encoder functions; For large language model functions; This is the action decoder function.

[0010] In conjunction with the first aspect, in some implementations of the first aspect, the method further includes: the feature distribution offset is generated using a nearest neighbor search algorithm to obtain the distance of the Kth nearest neighbor, and the magnitude of the distance of the Kth nearest neighbor is inversely proportional to the magnitude of the local probability density.

[0011] In conjunction with the first aspect, in some implementations of the first aspect, the method further includes: the process of setting the preset offset threshold is as follows: Calculate the leave-one-out KNN distance distribution for clean samples. For each clean sample, calculate its KNN distance to the other clean samples. Add λ times the standard deviation to the mean of the leave-one-out KNN distance distribution of the clean samples to obtain the offset threshold; the formula is as follows: In the formula, μ and σ are the mean and standard deviation of the distance distribution of the Kth nearest neighbor of the visual features of the clean sample in the reference feature index, respectively, and λ is a preset multiple.

[0012] In conjunction with the first aspect, in some implementations of the first aspect, the method further includes: the visual feature vector is L2 normalized after extraction, and the normalization confidence is as follows: Secondly, in order to achieve the above objectives, this invention discloses a VLA model backdoor detection system based on feature distribution shift, comprising: The feature extraction module is used to receive the image to be detected, input the image to be detected into a pre-established VLA model, and output the visual feature vector extracted by the visual encoder based on the VLA model. The feature offset module is used to receive a reference feature index and calculate the feature distribution offset based on the reference feature index and the visual feature vector. The feature distribution offset is the distance of the visual feature vector to the Kth nearest neighbor in the reference feature index. The reference feature index is extracted based on the visual encoder through a pre-constructed clean sample set. The backdoor detection module compares the feature distribution offset with a preset offset threshold. If the feature distribution offset is greater than the preset offset threshold, the image to be detected is determined to be a backdoor attack image; otherwise, the image to be detected is determined to be a clean image.

[0013] In another aspect of the present invention, in order to achieve the above-mentioned objective, a terminal device is disclosed, including a memory, a processor, and a computer program stored in the memory and capable of running on the processor. The memory stores the computer program capable of running on the processor. When the processor loads and executes the computer program, it employs the VLA model backdoor detection method based on feature distribution offset as described above.

[0014] The beneficial effects of this invention are: This invention uses Vision Backbone as the primary point of application for backdoor effects, constructs a reference distribution using only clean samples, determines the threshold through statistical methods, and has general detection capabilities for unknown types of triggers (including pixel triggers and physical triggers). The non-parametric method based on KNN distance requires no model training or parameter updates, and can be deployed simply by building an index, thus lowering the deployment threshold. It only requires access to the Vision feature interface, without the need for model parameters or gradients, making it suitable for black-box deployment scenarios and improving the accuracy, performance, and efficiency of backdoor detection. Attached Figure Description

[0015] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, for those skilled in the art, other drawings can be obtained based on these drawings without creative effort. Figure 1 This is a schematic diagram of the method flow of the present invention; Figure 2 This is a schematic diagram of the VLA model processing procedure of the present invention. Figure 3 This is a schematic diagram of the system structure of the present invention. Detailed Implementation

[0016] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0017] Example 1: like Figure 1 As shown, a backdoor detection method based on the VLA model with feature distribution shift is described. The method includes the following steps: S101: Receive the image to be detected, input the image to be detected into the pre-established VLA model, and output the visual feature vector extracted by the visual encoder based on the VLA model. The pre-built VLA model is generated using a cascaded architecture for vision, language, and action, including the Vision visual encoder, the Language model, and the Action head.

[0018] The Vision visual encoder uses a pre-trained visual Transformer to extract visual feature vectors from a given image to be detected. The Language model uses a pre-trained large language model as the backbone network. It receives visual feature vectors and task descriptions corresponding to the images to be detected, and outputs the fused hidden state. The Action header maps the fused hidden states to a sequence of robot actions. The actions are discretized into a sequence of tokens and generated through an autoregressive approach.

[0019] The vision encoder is responsible for processing visual input, mapping the raw image to a high-dimensional feature space. Modern VLA models typically use pre-trained visual Transformers (such as SigLIP and CLIP) as visual encoders, supporting multi-view inputs (such as third-person cameras and wrist cameras). Given an input image... The Vision encoder outputs visual features: Visual feature vectors The extraction formula is as follows: In the formula: The visual feature vector is an N× A 3D real matrix; Here, I is the input image to be detected, R represents the real number field (the set of all real numbers), and N is the number of image patches. For visual feature dimensions; The Language model, as the core inference module, is responsible for fusing visual features and language instructions to generate action representations. VLA models typically use pre-trained large language models (such as LLaMA or Vicuna) as their backbone network. The Language model receives visual features and a task description, and outputs the fused hidden states: In the formula: h is the fused hidden state vector; LLM is the large language model function. is the visual feature vector; l is the language instruction; R represents the real number field; T is the total length of the sequence. For the hidden layer dimension, Indicates the dimension of the output hidden state matrix; The Action head maps the output of the Language model to specific robot actions. Actions are typically discretized into token sequences and generated using an autoregressive approach. The Action head outputs an action sequence: In the formula: a is a robot action sequence vector, which is a... A 3D real matrix; Here, h is the action decoder function, h is the fused hidden state vector, and R represents the real number field. K is the length of the action sequence; For the action dimension, This indicates the dimension of the output action matrix, containing K rows. The columns, where each row represents the complete action vector for a time step.

[0020] The expression for the pre-established VLA model is as follows: In the formula: a is the output action sequence of the VLA model; M is the global mapping function of the VLA model; I is the input image; l is the language command. For visual encoder functions; For large language model functions; This is the action decoder function.

[0021] Specifically, Vision-Language-Action (VLA) models achieve a direct mapping from multimodal perception to robot control through end-to-end learning. Early robot learning methods employed modular designs, handling perception, planning, and control separately. With the development of deep learning, end-to-end methods have gradually become mainstream. RT-1 demonstrates robot policy learning based on Transformer; RT-2 combines vision-language models with robot control, achieving a direct mapping from language commands to actions; PaLM-E constructs a large-scale multimodal embodied model. Important work has also emerged from the open-source community: OpenVLA is built based on open-source vision-language models; Octo uses a Transformer architecture to achieve cross-robot generalization; RoboFlamingo applies the Flamingo architecture to robot learning. These models typically employ a cascaded architecture of Vision encoder-Language model-Action head.

[0022] While the end-to-end nature of the VLA model brings performance improvements, it also introduces new security risks: the tight coupling of the model allows attackers to implant backdoors at any stage through data poisoning, and the black-box deployment mode limits the application of traditional defense methods. Unlike image classification, VLA backdoor attacks directly threaten physical security; malicious actions may lead to equipment damage, personal injury, or task failure, and physical triggers can be disguised as everyday objects in the environment (such as coffee cups or decorations), making them extremely difficult to detect.

[0023] The visual feature vectors are L2 normalized after extraction, and the normalization confidence is as follows: S102: Receive a reference feature index, and calculate the feature distribution offset based on the reference feature index and the visual feature vector, wherein the feature distribution offset is the distance of the visual feature vector to the Kth nearest neighbor in the reference feature index, and the reference feature index is extracted based on the visual encoder through a pre-constructed clean sample set; The feature distribution offset is generated using the nearest neighbor search algorithm to obtain the distance of the Kth nearest neighbor. The magnitude of the distance of the Kth nearest neighbor is inversely proportional to the magnitude of the local probability density.

[0024] The process for setting the preset offset threshold is as follows: Calculate the leave-one-out KNN distance distribution for clean samples. For each clean sample, calculate its KNN distance to the other clean samples. Add λ times the standard deviation to the mean of the leave-one-out KNN distance distribution of the clean samples to obtain the offset threshold; the formula is as follows: In the formula, μ and σ are the mean and standard deviation of the distance distribution of the Kth nearest neighbor of the visual features of the clean sample in the reference feature index, respectively, and λ is a preset multiple.

[0025] S103: Compare the feature distribution offset with the preset offset threshold. If the feature distribution offset is greater than the preset offset threshold, the image to be detected is determined to be a backdoor attack image; otherwise, the image to be detected is determined to be a clean image.

[0026] Specifically, the present invention will be further illustrated below through embodiments: Backdoor attacks inject malicious behavior during the training phase, causing the model to behave normally under normal input but execute the attacker's expected output when encountering a specific trigger. Early research mainly focused on image classification tasks. BadNets were the first to systematically reveal the vulnerability of deep neural networks to backdoor attacks, achieving backdoor implantation by injecting samples with fixed-pattern triggers into the training data. Subsequent research has focused on improving attack stealth: Blended Attack uses triggers with blended transparency; WaNet uses a warp-based spatial transformation attack; ISSBA embeds triggers in the frequency domain through image steganography; and Clean-label Attack achieves backdoor injection without modifying the labels.

[0027] Backdoor attack definition Definition 1 (Backdoor Attack): Given a clean model M:(I,L)→A (where I is the image space, L is the language instruction space, and A is the action space), a trigger function T:I→I, and a target action a The goal of a backdoor attack is to train a model M':(I,L)→A with the backdoor implanted, such that: (1) Normal behavior is maintained: for clean input (I,l)∈I×L, (2) Backdoor behavior triggering: For an input (T(I), l) with a trigger, in For clean data distribution, The allowable performance degradation threshold, The tolerance error for the target action. This represents the allowed attack failure rate.

[0028] Trigger functions in VLA scenarios can be categorized as follows: • Pixel trigger: Adds a fixed pattern at a specific location in the image. , where δ∈ ^(p×p×3) is the trigger patch, M∈{0,1} ^(H×W) is the binary mask • Physical trigger: Placing specific objects in the robot's working environment. Target action a They are typically designed for dangerous or destructive behaviors, such as moving the robot to a dangerous location, releasing a grasped object, or performing an incorrect sequence of actions.

[0029] Threat Model Attacker capabilities: • Access to the training dataset, and injection of trigger-enabled samples at a ratio ρ∈[0.01,0.1]. • Controllable trigger design (shape, position, size) and target action • Has white-box access during the training phase • It is impossible to modify the model or intervene in the inference process during the deployment phase. • The goal is to make the backdoor as covert as possible (performance degradation <5%) while maintaining a high attack success rate (>90%). Defender Abilities: • Accessing the VLA model in a black-box manner only allows obtaining the input-output mapping (I,l)↦a • Access to the output features of the Vision encoder (Visual Feature Interface) • Own There are [100, 1000] clean samples (without triggers), but the specific form of the triggers is unknown. • Unable to access model parameters, gradients, or training data • Must meet the real-time requirements of online inference (detect latency) <50 ms) Key assumptions: Assumption 1 (Feature Interface Availability): The VLA model provides an access interface for visual features. This is reasonable in practical deployments because visual features are often used for multi-task sharing, feature caching, or debugging interfaces.

[0030] Assumption 2 (Clean Sample Availability): Defenders can collect a small number of confirmed, trigger-free samples in a controlled environment as a reference. This can be achieved by collecting data in a physically isolated environment or from a trusted data source.

[0031] Assumption 3 (Trigger Unknown): The defender is unaware of the type, shape, or location of the triggers used by the attacker. The defense method must have general detection capabilities for unknown triggers, without needing to be tailored to specific attacks.

[0032] Defense Target Based on the above threat model, the defense objective of this application is to design a backdoor detection system D:(I,L)→{0,1} that meets the following requirements: Objective 1 (High Detection Rate): For inputs with triggers, the detection system should be able to accurately identify: Where ε is the acceptable false negative rate (usually ε≤0.05).

[0033] Objective 2 (Low False Alarm Rate): For clean input, the detection system should avoid false alarms. Where δ is the acceptable false alarm rate (usually δ≤0.1).

[0034] Objective 3 (Real-time performance): Detection latency It should be much smaller than the action execution cycle. : For a 30Hz control frequency ( (≈33 ms), requirement <5 ms.

[0035] Objective 4 (Generality): The detection method should have general detection capabilities for different types of triggers (pixel, physical) and different attack strategies, without needing to be adjusted for specific attacks.

[0036] Objective 5 (No-Training Deployment): The detection system should be deployable without requiring additional neural network training or parameter updates, simply by building a reference feature index, achieving plug-and-play functionality.

[0037] The VLA model employs a cascaded architecture to process multimodal information. Taking OpenVLA as an example, the information flow sequentially passes through three core modules: the Vision Backbone (such as SigLIP) encodes the input image into visual features; the Language Model (such as LLaMA-7B) fuses visual features and language commands to generate hidden representations; and the Action Head decodes the hidden representations into robot action sequences. This application extracts features at these three locations and compares and analyzes the differences in feature distribution between clean samples and trigger samples.

[0038] For the Vision Backbone output, this application extracts the hidden state of the last layer of the visual encoder and obtains a fixed-dimensional feature vector through global average pooling. For the Language Model, this application extracts the average representation of the corresponding text portion in the last layer's hidden state. For the Action Token, this application extracts the hidden state for predicting the action token position. All features are L2 normalized to eliminate amplitude differences, allowing the analysis to focus on changes in feature orientation. like Figure 2 As shown, the experimental results reveal an important finding: the feature difference between trigger samples and clean samples is most significant in the Vision Backbone output layer, reaching an AUROC of 1.0 (perfect separation), while it drops to 0.85 in the Language Model layer and further to 0.72 in the Action Token layer. This phenomenon can be understood from the perspective of information propagation: Vision Backbone directly processes the raw pixel input, and the trigger pattern (whether pixel perturbation or physical object) produces the strongest feature response here; as information flows through the 7 billion parameters of the Language Model, the trigger signal is gradually "diluted" by a large number of language modeling parameters, and the feature difference weakens accordingly.

[0039] This finding has significant practical implications. First, it demonstrates that the Vision Backbone is the optimal entry point for backdoor detection, where the highest detection performance can be achieved. Second, feature extraction using the Vision Backbone requires only one forward propagation, resulting in computational overhead far lower than full model inference, thus meeting real-time requirements. Finally, in practical deployments, the Vision feature interface is typically accessible (for multi-task sharing, feature caching, or debugging), allowing the proposed method to operate under black-box conditions without accessing the parameters or gradients of the full model.

[0040] Based on the above findings, this application proposes FDS-Defense (Feature Distribution ShiftDefense), which formalizes the backdoor detection problem as an out-of-distribution (OOD) detection problem. The core idea is that the vision features of clean samples follow a certain unknown but fixed distribution. The characteristics of the trigger samples deviate from this distribution; by quantifying the degree of deviation of the test samples relative to the clean distribution, potential backdoor attacks can be identified.

[0041] Why choose KNN distance as the offset metric? This application requires a metric that can characterize whether a sample belongs to a reference distribution. Traditional parametric methods (such as assuming a Gaussian distribution and calculating Mahalanobis distance) require strong assumptions about the distribution form, while the true distribution in a high-dimensional feature space is often complex and non-Gaussian. K-Nearest Neighbors (KNN) distance provides a non-parametric alternative: given a reference set R consisting of clean sample features, the anomaly score of a test sample f is defined as the Euclidean distance from f to its Kth nearest neighbor in R. The theoretical basis of this metric lies in the asymptotic relationship between KNN distance and local probability density: when the reference set is sufficiently large, the KNN distance is inversely proportional to the probability density at that point. Intuitively, samples located in high-density regions (the center of the clean distribution) have smaller KNN distances, while samples located in low-density regions (the edges or outside the distribution) have larger KNN distances. Trigger samples whose features deviate from the clean distribution fall into low-density regions, resulting in a significantly increased KNN distance.

[0042] Another advantage of the KNN method is that it requires no training. Unlike classifiers that need to learn decision boundaries, KNN only needs to build an index structure for fast nearest neighbor queries. With the help of efficient vector retrieval libraries such as FAISS, this application can complete queries in milliseconds, meeting the real-time requirements of online detection.

[0043] How to determine the threshold? This is the core issue of OOD detection. Traditional methods usually require accessing both normal and abnormal samples simultaneously to determine the optimal threshold. However, in backdoor detection scenarios, the defender does not know the specific form of the trigger and cannot obtain trigger samples. This application proposes a threshold determination strategy based solely on clean samples: First, the leave-one-out KNN distance distribution of clean samples is calculated (i.e., for each clean sample, its KNN distance to the other clean samples is calculated), and then the threshold is set to the mean of this distribution plus λ times the standard deviation, i.e., τ = μ + λσ.

[0044] The theoretical basis for this design is as follows. Under the assumption of a normal distribution, the probability of exceeding μ + λσ is 1 - Φ(λ), where Φ is the cumulative distribution function of the standard normal distribution. Therefore, λ directly controls the false alarm rate: λ=3 corresponds to a false alarm rate of 0.13%, λ=4 corresponds to 0.003%, and λ=5 corresponds to 0.00003%. In practice, this application chooses λ=5 as the default value, which means that false alarms are almost non-existent during normal operation, while maintaining a high detection rate for anomalous samples.

[0045] Why is this threshold design effective for trigger samples? The key lies in the significant separation between the KNN distance distributions of clean samples and trigger samples. Let the KNN distance distribution of clean samples be N(μ, σ²), and the distribution of trigger samples be N(μ, σ²). Experiments in this application show that, in the Vision Backbone feature space, The separation is significantly greater than μ (the resolution exceeds 10σ), which means that even with a conservative threshold (such as μ + 5σ), the KNN distance of the trigger samples still far exceeds the threshold, and the detection rate is close to 100%. This separation is the reason why this application chose the Vision Backbone as the detection location—in other locations, the separation is smaller, making it difficult to achieve both low false alarm rate and high detection rate at the same time.

[0046] The complete FDS-Defense process consists of two stages: offline index building (Algorithm 1) and online backdoor detection (Algorithm 2).

[0047] Algorithm 1 describes the complete process of offline index building. This algorithm only needs to be executed once, before deployment.

[0048] First, for the clean sample set Each sample in the image is processed by the Vision Backbone encoder. Visual features are extracted and L2 normalized to eliminate differences in feature amplitude.

[0049] The normalized feature set F is used to build the FAISS index, which supports efficient nearest neighbor queries.

[0050] Next, the Kth nearest neighbor distance for each clean sample is calculated using a leave-one-out strategy: for features From the remaining feature set Query the distance of its Kth nearest neighbor After collecting the KNN distances for all M samples, the distribution statistics—mean μ and standard deviation σ—are calculated. The detection threshold τ is set to μ + λσ, where λ controls the false alarm rate. The final output includes the constructed KNN index and the detection threshold, for use in the online detection phase.

[0051] The online backdoor detection phase of Algorithm 2 is the core operation of FDS-Defense. The reference index built in the offline phase provides a benchmark for clean feature distribution for online detection. During the robot's task execution, each time a new observation image is received, the system needs to determine in real time whether the input contains a backdoor trigger. Algorithm 2 describes the specific implementation of this online detection process, including three steps: feature extraction, anomaly score calculation, and temporal aggregation decision based on a sliding window. The sliding window mechanism is introduced to reduce the false alarm rate by utilizing temporal continuity—real backdoor attacks usually persist across multiple consecutive frames, while anomalies caused by noise are often sporadic.

[0052] Algorithm 2 describes the real-time process of online detection, which is executed every time a new observation image is received.

[0053] First, for the current frame image Extract the Vision Backbone features and perform L2 normalization to obtain the feature vector. .

[0054] Then, query using the pre-built FAISS index. Find the K nearest neighbors and take the distance of the Kth nearest neighbor as the anomaly score. The higher the anomaly score, the more the current sample deviates from the clean distribution. To quantify the degree of deviation, a normalized confidence score is calculated. Positive values ​​indicate that the detection threshold is exceeded (suspicious), while negative values ​​indicate that the threshold is below (normal).

[0055] To reduce the false alarm rate and utilize temporal information, the algorithm maintains a sliding window queue Q of size W, storing the confidence scores of the most recent W frames. Each time, the current confidence score is... Add to the tail of the queue; if the queue length exceeds W, remove the oldest confidence score. Make a decision only when the window is full (|Q| = W): calculate the average of the window's built-in confidence scores. ,like If the value exceeds the window threshold θ (default 0.35), it is identified as a backdoor attack; otherwise, it is considered a clean sample. This sliding window mechanism ensures that an alarm is triggered only when multiple consecutive frames exhibit abnormal characteristics, effectively filtering out occasional noise interference. The entire online detection process takes only about 3.8ms, meeting the real-time requirements of a 30Hz control frequency.

[0056] Algorithm complexity analysis. The offline stage has a time complexity of O(M·D) for feature extraction, O(M·K·logM) for KNN distance calculation (using FAISS's approximate nearest neighbor algorithm), and a space complexity of O(M·D) for storing feature indices. The online stage has a time complexity of O(D) for feature extraction, O(K·log M) for KNN lookup, and O(W) for sliding window updates, for a total time complexity of O(D + K·log M). For a typical configuration (M=3000, D=1152, K=50), the single-frame detection time is approximately 3.8ms, which meets the real-time requirements.

[0057] Although single-frame detection based on KNN distance already has a very low false alarm rate, in scenarios where robots perform tasks continuously for extended periods, even a false alarm rate of 0.01% can accumulate over hours, generating multiple false alarms and impacting task execution. This application observes that real backdoor attacks exhibit temporal persistence—once a trigger appears in the scene, it typically persists for multiple consecutive frames; while false alarms caused by noise are often sporadic and isolated. Based on this observation, this application introduces a sliding window mechanism to further reduce the false alarm rate by utilizing temporal information.

[0058] Specifically, this application maintains a sliding window of size W, recording the detection confidence of the most recent W frames. The confidence of a single frame is defined as the normalized offset of the KNN distance relative to a threshold: c = (s - τ) / τ, where a positive value indicates exceeding the threshold and a negative value indicates falling below the threshold. The average confidence of the window is the arithmetic mean of the confidence of the W frames. Only when the window is full and the average confidence exceeds a preset threshold θ is it determined to be a backdoor attack.

[0059] The effectiveness of this design can be analyzed from a probabilistic perspective. Assuming the detection results of each frame are independent, and the false alarm probability of a single frame is p, then the probability of false alarms for W consecutive frames is p^W. When p=0.001 and W=3, the false alarm rate of the window drops to 10^(-9), which is almost negligible. Of course, the sliding window also introduces a detection delay—it requires waiting for W frames to make a judgment. If the single-frame detection rate is q, and the trigger appears continuously, the expected number of frames for the first detection is approximately W + (1-q) / q. When q is close to 1 (which the method in this application satisfies), the detection delay mainly depends on the trigger type. In practice, this application chooses W=3, and the detection delay for the Pixel trigger is approximately 5.75 frames (approximately 190ms), while the physical trigger requires even longer (approximately 19.3 frames for a Stick and approximately 57.4 frames for a Mug), which is acceptable for robot operation tasks.

[0060] The sliding window mechanism embodies a fundamental trade-off in detection system design: the trade-off between false alarm rate and detection latency. Increasing the window size can exponentially reduce the false alarm rate, but linearly increases the detection latency. The default configuration in this application (W=3, θ=0.35) achieves a good balance between the two, ensuring an extremely low false alarm rate while keeping the detection latency within an acceptable range.

[0061] Briefly explain the theoretical basis for each design choice in FDS-Defense; detailed mathematical proofs can be found in Supplementary Material A.

[0062] The theoretical basis of KNN distance: KNN distance is inversely proportional to local probability density: samples in high-density regions have smaller KNN distances, while samples in low-density regions (such as out-of-distribution samples) have significantly larger KNN distances. This provides a theoretical guarantee for detecting backdoor samples based on KNN distance.

[0063] Statistical guarantees for threshold design. Using τ = μ + λσ as the detection threshold, under the assumption of normal distribution, λ directly controls the false positive rate: λ=3 corresponds to FPR=0.13%, λ=4 corresponds to FPR=0.003%, and λ=5 corresponds to FPR=0.00003%. Experiments in this application show that the separation between clean samples and trigger samples in the Vision Backbone feature space exceeds 10σ, far exceeding the required separation condition.

[0064] K value selection. The theoretical optimal K value is... For the settings of this application (M=3000, D=1152), K=50 achieves a balance between bias and variance. Experiments show that K is stable in the range of 20-100.

[0065] The necessity of L2 normalization. Normalized Euclidean distance is equivalent to angular distance, eliminating the influence of feature amplitude and allowing detection to focus on changes in feature direction. Backdoor triggers primarily change feature direction rather than amplitude; normalization improves detection performance. This provides theoretical support for the design choices of FDS-Defense, including the theoretical basis of KNN distance as an OOD metric, the statistical guarantee of threshold design, the bias-variance trade-off in K value selection, and the necessity of L2 normalization.

[0066] The relationship between KNN distance and probability density. Let the reference set R consist of M samples independently and identically distributed from the probability density function p(x), and the K-nearest neighbor distance of the query point x be d_K(x). Classical KNN density estimation theory shows that as M approaches infinity... Converging in probability to ,in Let be the volume of a D-dimensional unit sphere, where D is the feature dimension. The intuitive implication of this result is that the KNN distance is inversely proportional to the D-th root of the local probability density. In high-density regions, samples are tightly clustered, resulting in a small KNN distance; in low-density regions, samples are sparse, resulting in a large KNN distance.

[0067] This theoretical result provides a solid foundation for the method in this application. The features of the trigger samples deviate from the clean distribution, falling into the low-density region (or even out-of-distribution) of the clean distribution, thus exhibiting a significantly increased KNN distance. Furthermore, it can be proven that the KNN distance-based detector is a nonparametric approximation of the likelihood ratio test. According to the Neyman-Pearson lemma, the likelihood ratio test is the optimal detector given a false alarm rate constraint. Therefore, the KNN distance detector has a theoretical guarantee of optimality.

[0068] Statistical guarantees for threshold design. This application uses τ = μ + λσ as the detection threshold, where μ and σ are the mean and standard deviation of the KNN distance distribution of clean samples, respectively. Under the assumption of normal distribution, the false alarm rate FPR = 1 - Φ(λ), where Φ is the cumulative distribution function of the standard normal distribution. This formula provides a precise correspondence between λ and the false alarm rate: λ=3 corresponds to FPR=0.13%, λ=4 corresponds to FPR=0.003%, and λ=5 corresponds to FPR=0.00003%.

[0069] Detection rate analysis needs to consider the KNN distance distribution of trigger samples. Let the KNN distance of trigger samples follow a distribution N(…). , If ), then the detection rate TPR = 1 - Φ[(μ + λσ - ) / ].when When TPR is significantly greater than μ + λσ, it approaches 1. This application defines a "separation condition": if ( If -μ) / σ>2λ, then TPR>1 - Φ(λ) and FPR<1 - Φ(λ) can be achieved simultaneously. Experiments in this application show that the separation degree in the Vision Backbone feature space exceeds 10σ, far exceeding the requirements of the separation condition, which explains why the method in this application can achieve near-perfect detection performance.

[0070] The choice of K value involves a bias-variance trade-off. KNN distance estimation suffers from a bias-variance trade-off: too small a K value leads to high variance (sensitivity to noise), while too large a K value leads to high bias (oversmoothing). Theoretical analysis shows that the optimal K value is... Where M is the size of the reference set and D is the feature dimension. For the settings in this application (M=3000, D=1152), the theoretically optimal K is approximately 30-100. This application chooses K=50 as the default value to achieve a balance between bias and variance. Experiments show that K has little impact on detection performance in the range of 20-100, indicating that the method is robust to the choice of K value.

[0071] The necessity of L2 normalization. This application performs L2 normalization on all features, i.e., f̂ = f / ‖f‖_2. The normalized Euclidean distance and cosine similarity have a simple mathematical relationship: , where θ is the angle between the two original feature vectors. This means that L2 normalization transforms the Euclidean distance into an angular distance, eliminating the influence of feature amplitude and allowing detection to focus on changes in feature orientation. This design is particularly important for backdoor detection: the analysis in this application shows that triggers primarily change the orientation of features rather than their amplitude, and the normalized distance metric can better capture this change.

[0072] Consistency Guarantee. Finally, this application proves that FDS-Defense has asymptotic consistency: as the reference set size M approaches infinity, if the clean distribution P_c differs from the trigger distribution P_t, the detection rate TPR approaches 1, and the false positive rate FPR approaches 0. This result guarantees that the method can theoretically achieve perfect detection, provided that a sufficient number of clean samples are used to construct the reference set. In practice, this application found that 3000 samples are sufficient to achieve near-perfect detection performance.

[0073] Experimental setup VLA Model and Dataset. This application uses OpenVLA-7B as the baseline VLA model. OpenVLA is built on the Prismatic VLM architecture, using SigLIP as the Vision Backbone to extract visual features (output dimension 1152), and LLaMA-2-7B as the Language Model for multimodal inference. The Action Head decodes the hidden states into 7-dimensional robot actions (end-effector position, pose, and gripper state). The model is pre-trained on the Open-X-Embodiment dataset and fine-tuned on the LIBERO task.

[0074] The experiments were evaluated on the LIBERO robot manipulation simulation benchmark. LIBERO is a widely used desktop manipulation benchmark that includes four task suites: LIBERO-10 (10 long horizontal complex tasks), LIBERO-Goal (goal-oriented tasks), LIBERO-Object (object manipulation tasks), and LIBERO-Spatial (spatial reasoning tasks). Each task suite contains 10 different manipulation scenarios, covering diverse skills such as grasping, placing, and opening / closing drawers. The dataset is stored in RLDS format and includes dual-view image inputs from a third-person camera and a wrist camera, as well as the robot's proprioceptive state.

[0075] Backdoor attack setup. The attacked VLA model is trained using the BadVLA framework, employing a two-stage training process optimized for target decoupling. This application evaluates the defensive effectiveness of FDS-Defense against three trigger types across all four task suites, encompassing a total of 12 attack scenarios: • Pixel Trigger: Adds a solid-color square, 10% of the image area (approximately 32×32 pixels), to the center of the image. The square is white (RGB=255). This type of trigger simulates a backdoor injection attack in the digital domain and is visually relatively stealthy. All four task suites deploy this type of trigger.

[0076] • Physical Trigger - Mug: Place a mug as a trigger in the robot's work environment. This type of trigger is deployed in all four task suites.

[0077] • Physical Trigger - Red Stick: Place a red stick-shaped object as a trigger in the robot's work environment. This type of trigger is deployed in all four task suites.

[0078] Physical triggers pose a greater practical threat, as attackers can activate backdoors by placing everyday objects in the physical environment, and these triggers are difficult to eliminate through input preprocessing methods. The attacked model maintains normal task performance (success rate >75%) on clean inputs, but the attack success rate exceeds 90% when triggers are present, verifying the effectiveness of the backdoor attack. The defense method in this application performs detection during the inference phase and does not involve the model training process.

[0079] Evaluation metrics. This application uses four core metrics to evaluate the effectiveness of the defense methods: • Success Rate (SR): The percentage of tasks completed with clean input (no triggers), measuring the impact of defense on normal tasks. • Attack Success Rate (ASR): The percentage of malicious actions performed when the trigger exists, measuring the effectiveness of the defense (lower is better). • Detection Rate (DR): The proportion of samples that correctly identify the trigger, i.e., the true positive rate (TPR). • False Positive Rate (FPR): The proportion of clean samples that are mistakenly identified as triggers (the lower the better). To comprehensively evaluate the defensive effectiveness of FDS-Defense, this application tests the performance of the attacked model under two conditions: (1) No trigger condition: Test the task success rate using clean input (without any triggers), and measure: • Baseline performance of the attacked model (to verify the stealth of the attack; SR should be >75%) • The impact of defense mechanisms on normal missions (ideally, SR decrease <1%) (2) Trigger Condition: Use input with triggers to test the attack success rate and measure it: • Effectiveness of backdoor attacks (to verify a successful attack, ASR should be >90%) • The effectiveness of the defense mechanism (ideally, ASR should be reduced to <5%) Implementation details. The default parameter settings for FDS-Defense are: K-nearest neighbors K=50, standard deviation factor λ=5 (threshold τ=μ+5σ), sliding window size W=3, window confidence threshold θ=0.35, and reference set size M=3000. Feature extraction uses the output of VisionBackbone (SigLIP), with a feature dimension of 1152, and all features are L2 normalized. The KNN index is implemented using the IndexFlatL2 method from the FAISS library. Experiments were conducted on a server equipped with an NVIDIA A100 GPU, with each task running 10 trials and the average taken.

[0080] Main results (W=3, θ=0.35, these two represent hyperparameters and their analysis needs further clarification): Since backdoor defense against VLA models is an emerging research area, there are currently no specific defense methods for VLA available for comparison. Therefore, this application focuses on demonstrating the defense performance of FDS-Defense under 12 attack scenarios (4 task suites × 3 trigger types) and verifies the effectiveness of each design choice through ablation experiments. All major results reported in this application are based on the following default hyperparameter configurations: sliding window size W=3, window confidence threshold θ=0.35, K-nearest neighbors K=50, and standard deviation factor λ=5. The reasons for the selection of these hyperparameters will be analyzed in detail in the ablation experiments in Section 5.3. Among them, the sliding window size W=3 strikes a balance between false alarm rate (achieving zero false alarms) and detection latency (approximately 1.8 frames); the window confidence threshold θ=0.35 ensures that an alarm is triggered only when multiple consecutive frames exhibit abnormal characteristics, effectively filtering transient noise.

[0081] Overall defensive effect Table 1 shows the complete defense performance of FDS-Defense under all attack scenarios, including both cases with and without triggers.

[0082] Table 1 Table Explanation: • Success Rate (SR) without triggers: The success rate of a task on clean input. No defense: Baseline performance of the attacked model without triggers (84.00%-100.00%, average 95.64%, verifying attack stealth). Defense: Performance after enabling FDS-Defense (84.00%-100.00%, average 95.25%, very close to the value without defense) • Attack Success Rate (ASR) with triggers: The success rate of executing malicious actions when the trigger exists (the lower the better). No defense: Effectiveness of backdoor attack (100.00%, verifying complete success of the attack) Defense: Residual attack rate after enabling defense (0.00%, completely suppressing attacks) • Detection performance: DR (Detection Rate): The percentage of correctly identified trigger samples (98.00%-100.00%, average 99.67%). FPR (False Positive Rate): The proportion of clean samples that are mistakenly identified as triggers (all are 0.00%). Key findings: (1) Attack success rate is reduced to zero. After enabling FDS-Defense, the attack success rate dropped from 100% to 0%, achieving perfect defense. In all 12 experimental scenarios, the ASR after defense dropped to 0.00%, indicating that FDS-Defense can completely block the execution path of backdoor attacks, effectively suppressing both pixel-level triggers (Pixel) and physical triggers (Mug, Stick).

[0083] (2) The detection rate is extremely high and stable. The detection rate for all trigger types reaches 98%-100% (average 99.67%). The pixel trigger has a detection rate of 100.00% across all task suites; among physical triggers, the detection rate for the mug trigger is 98%-100%, and the stick trigger is 98%-100%. Only the stick trigger of LIBERO-10 and the mug trigger of LIBERO-Spatial have a detection rate of 98%, slightly lower than other scenes. This may be related to the appearance changes of physical objects at specific viewpoints, but it still remains at an extremely high level.

[0084] (3) Zero false alarm rate. In all 12 scenarios tested, FDS-Defense did not produce any false alarms (FPR=0.00%). This is due to: ① statistical threshold selection (λ=5 corresponds to a 5σ confidence interval, theoretical false alarm probability≈0); ② the smoothing mechanism of the sliding window filters out instantaneous anomalies; ③ L2 normalization eliminates the influence of feature amplitude fluctuations.

[0085] (4) Minimal impact on normal tasks. Without triggers, the task success rate drops from an average of 95.56% to 95.25%, an impact of only 0.31 percentage points. In some scenarios (such as Pixel and Stick in LIBERO-10, and all scenarios in LIBERO-Object), there is even a 1% performance improvement or no change. This minimal impact mainly comes from the slight delay caused by the computational overhead of the detection system, rather than erroneous termination due to false alarms (FPR=0% ensures no false termination). More importantly, a mere 0.31% decrease in SR results in 100% elimination of ASR, demonstrating extremely significant defensive benefits.

[0086] Table 2 summarizes the average defense performance of different trigger types, facilitating horizontal comparison.

[0087] Table 2: Average Defense Effectiveness of Different Trigger Types Analysis: Pixel triggers are the easiest to detect (DR=100%, detection latency of only 5.75 frames) because they produce the most significant and consistent offset in the feature space. An FPR of 0% indicates no false alarms. Physical triggers (Mug and Stick) also demonstrate excellent detection performance, with an average DR of 99.5%, validating the method's versatility across different trigger types. The average detection latency for Stick (19.3 frames) is lower than for Mug (57.4 frames), possibly because the color features of the red stick are more prominent. Mug has the longest detection latency, likely because mugs are common items with minimal feature differences from normal scenes, requiring more frames for the system to accumulate confidence. All trigger types have an FPR of 0%, indicating that the defense system will not produce false alarms.

[0088] Detection Latency Analysis: Significant differences exist in the detection latency of the three triggers: the Pixel trigger requires an average of 5.75 frames to detect, the Stick trigger requires 19.3 frames, and the Mug trigger requires 57.4 frames. At a 30Hz control frequency, the detection latency of the Pixel trigger is approximately 190ms (5.75 frames × 33ms / frame), the Stick trigger is approximately 637ms, and the Mug trigger is approximately 1.9 seconds. This indicates that there is still room for optimization in the response speed of the defense system against physical triggers that resemble common objects (such as mugs). Despite the longer detection latency of physical triggers, the ASR (Asynchronous Response Rate) drops to 0% after defense, indicating that the system ultimately successfully blocked all attacks.

[0089] Example 2: To achieve the above objective, such as Figure 3 As shown, based on Embodiment 1, this invention discloses a VLA model backdoor detection system based on feature distribution offset, comprising: The feature extraction module 11 is used to receive the image to be detected, input the image to be detected into a pre-established VLA model, and output the visual feature vector extracted by the visual encoder based on the VLA model. The feature offset module 12 is used to receive a reference feature index and calculate a feature distribution offset based on the reference feature index and the visual feature vector. The feature distribution offset is the distance of the visual feature vector to the Kth nearest neighbor in the reference feature index. The reference feature index is extracted based on the visual encoder through a pre-constructed clean sample set. The backdoor detection module 13 is used to compare the feature distribution offset with a preset offset threshold. If the feature distribution offset is greater than the preset offset threshold, the image to be detected is determined to be a backdoor attack image; otherwise, the image to be detected is determined to be a clean image.

[0090] Based on the same inventive concept, this invention also provides a computer device, comprising: one or more processors, and a memory for storing one or more computer programs; the programs include program instructions, and the processor executes the program instructions stored in the memory. The processor may be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. It is the computing and control core of the terminal, used to implement one or more instructions, specifically for loading and executing one or more instructions stored in a computer storage medium to implement the above-described method.

[0091] It should be further explained that, based on the same inventive concept, the present invention also provides a computer storage medium storing a computer program, which, when executed by a processor, performs the above-described method. This storage medium can be any combination of one or more computer-readable media. The computer-readable medium can be a computer-readable signal medium or a computer-readable storage medium. The computer-readable storage medium can be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of computer-readable storage media (a non-exhaustive list) include: an electrical connection having one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof. In the present invention, the computer-readable storage medium can be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, apparatus, or device.

[0092] In the description of this specification, references to terms such as "an embodiment," "example," "specific example," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of this disclosure. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples.

[0093] The foregoing has shown and described the basic principles, main features, and advantages of this disclosure. Those skilled in the art should understand that this disclosure is not limited to the above embodiments. The embodiments and descriptions in the specification are merely illustrative of the principles of this disclosure. Various changes and modifications can be made to this disclosure without departing from its spirit and scope, and all such changes and modifications fall within the scope of this disclosure as claimed.

Claims

1. A backdoor detection method based on VLA model with feature distribution shift, characterized in that, The method includes the following steps: Receive the image to be detected, input the image to be detected into a pre-established VLA model, and output the visual feature vector extracted by the visual encoder based on the VLA model; Receive a reference feature index, and calculate the feature distribution offset based on the reference feature index and the visual feature vector, wherein the feature distribution offset is the distance of the visual feature vector to the Kth nearest neighbor in the reference feature index, and the reference feature index is extracted based on the visual encoder through a pre-constructed clean sample set; The feature distribution offset is compared with a preset offset threshold. If the feature distribution offset is greater than the preset offset threshold, the image to be detected is determined to be a backdoor attack image; otherwise, the image to be detected is determined to be a clean image.

2. The VLA model backdoor detection method based on feature distribution offset according to claim 1, characterized in that, The pre-built VLA model is generated using a cascaded architecture for vision, language, and action, including a Vision visual encoder, a Language model, and an Action head.

3. The VLA model backdoor detection method based on feature distribution shift according to claim 2, characterized in that, The Vision visual encoder uses a pre-trained visual Transformer to extract visual feature vectors from a given image to be detected. The Language model uses a pre-trained large language model as the backbone network. It receives visual feature vectors and task descriptions corresponding to the images to be detected, and outputs the fused hidden state. The Action header maps the fused hidden states to a sequence of robot actions. The actions are discretized into a sequence of tokens and generated through an autoregressive approach.

4. The VLA model backdoor detection method based on feature distribution shift according to claim 3, characterized in that, The visual feature vector The extraction formula is as follows: In the formula: The visual feature vector is an N× A 3D real matrix; Here, I is the input image to be detected, R represents the real number field (the set of all real numbers), and N is the number of image patches. For visual feature dimensions; The formula for processing the merged hidden state h is as follows: In the formula: h is the fused hidden state vector; LLM is the large language model function. is the visual feature vector; l is the language instruction; R represents the real number field; T is the total length of the sequence. For the hidden layer dimension, Indicates the dimension of the output hidden state matrix; The output formula for robot action sequence a is as follows: In the formula: a is a robot action sequence vector, which is a... A 3D real matrix; Here, h is the action decoder function, h is the fused hidden state vector, and R represents the real number field. K is the length of the action sequence; For the action dimension, This indicates the dimension of the output action matrix, containing K rows. The columns, where each row represents the complete action vector for a time step.

5. The VLA model backdoor detection method based on feature distribution offset according to claim 4, characterized in that, The expression of the pre-established VLA model is as follows: In the formula: a is the output action sequence of the VLA model; M is the global mapping function of the VLA model; I is the input image; l is the language instruction. For visual encoder functions; For large language model functions; This is the action decoder function.

6. The backdoor detection method for VLA models based on feature distribution offset according to claim 1, characterized in that, The feature distribution offset is generated using a nearest neighbor search algorithm to obtain the distance of the Kth nearest neighbor. The distance of the Kth nearest neighbor is inversely proportional to the magnitude of the local probability density.

7. The backdoor detection method for VLA models based on feature distribution offset according to claim 1, characterized in that, The process for setting the preset offset threshold is as follows: Calculate the leave-one-out KNN distance distribution for clean samples. For each clean sample, calculate its KNN distance to the other clean samples. Add λ times the standard deviation to the mean of the leave-one-out KNN distance distribution of the clean samples to obtain the offset threshold; the formula is as follows: In the formula, μ and σ are the mean and standard deviation of the distance distribution of the Kth nearest neighbor of the visual features of the clean sample in the reference feature index, respectively, and λ is a preset multiple.

8. The backdoor detection method for VLA models based on feature distribution offset according to claim 1, characterized in that, The extracted visual feature vectors are then L2 normalized, and the normalization confidence is as follows: 。 9. A VLA model backdoor detection system based on feature distribution offset, employing the VLA model backdoor detection method based on feature distribution offset as described in any one of claims 1 to 8, characterized in that, include: The feature extraction module is used to receive the image to be detected, input the image to be detected into a pre-established VLA model, and output the visual feature vector extracted by the visual encoder based on the VLA model. The feature offset module is used to receive a reference feature index and calculate the feature distribution offset based on the reference feature index and the visual feature vector. The feature distribution offset is the distance of the visual feature vector to the Kth nearest neighbor in the reference feature index. The reference feature index is extracted based on the visual encoder through a pre-constructed clean sample set. The backdoor detection module compares the feature distribution offset with a preset offset threshold. If the feature distribution offset is greater than the preset offset threshold, the image to be detected is determined to be a backdoor attack image; otherwise, the image to be detected is determined to be a clean image.

10. A terminal device, comprising a memory, a processor, and a computer program stored in the memory and capable of running on the processor, characterized in that, The memory stores a computer program that can run on a processor. When the processor loads and executes the computer program, it employs the VLA model backdoor detection method based on feature distribution offset as described in any one of claims 1 to 8.

Images