Safety evaluation methods, devices and electronic equipment for visual language action models

By inputting initial and perturbed video data, calculating motion offsets, filtering effective perturbation patches, determining external decision boundary parameters, and iteratively optimizing candidate patches, this method solves the problems of low efficiency and unstable evaluation in existing visual language action model security evaluation methods in black-box systems, and achieves efficient and robust security evaluation.

CN123116284APending Publication Date: 2026-11-17CHINA MOBILEHANGZHOUINFORMATION TECH CO LTD +1
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202611278077.4
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2026-08-21
Publication Date
2026-11-17

AI Technical Summary

Technical Problem

Existing security assessment methods for visual language action models are mostly geared towards white-box or gray-box scenarios, making it difficult to adapt to black-box systems. Attack designs rely on preset physical constraints, resulting in redundant optimization space, low search efficiency, and failure to consider the temporal continuity of video, leading to large fluctuations in assessment results.

Method used

By inputting initial video data and multiple sets of perturbed video data into the visual language action model, the action offset is calculated, effective perturbation patches are screened, externally observable decision boundary parameters are determined, and candidate patches are iteratively optimized using the cross-entropy algorithm and scoring function to generate security evaluation results without relying on internal model information.

Benefits of technology

It achieves efficient security evaluation of black-box models, enhances the physical concealment and attack robustness of anti-patch measures, improves optimization efficiency, and reduces computational overhead.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN123116284A_ABST
    Figure CN123116284A_ABST
Patent Text Reader

Abstract

This invention provides a security evaluation method, apparatus, and electronic device for visual language action models. The method includes: inputting initial video data into the visual language action model to obtain baseline action vectors; inputting multiple sets of perturbed video data into the visual language action model to obtain multiple sets of perturbed action vectors; calculating the action offset of each set of perturbed action vectors relative to the baseline action vectors, filtering out a set of effective perturbed patches that cause task failure, determining externally observable decision boundary parameters based on the distribution of action offsets corresponding to the effective perturbed patch set; initializing a candidate patch set using the decision boundary parameters as constraints, iteratively optimizing the candidate patch set based on the cross-entropy algorithm and a scoring function, outputting the candidate patch with the highest score as the adversarial patch, and generating a security evaluation result. This invention, by constraining the optimization space through decision boundary parameters, can achieve security evaluation of black-box models and improve optimization efficiency.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] This invention relates to the field of artificial intelligence security technology, and in particular to a security evaluation method, apparatus and electronic device for visual language action models. Background Technology

[0002] Existing security evaluation schemes for Vision-Language-Action (VLA) models are mostly geared towards white-box or gray-box scenarios, relying on the model's internal structure, gradient information, or intermediate feature outputs. This makes them difficult to adapt to black-box VLA systems that provide services via application programming interfaces (APIs) in real-world deployments. While some research has explored VLA adversarial attack paradigms, they generally suffer from limitations: attack design relies on pre-set physical constraints, making it impossible to accurately locate model vulnerabilities beforehand, and transferability is limited when facing unknown models; optimization space redundancy is low, search efficiency is low, and physical concealment constraints are ignored, making them difficult to adapt to real-world robotic task scenarios. Furthermore, existing methods are mostly designed for single-frame images, failing to consider the impact of video temporal continuity on the stability of evaluation results, leading to large fluctuations in evaluation results. Summary of the Invention

[0003] This invention provides a method, apparatus, and electronic device for the safety evaluation of visual language action models, which addresses the shortcomings of existing methods for the safety evaluation of visual language action models, such as reliance on internal model information and low optimization search efficiency.

[0004] This invention provides a safety evaluation method for visual language action models, including: The initial video data is input into the visual language action model to be evaluated to obtain the baseline action vector output by the visual language action model. Multiple sets of perturbation video data are input into the visual language action model to obtain multiple sets of perturbation action vectors output by the visual language action model. Each set of perturbation video data is obtained by perturbing the initial video data with multiple corresponding perturbation patches. Calculate the action offset of each group of perturbation action vectors relative to the baseline action vector, and filter out the effective perturbation patch set that causes task failure from the multiple perturbation patches in combination with the task failure criteria. Based on the distribution of each action offset corresponding to the effective perturbation patch set, determine the external observable decision boundary parameters of the visual language action model. Using the externally observable decision boundary parameters as constraints, a candidate patch set is initialized based on the effective perturbation patch set. The candidate patch set is iteratively optimized based on the cross-entropy algorithm and the constructed scoring function until the convergence condition is met. The candidate patch with the highest score is output as the adversarial patch, and the security evaluation result of the visual language action model is generated.

[0005] In some embodiments, the externally observable decision boundary parameters include an attack success threshold, a minimum effective perturbation intensity, and a sensitive background region distribution.

[0006] In some embodiments, determining the external observable decision boundary parameters of the visual language action model based on the distribution of action offsets corresponding to the effective perturbation patch set includes treating the perturbation test of the initial video data as a test round and repeating the test multiple times; in each test round, the following steps are performed: The action offsets corresponding to the effective perturbation patch set selected in this round are sorted from smallest to largest, and the lower quartile value after sorting is taken as the success threshold of this round of attack. Within a preset disturbance intensity range, the disturbance intensity of the test patch is gradually increased by a preset step size. The proportion of test patches that cause task failure under each disturbance intensity to the total number of test patches is counted. The disturbance intensity that first makes the proportion reach a preset success rate threshold is determined as the minimum effective disturbance intensity of this round. The execution results of multiple rounds of testing are summarized, and the average value of the success threshold of each round of attack obtained from multiple rounds of testing is determined as the success threshold of attack, and the average value of the minimum effective perturbation strength of each round of attack obtained from multiple rounds of testing is determined as the minimum effective perturbation strength. The location distribution of each perturbation patch with an action offset greater than or equal to the attack success threshold is statistically analyzed in multiple rounds of testing, and the area where each perturbation patch is concentrated is determined as the sensitive background area distribution.

[0007] In some embodiments, the iterative optimization of the candidate patch set based on the cross-entropy algorithm and the constructed scoring function includes iteratively executing the following steps until the convergence condition is met: The score for each candidate patch in the candidate patch set is calculated based on the scoring function. Based on the scores, all candidate patches in the candidate patch set are sorted, and a predetermined number of top-ranked candidate patches are selected to form an elite sample set. For the parameter distribution of each candidate patch in the elite sample set, fit a probability distribution model; New candidate patches are generated by sampling from the probability distribution model, and the candidate patch set is updated.

[0008] In some embodiments, calculating the score for each candidate patch in the candidate patch set based on the scoring function includes: The candidate perturbation video data is input into the visual language action model to obtain the candidate perturbation action vector output by the visual language action model. The candidate perturbation video data is obtained by perturbing the initial video data based on the candidate patch. Calculate the candidate action offset of the candidate perturbation action vector relative to the baseline action vector, calculate the difference between the candidate action offset and the attack success threshold, and obtain the basic score; Obtain the texture complexity, size parameter, and position parameter of the candidate patch; calculate the texture penalty value based on the texture complexity; calculate the size penalty value based on the size parameter; and calculate the position penalty value based on the position parameter. Determine the first weight of the texture penalty value, the second weight of the size penalty value, and the third weight of the position penalty value; Based on the first weight, the second weight, and the third weight, the texture penalty value, the size penalty value, and the position penalty value are weighted and summed to obtain the total penalty value; The score of the candidate patch is obtained by subtracting the total penalty value from the base score.

[0009] In some embodiments, the convergence condition includes at least one of the following: In consecutive preset rounds of iteration, the average score change of each candidate patch in the elite sample set is less than a preset score fluctuation threshold. The number of iterations for optimization reaches the preset maximum number of iterations; The candidate patch with the highest current score has an attack success rate on the verification sample that meets the preset requirements, and the action offset caused by the candidate patch with the highest current score relative to the baseline action vector is continuously greater than the attack success threshold.

[0010] In some embodiments, before inputting the multiple sets of perturbed video data into the visual language action model, the method further includes: Based on the object detection model, according to the text prompts related to the current evaluation task, the target foreground region and non-target background region of each frame of the initial video data are identified. The target foreground region is the region where the target object related to the current evaluation task is located, and the non-target background region is the region outside the target foreground region. Based on the target foreground region, an undisturbed region is determined, and the non-target background region is set as a perturbable region to generate a video frame-level binary mask.

[0011] In some embodiments, each of the perturbation patches must satisfy the following physical constraints: The perturbation patch is located entirely within the perturbable region of the video frame-level binary mask and does not cover the undisturbable region. The size of the disturbance patch is within a preset size range; The pixel values ​​of the perturbation patch are within the pixel value range of the standard image; The texture complexity of the perturbation patch does not exceed a preset texture threshold to avoid abrupt texture changes. The pixel change amplitude of the perturbation patch does not exceed a preset perturbation intensity threshold.

[0012] In some embodiments, determining an undisturbed region based on the target foreground region, setting the non-target background region as a perturbable region, and generating a video frame-level binary mask includes: Correct the positional offset of the target box between adjacent frames in the initial video data, and combine the motion trajectory of the target object in the adjacent frames to smooth the target foreground region to obtain a target foreground region with consistent time sequence. The target foreground region with consistent time sequence is set as an undisturbed region, and the non-target background region is set as a perturbable region to generate an initial video frame-level binary mask. Morphological processing is performed on the initial video frame-level binary mask to fill the holes at the edges of the undisturbed region, and connected component analysis is performed on the perturbed region to filter out noise regions with an area smaller than a preset threshold.

[0013] The present invention also provides a safety evaluation device for visual language action models, comprising: The baseline acquisition unit is used to input initial video data into the visual language action model to be evaluated, and obtain the baseline action vector output by the visual language action model. The perturbation query unit is used to input multiple sets of perturbation video data into the visual language action model to obtain multiple sets of perturbation action vectors output by the visual language action model. Each set of perturbation video data is obtained by perturbing the initial video data based on multiple corresponding perturbation patches. The boundary calibration unit is used to calculate the action offset of each group of perturbation action vectors relative to the baseline action vector, filter out the effective perturbation patch set that causes task failure from the multiple perturbation patches in combination with the task failure criteria, and determine the external observable decision boundary parameters of the visual language action model based on the distribution of each action offset corresponding to the effective perturbation patch set. The patch optimization unit is used to initialize a candidate patch set based on the effective perturbation patch set, constrained by the externally observable decision boundary parameters, and iteratively optimize the candidate patch set based on the cross-entropy algorithm and the constructed scoring function until the convergence condition is met. The unit outputs the candidate patch with the highest score as the adversarial patch and generates the security evaluation result of the visual language action model.

[0014] The present invention also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement a security evaluation method for a visual language action model as described above.

[0015] The present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements a security evaluation method for a visual language action model as described above.

[0016] The present invention also provides a computer program product, including a computer program that, when executed by a processor, implements a security evaluation method for a visual language action model as described above.

[0017] This invention provides a security evaluation method, apparatus, and electronic device for a visual language action model. It involves inputting initial video data into the visual language action model to obtain a baseline action vector; inputting multiple sets of perturbed video data into the model to obtain multiple sets of perturbed action vectors; calculating the action offset of each perturbed action vector relative to the baseline action vector; filtering out a set of effective perturbed patches that cause task failure; determining externally observable decision boundary parameters based on the distribution of action offsets corresponding to the effective perturbed patch set; achieving security evaluation of the black-box model without relying on internal model information; initializing a candidate patch set using the decision boundary parameters as constraints; iteratively optimizing the candidate patch set based on the cross-entropy algorithm and scoring function; reducing the optimization space; efficiently generating adversarial patches; and generating security evaluation results. This enhances the physical concealment and attack robustness of the adversarial patches, improves optimization efficiency, and reduces computational overhead. Attached Figure Description

[0018] To more clearly illustrate the technical solutions in this invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0019] Figure 1This is a flowchart illustrating the safety evaluation method for visual language action models provided by this invention.

[0020] Figure 2 This is a flowchart illustrating the process of determining the externally observable decision boundary parameters of a visual language action model, as provided by the present invention.

[0021] Figure 3 This is a structural schematic diagram of the safety evaluation device for the visual language action model provided by the present invention.

[0022] Figure 4 This is a schematic diagram of the structure of the electronic device provided by the present invention. Detailed Implementation

[0023] To make the objectives, technical solutions, and advantages of this invention clearer, the technical solutions of this invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this invention. All other embodiments obtained by those skilled in the art based on the embodiments of this invention without creative effort are within the scope of protection of this invention.

[0024] Figure 1 This is a flowchart illustrating the safety evaluation method for visual language action models provided by this invention, as shown below. Figure 1 As shown, the method includes the following steps: Step 110: Input the initial video data into the visual language action model to be evaluated to obtain the baseline action vector output by the visual language action model.

[0025] The initial video data refers to clean video footage, without any added interference elements, collected by the robot or embodied intelligent device through visual sensors in normal working scenarios. The visual language action model to be evaluated is an end-to-end deep learning network that integrates visual input and natural language understanding to directly output action commands. In practical applications, it is often provided to the outside world as a black-box service with an external calling interface.

[0026] Specifically, initial video data and text commands related to the current task are input into the visual language action model to be evaluated. After feature extraction and modality fusion, the visual language action model outputs a baseline motion vector representing the normal execution of the task. The baseline motion vector typically contains multi-dimensional continuous control parameters, such as the three-dimensional spatial coordinates and attitude angles of the robotic arm's end effector, and the opening and closing control values ​​of the gripper. This baseline motion vector serves as a benchmark reference for subsequent measurement of model motion deviation and safety evaluation.

[0027] Step 120: Input multiple sets of perturbation video data into the visual language action model to obtain multiple sets of perturbation action vectors output by the visual language action model. Each set of perturbation video data is obtained by perturbing the initial video data with multiple corresponding perturbation patches.

[0028] In this context, a perturbation patch refers to a manually constructed local image interference block with specific pixel values, texture, size, and positional attributes. To actively detect the model's response to interference, perturbation patches with varying parameters are superimposed onto the video frames of the initial video data, generating multiple sets of perturbation video data with parallax differences. Each set of perturbation video data is then input into the visual language action model to be evaluated for black-box inference. Due to the influence of local interference images, the visual language action model's output action commands will change to varying degrees, thus obtaining multiple sets of perturbation action vectors generated by the visual language action model in response to each set of interference inputs.

[0029] Step 130: Calculate the action offset of each group of perturbation action vectors relative to the baseline action vectors. Combine the task failure criteria to select the effective perturbation patch set that causes task failure from multiple perturbation patches. Based on the distribution of each action offset corresponding to the effective perturbation patch set, determine the external observable decision boundary parameters of the visual language action model.

[0030] Among them, the action offset is used to quantify the severity of the deviation of the model output action from the normal operating trajectory. It can be obtained by calculating the distance deviation between each perturbation action vector and the baseline action vector in the multidimensional continuous action space.

[0031] Optionally, the motion offset can be represented using Euclidean distance: ; in, Indicates the first The action offset corresponding to each perturbation patch Indicates the first The perturbation action vector corresponding to the perturbation patch is the first... Dimensional action value, The first line represents the baseline action vector. Dimensional action value, This represents the dimension of the perturbation action vector and the baseline action vector.

[0032] The task failure criteria are based on the pre-set operational failure judgment criteria for the actual robot control task, such as the robotic arm deviating from the preset distance of the target object, the navigation deviating from the predetermined route, or the gripper failing to accurately contact the target object.

[0033] Specifically, based on the feedback results of the task simulation, samples that actually caused task failure are selected from all previously generated perturbation patches and aggregated to form a valid perturbation patch set. Further statistical analysis is performed on the action offsets caused by each sample in the valid perturbation patch set to explore the distribution pattern of their deviation values. This allows for the inverse fitting of externally observable decision boundary parameters characterizing the model's transition from a normal output state to an abnormal failure state. Thus, key boundary indicators reflecting the model's weak points are obtained without relying on the model's internal structure and parameter information.

[0034] Step 140: Using the externally observable decision boundary parameters as constraints, initialize the candidate patch set based on the effective perturbation patch set. Based on the cross-entropy algorithm and the constructed scoring function, iteratively optimize the candidate patch set until the convergence condition is met. Output the candidate patch with the highest score as the adversarial patch and generate the security evaluation result of the visual language action model.

[0035] Optionally, after obtaining the externally observable decision boundary parameters, these are used as prior constraints for subsequent black-box optimization, thereby significantly reducing the search and optimization space for adversarial examples. An initial set of samples is initialized based on the effective perturbation patch set to form a candidate patch set. Then, a gradient-free global optimization algorithm suitable for black-box scenarios, namely the cross-entropy algorithm, is introduced, combined with a scoring function constructed to comprehensively evaluate the effectiveness and physical characteristics of patch attacks, to iteratively optimize the candidate patch set.

[0036] Specifically, in each iteration, a scoring function is used to evaluate and score each candidate patch, and the best patch is selected for further sampling and propagation until a preset convergence condition is met, such as the score change during the iteration process becoming stable or reaching a set upper limit for the number of iterations. After the iteration stops, the candidate patch with the highest comprehensive score is selected as the final optimized adversarial patch. At the same time, combined with the derived data such as the task failure rate and action offset value caused by the adversarial patch, a security evaluation result for the visual language action model is generated to quantify its robustness risk to visual interference in a black-box deployment environment.

[0037] In this embodiment of the invention, initial video data is input into a visual language action model to obtain a baseline action vector; multiple sets of perturbed video data are input into the visual language action model to obtain multiple sets of perturbed action vectors; the action offset of each set of perturbed action vectors relative to the baseline action vector is calculated, and a set of effective perturbed patches that cause task failure is selected. Based on the distribution of each action offset corresponding to the set of effective perturbed patches, externally observable decision boundary parameters are determined. Security evaluation of the black-box model can be achieved without relying on internal model information; a candidate patch set is initialized with the decision boundary parameters as constraints, and the candidate patch set is iteratively optimized based on the cross-entropy algorithm and scoring function, thereby reducing the optimization space, efficiently generating adversarial patches, and generating security evaluation results. This enhances the physical concealment and attack robustness of the adversarial patches, improves optimization efficiency, and reduces computational overhead.

[0038] In some embodiments, externally observable decision boundary parameters include an attack success threshold, a minimum effective perturbation strength, and a sensitive background region distribution.

[0039] The attack success threshold refers to the critical value of the action offset that characterizes the model's shift from outputting normal actions to outputting abnormal actions, leading to task failure. The minimum effective perturbation intensity refers to the minimum image pixel modification required while maintaining a certain probability of task failure. Sensitive background region distribution refers to the set of spatial locations in the initial video data that are more likely to cause significant deviations in the action decisions of the visual language action model after being disturbed.

[0040] Figure 2 This is a flowchart illustrating the process of determining the externally observable decision boundary parameters of a visual language action model, as provided by the present invention. In some embodiments, step 130 determines the externally observable decision boundary parameters of the visual language action model based on the distribution of each action offset corresponding to the effective perturbation patch set, including treating the perturbation test of the initial video data as a test round and repeating the test multiple times; in each test round, the following steps are performed: Step 131: Sort the action offsets corresponding to the effective perturbation patch set selected in this round in ascending order, and take the lower quartile value after sorting as the success threshold of this round of attack.

[0041] Specifically, in order to eliminate inaccurate threshold calibration caused by individual extreme minimum values, the action offsets caused by all effective perturbation patches obtained in this round are sorted in ascending order, and the lower quartile statistical value at the quarter position is selected as the success threshold of this round of attack. This value can both filter abnormal noise samples and accurately reflect the early critical state of perturbation samples entering the mission failure region.

[0042] Optionally, the formula for calculating the success threshold of this attack is as follows: ; in, This indicates the threshold for a successful attack in this round. This represents the lower quartile value. This represents the set of action offsets corresponding to the set of valid perturbation patches.

[0043] Step 132: Gradually increase the perturbation intensity of the test patch within the preset perturbation intensity range and at a preset step size. Calculate the proportion of test patches that cause task failure under each perturbation intensity to the total number of test patches. Determine the perturbation intensity at which the proportion first reaches the preset success rate threshold as the minimum effective perturbation intensity for this round.

[0044] The test patch is an exploratory sample specifically used to probe the boundary of the model's anti-interference strength. Starting from a low initial strength, the pixel change amplitude of the test patch is increased incrementally at a fixed preset step size, and the percentage of test patches that cause task execution failure is calculated at each strength level. When this percentage first climbs to a preset success rate threshold, such as 70%, the strength value at this time is recorded as the minimum effective perturbation strength for this round, thereby determining the lower limit of the minimum modification cost required to have an effective physical impact on the model.

[0045] Step 133: Summarize the execution results of multiple rounds of testing, determine the average of the success thresholds of each round of attack obtained from multiple rounds of testing as the success threshold of attack, and determine the average of the minimum effective perturbation strengths of each round of attack obtained from multiple rounds of testing as the minimum effective perturbation strength.

[0046] Understandably, due to the data fluctuations caused by random sampling in single-round black-box queries, by collecting various indicators obtained from multiple independent and repeated tests and using the arithmetic mean to smooth the data, the final noise-reduced and stable attack success threshold and minimum effective perturbation intensity can be obtained.

[0047] Optionally, the formula for calculating the attack success threshold is as follows: ; in, This represents the final threshold for a successful attack. Indicates the total number of rounds of testing. Indicates the first The threshold for a successful attack on a wheel.

[0048] Step 134: Statistically analyze the location distribution of each perturbation patch whose action offset is greater than or equal to the attack success threshold in multiple rounds of testing, and determine the area where each perturbation patch is concentrated as the sensitive background area distribution.

[0049] Specifically, the coordinate parameters of the perturbation patches that can produce significant motion deviation effects in each test are extracted. The frequency and spatial location of these perturbation patches in the video are superimposed and heat-statistically analyzed. The image blocks with high frequency hits and dense overlap are defined as sensitive background regions, which are used to guide the generation of targeted perturbations in vulnerable spatial regions that are more likely to cause model errors in the subsequent optimization stage.

[0050] In this embodiment of the invention, by employing multi-round queries and mean smoothing, the random noise and data fluctuations in the black-box random sampling process can be effectively eliminated. By combining the lower quartile values ​​and the incremental search strategy, the critical point of model action failure and the minimum interference cost are accurately quantified and located, avoiding the distortion of boundary parameter evaluation caused by a single extreme sample.

[0051] In some embodiments, step 140 iteratively optimizes the candidate patch set based on the cross-entropy algorithm and the constructed scoring function, including iteratively executing the following steps until the convergence condition is met: Step 141: Calculate the score of each candidate patch in the candidate patch set based on the scoring function.

[0052] The candidate patch set refers to a group of multi-dimensional interference image patches participating in the test in the current iteration round. The scoring function is a pre-defined comprehensive evaluation mathematical equation based on the motion deviation effect caused by the patch and its own physical property constraints.

[0053] Specifically, by substituting the test performance parameters of each candidate patch within the candidate patch set into the scoring function, a specific score value that can comprehensively reflect the attack capability and physical concealment of the candidate patch is calculated.

[0054] Step 142: Sort all candidate patches in the candidate patch set according to the scores, and select a preset number of top-ranked candidate patches to form an elite sample set.

[0055] Specifically, based on the aforementioned calculated scores, all candidate patches participating in this round of evaluation are sorted in descending order from highest to lowest. A preset number can be determined based on the system's actual computing resources and the overall population size. The preset number of high-scoring candidate patches at the top of the ranking are selected. Since these candidate patches demonstrate superior effectiveness in disrupting the model's normal decision-making, they are aggregated and combined to form an elite sample set representing the high-quality interference characteristics of this round.

[0056] Step 143: Fit a probability distribution model to the parameter distribution of each candidate patch in the elite sample set.

[0057] Specifically, we extract detailed parameter information such as pixel color values, spatial coordinates, and perturbation intensity from each candidate patch within the elite sample set. We then use statistical methods to calculate characteristic indicators such as the mean and variance of each parameter. Based on these statistical characteristics, we establish a probability distribution model that describes the patterns of high-quality patch parameter presentation. For example, we construct a Gaussian probability model for pixel value distribution, thereby transforming the scattered experience of high-quality patch samples into a continuous mathematical probability distribution representation.

[0058] Step 144: Sample from the probability distribution model to generate new candidate patches and update the candidate patch set.

[0059] Optionally, numerical sampling can be performed based on a probability distribution model. Since the probability distribution model is already biased towards high-scoring feature regions, the new generation of candidate patches generated by sampling naturally carries the high-quality feature parameters of the previous round of elite samples. These newly generated candidate patches are used to replace the data of the original batch to complete the overall update of the candidate patch set, and the updated candidate patch set is then put back into the next round of validation for continuous optimization.

[0060] In this embodiment of the invention, by scoring and ranking candidate patches based on a scoring function and extracting an elite sample set, and then fitting a probability distribution model to the parameter distribution of the elite samples to guide the sampling and updating of the next round of candidate patches, the subsequently generated candidate patches can continuously converge in the search space toward high-quality solution regions that are more likely to cause model misjudgment. This avoids blind and inefficient exhaustive search in a large and high-dimensional continuous pixel space, and improves the efficiency of adversarial patch generation and parameter convergence speed for black-box visual language action models.

[0061] In some embodiments, step 141 calculates a score for each candidate patch in the candidate patch set based on a scoring function, including: Step 1411: Input the candidate perturbation video data into the visual language action model to obtain the candidate perturbation action vector output by the visual language action model. The candidate perturbation video data is obtained by perturbing the initial video data based on the candidate patch.

[0062] Candidate perturbation video data refers to video footage with specific test interference formed by overlaying and fusing candidate patches from the current iteration round onto the initial video data. This candidate perturbation video data is input into the visual language action model via an interface call. After internal calculation, the model outputs candidate perturbation action vectors for the current test interference footage, which serve as the original analytical basis for subsequent evaluation of the attack effectiveness of the candidate patch.

[0063] Step 1412: Calculate the candidate action offset of the candidate perturbation action vector relative to the baseline action vector, calculate the difference between the candidate action offset and the attack success threshold, and obtain the basic score.

[0064] The candidate action offset characterizes the degree of action deviation caused by the current candidate patch. This offset is obtained by calculating the distance difference between the candidate perturbation action vector and the baseline action vector across each parameter dimension. Further, the obtained candidate action offset is subtracted from the attack success threshold defined in the previous stage, and the calculated difference is set as the base score. A base score greater than zero indicates that the candidate patch possesses the basic attack capability to break through the model's decision boundary; a larger base score reflects stronger basic destructive effectiveness.

[0065] Optionally, the formula for calculating the candidate action offset is as follows: ; in, Indicates the first One candidate patch The corresponding candidate action offset, Indicates the first One candidate patch The corresponding candidate perturbation action vector, This represents the baseline action vector.

[0066] Step 1413: Obtain the texture complexity, size parameter, and position parameter of the candidate patch. Calculate the texture penalty value based on the texture complexity, the size penalty value based on the size parameter, and the position penalty value based on the position parameter.

[0067] Optionally, texture complexity can be calculated by methods such as calculating the variance of pixels within a candidate patch. Texture complexity is used to quantify the smoothness or abruptness of the image texture.

[0068] Among them, the size parameter represents the area occupied by the candidate patch in the video frame, and the position parameter reflects the spatial coordinate region where the candidate patch is located.

[0069] Specifically, by constructing corresponding regularization equations, texture penalty values, size penalty values, and position penalty values ​​are calculated according to conditions such as excessively high texture complexity, excessively large size area, and non-compliant position, which are used to suppress abnormal patch samples with overly conspicuous physical features during the optimization process.

[0070] Step 1414: Determine the first weight of the texture penalty value, the second weight of the size penalty value, and the third weight of the position penalty value.

[0071] It should be noted that, since different physical constraints vary in importance in real-world application scenarios, corresponding proportional coefficients need to be assigned to the aforementioned penalty values. The first, second, and third weights can be customized or dynamically adjusted based on the emphasis placed on concealment, patch size, and spatial compliance in the current model security evaluation task, thereby achieving flexible control over the influence of various penalty items.

[0072] Step 1415: Based on the first weight, the second weight, and the third weight, the texture penalty value, the size penalty value, and the position penalty value are weighted and summed to obtain the total penalty value.

[0073] The total penalty value represents the degree of deficiency of the candidate patch in terms of physical concealment and implementation feasibility. The total penalty value is obtained by multiplying each penalty value by its corresponding weight coefficient and summing them. The higher the value, the easier it is for the candidate patch to be detected by the human eye in terms of visual features or the less it meets the set physical constraints.

[0074] Step 1416: Subtract the total penalty value from the base score to obtain the score of the candidate patch.

[0075] Specifically, the final score of a candidate patch is obtained by subtracting the total penalty value representing physical implementation flaws from the base score characterizing the attack's destructive capability. This score comprehensively weighs the patch's destructive effect and physical concealment characteristics, serving as the core quantitative criterion for subsequently selecting high-quality elite samples.

[0076] Optionally, the calculation formula for the scoring function is as follows: ; in, Indicates the first One candidate patch The rating, Indicates the first One candidate patch The offset of the candidate action. Indicates the threshold for a successful attack; Indicates the first One candidate patch Texture penalty value, Indicates the first One candidate patch Size penalty value, Indicates the first One candidate patch Position penalty value; As the first weight, As the second weight, It is the third weight.

[0077] In this embodiment of the invention, a basic score is determined by calculating the difference between the candidate action offset and the attack success threshold. The total penalty value is obtained by weighted summation of the texture complexity, size parameters, and position parameters of the candidate patch. Finally, the score of the candidate patch is obtained by subtracting the total penalty value from the basic score. This can guide the optimization algorithm to take into account the concealment and physical compliance of the interference image while pursuing the output of the destructive model action. It effectively eliminates unrealistic samples with good attack effects but abrupt textures or excessively large areas, so that the final generated adversarial patch has stronger concealment and physical implementation value in actual operation scenarios.

[0078] In some embodiments, the convergence criteria include at least one of the following: In consecutive preset rounds of iteration, the average score change of each candidate patch in the elite sample set is less than the preset score fluctuation threshold. The number of iterations for optimization has reached the preset maximum number of iterations; The candidate patch with the highest score currently achieves a success rate of attack on the verification sample that meets the preset requirements, and the action offset caused by the candidate patch with the highest score relative to the baseline action vector is consistently greater than the attack success threshold.

[0079] The convergence condition is the basis for determining whether the iterative optimization process of the cross-entropy algorithm has ended.

[0080] Optionally, the average score change of each candidate patch in the elite sample set can be monitored in consecutive preset rounds of iteration. When the change is less than the preset score fluctuation threshold, it indicates that the algorithm has searched to the high-score flat region of the solution space, and the quality of the candidate patch tends to stabilize and is difficult to continue to improve significantly, thus triggering early termination.

[0081] Alternatively, the number of iterations can be compared with a preset maximum number of iterations. When the preset maximum number of iterations is reached, the loop is forcibly terminated, thus preventing the algorithm from getting stuck in infinite running.

[0082] Optionally, the candidate patch with the highest current score can be extracted and applied to additional verification samples for attack testing. When it is found that its attack success rate reaches the preset requirement and the action offset caused by its relative to the baseline action vector is consistently and stably greater than the attack success threshold, it indicates that the optimal patch has mature and generalized black-box attack destruction capabilities, thus meeting the output criteria.

[0083] In some embodiments, before inputting multiple sets of perturbed video data into the visual language action model, the method further includes: Based on the object detection model, according to the text prompts related to the current evaluation task, the target foreground region and non-target background region of each frame of the initial video data are identified. The target foreground region is the region where the target object related to the current evaluation task is located, and the non-target background region is the region outside the target foreground region. Based on the target foreground region, an undisturbed region is determined, and the non-target background region is set as a perturbable region to generate a video frame-level binary mask.

[0084] Among them, the object detection model refers to a deep learning network with cross-modal recognition and localization capabilities from text to images. Text prompts are instruction phrases that describe the name or attribute characteristics of the object being operated on, such as the word "cup" in a task to grab a cup.

[0085] Specifically, the initial video data is input into the object detection model and processed in conjunction with text prompts to extract the pixel blocks in each frame containing the target object that directly interacts with the current evaluation task, i.e., the target foreground region. Simultaneously, other environmental pixel blocks within the image, excluding the target foreground region, are classified as non-target background regions.

[0086] Specifically, to protect the key visual features of the target from damage, the identified target foreground region is strictly designated as an undisturbed area, prohibiting the overlay of any anti-interference elements within this region. Correspondingly, the non-target background region is designated as a perturbable area, allowing the addition of interfering image blocks. Based on the aforementioned region division, a video frame-level binary mask of the same size is constructed for each frame image. The video frame-level binary mask is a matrix array composed of identifier elements such as 0s and 1s.

[0087] In some embodiments, each perturbation patch must satisfy the following physical constraints: The perturbation patch is located within the perturbable region of the corresponding frame-level binary mask in the video, and does not cover the undisturbable region. The size of the disturbance patch is within a preset size range; The pixel values ​​of the perturbation patch are within the range of pixel values ​​of the standard image; The texture complexity of the perturbation patch does not exceed a preset texture threshold to avoid abrupt texture changes. The pixel change of the perturbation patch does not exceed the preset perturbation intensity threshold.

[0088] Among them, the physical constraints are multi-dimensional restriction rules set to ensure that the generated adversarial examples have high concealment and feasibility in the real physical world.

[0089] Specifically, the spatial coordinates of the perturbation patch are verified using the previously generated video frame-level binary mask to ensure that all pixels of the perturbation patch fall within the perturbable area representing the non-target background region, without overriding the undisturbed area representing the core task object. The size of the perturbation patch is limited to a preset range, for example, its length and width are set to a threshold not exceeding the total aspect ratio of the video frame, preventing visual perception due to excessive interference area. The pixel values ​​of the perturbation patch are verified to be within the pixel value range of a standard image, i.e., the red, green, and blue channel values ​​are all within the legal color space of 0 to 255, ensuring the output is a normal, displayable digital image. The texture complexity of the perturbation patch is evaluated by calculating internal pixel variance and other methods, ensuring it does not exceed a preset texture threshold. This suppresses high-frequency noise with drastic color jumps, preventing the image from presenting a jarring and abrupt visual experience. The pixel variation amplitude of the perturbation patch is limited to a preset perturbation intensity threshold, ensuring that the difference in original background pixels before and after the perturbation is strictly controlled within a minimal range.

[0090] In some embodiments, an undisturbed region is determined based on the target foreground region, and a non-target background region is set as a perturbable region to generate a video frame-level binary mask, including: Correct the positional offset of the target box between adjacent frames in the initial video data, and combine the motion trajectory of the target object in adjacent frames to smooth the target foreground region to obtain a target foreground region with consistent time sequence. The target foreground region with consistent temporal sequence is set as an undisturbed region, and the non-target background region is set as a perturbable region to generate an initial video frame-level binary mask. Morphological processing is performed on the initial video frame-level binary mask to fill the holes at the edges of undisturbed regions, and connected component analysis is performed on the perturbable regions to filter out noise regions with an area smaller than a preset threshold.

[0091] Specifically, because the initial video data contains a dynamic and continuous sequence of image frames, the target detection model is prone to slight positional shifts, such as flickering edges of the detection box or coordinate jumps, when performing independent detection on a single frame. By employing techniques such as inter-frame interpolation to correct the coordinate deviation of the target box between adjacent frames, and combining this with the actual motion trajectory of the target object moving in the frame within adjacent frames, the position and boundary size of the target foreground region can be smoothed and filtered. This eliminates visual abrupt changes between consecutive frames that do not conform to the laws of physical motion, thereby obtaining a temporally consistent target foreground region that transitions naturally and is stable and continuous in the time series.

[0092] Specifically, due to variations in image lighting or detection accuracy errors, the edges of the initially generated mask are often not smooth enough, and there may even be internal errors. Morphological processing methods, such as image morphological closing operations, are used to perform secondary optimization on the initial video frame-level binary mask. A convolutional kernel of a specific size is used to fill the tiny holes caused by pixel misjudgments at the edges of undisturbed regions, preventing subsequent perturbation patch pixels from penetrating into the target object. Simultaneously, connected component analysis is performed on the pixel patches marked as perturbable regions, calculating the specific pixel area of ​​each interconnected background block. Small background fragments or isolated noise points with areas too small to accommodate a single complete perturbation patch are identified as noise regions and forcibly filtered out. This ensures that the final retained perturbable regions have sufficient and continuous physical space for arrangement.

[0093] The following describes the safety evaluation device for the visual language action model provided by the present invention. The safety evaluation device for the visual language action model described below and the safety evaluation method for the visual language action model described above can be referred to in correspondence.

[0094] Figure 3 This is a structural schematic diagram of the safety evaluation device for the visual language action model provided by the present invention. Figure 3 As shown, the safety evaluation device 300 for the visual language action model includes: The baseline acquisition unit 310 is used to input the initial video data into the visual language action model to be evaluated, and obtain the baseline action vector output by the visual language action model. The perturbation query unit 320 is used to input multiple sets of perturbation video data into the visual language action model to obtain multiple sets of perturbation action vectors output by the visual language action model. Each set of perturbation video data is obtained by perturbing the initial video data based on multiple corresponding perturbation patches. Boundary calibration unit 330 is used to calculate the action offset of each group of perturbation action vectors relative to the baseline action vector, and to filter out the effective perturbation patch set that causes task failure from multiple perturbation patches in combination with the task failure criteria. Based on the distribution of each action offset corresponding to the effective perturbation patch set, the external observable decision boundary parameters of the visual language action model are determined. The patch optimization unit 340 is used to initialize the candidate patch set based on the effective perturbation patch set with externally observable decision boundary parameters as constraints, and to iteratively optimize the candidate patch set based on the cross-entropy algorithm and the constructed scoring function until the convergence condition is met. The candidate patch with the highest score is output as the adversarial patch, and the security evaluation result of the visual language action model is generated.

[0095] Optionally, externally observable decision boundary parameters include the attack success threshold, the minimum effective perturbation strength, and the distribution of sensitive background regions.

[0096] Optionally, based on the distribution of each action offset corresponding to the effective perturbation patch set, the external observable decision boundary parameters of the visual language action model are determined, including treating the perturbation test of the initial video data as a test round and repeating the test multiple times; in each test round, the following steps are performed: Sort the action offsets corresponding to the set of valid perturbation patches selected in this round in ascending order, and take the lower quartile value after sorting as the success threshold of this round of attack. Within the preset disturbance intensity range, the disturbance intensity of the test patch is gradually increased by a preset step size. The proportion of the number of test patches that cause task failure under each disturbance intensity to the total number of all test patches is counted. The disturbance intensity that first makes the proportion reach the preset success rate threshold is determined as the minimum effective disturbance intensity of this round. The execution results of multiple rounds of testing are summarized, and the average value of the attack success thresholds obtained from multiple rounds of testing is determined as the attack success threshold. The average value of the minimum effective perturbation strength obtained from multiple rounds of testing is determined as the minimum effective perturbation strength. The location distribution of each perturbation patch with an action offset greater than or equal to the attack success threshold was statistically analyzed in multiple rounds of testing, and the areas where each perturbation patch was concentrated were identified as sensitive background areas.

[0097] Optionally, based on the cross-entropy algorithm and the constructed scoring function, the candidate patch set is iteratively optimized, including iteratively executing the following steps until the convergence condition is met: The score for each candidate patch in the candidate patch set is calculated based on the scoring function; All candidate patches in the candidate patch set are sorted according to the scores, and a predetermined number of the top-ranked candidate patches are selected to form an elite sample set. For the parameter distribution of each candidate patch in the elite sample set, fit a probability distribution model; New candidate patches are generated by sampling from the probability distribution model, and the candidate patch set is updated.

[0098] Optionally, a score is calculated for each candidate patch in the candidate patch set based on a scoring function, including: The candidate perturbation video data is input into the visual language action model to obtain the candidate perturbation action vector output by the visual language action model. The candidate perturbation video data is obtained by perturbing the initial video data based on the candidate patch. Calculate the candidate action offset of the candidate perturbation action vector relative to the baseline action vector, calculate the difference between the candidate action offset and the attack success threshold, and obtain the basic score; Obtain the texture complexity, size parameter, and position parameter of the candidate patch; calculate the texture penalty value based on the texture complexity; calculate the size penalty value based on the size parameter; and calculate the position penalty value based on the position parameter. Determine the first weight for the texture penalty value, the second weight for the size penalty value, and the third weight for the position penalty value; Based on the first weight, the second weight, and the third weight, the texture penalty value, the size penalty value, and the position penalty value are weighted and summed to obtain the total penalty value; Subtract the total penalty value from the base score to obtain the score of the candidate patch.

[0099] Optionally, the convergence criteria include at least one of the following: In consecutive preset rounds of iteration, the average score change of each candidate patch in the elite sample set is less than the preset score fluctuation threshold. The number of iterations for optimization has reached the preset maximum number of iterations; The candidate patch with the highest score currently achieves a success rate of attack on the verification sample that meets the preset requirements, and the action offset caused by the candidate patch with the highest score relative to the baseline action vector is consistently greater than the attack success threshold.

[0100] Optionally, the above-mentioned device further includes a recognition unit for performing the following steps before inputting multiple sets of disturbed video data into the visual language action model: Based on the object detection model, according to the text prompts related to the current evaluation task, the target foreground region and non-target background region of each frame of the initial video data are identified. The target foreground region is the region where the target object related to the current evaluation task is located, and the non-target background region is the region outside the target foreground region. Based on the target foreground region, an undisturbed region is determined, and the non-target background region is set as a perturbable region to generate a video frame-level binary mask.

[0101] Optionally, each perturbation patch must satisfy the following physical constraints: The perturbation patch is located within the perturbable region of the corresponding frame-level binary mask in the video, and does not cover the undisturbable region. The size of the disturbance patch is within a preset size range; The pixel values ​​of the perturbation patch are within the range of pixel values ​​of the standard image; The texture complexity of the perturbation patch does not exceed a preset texture threshold to avoid abrupt texture changes. The pixel change of the perturbation patch does not exceed the preset perturbation intensity threshold.

[0102] Optionally, based on the target foreground region, an undisturbed region is determined, and non-target background regions are set as perturbable regions to generate a video frame-level binary mask, including: Correct the positional offset of the target box between adjacent frames in the initial video data, and combine the motion trajectory of the target object in adjacent frames to smooth the target foreground region to obtain a target foreground region with consistent time sequence. The target foreground region with consistent temporal sequence is set as an undisturbed region, and the non-target background region is set as a perturbable region to generate an initial video frame-level binary mask. Morphological processing is performed on the initial video frame-level binary mask to fill the holes at the edges of undisturbed regions, and connected component analysis is performed on the perturbable regions to filter out noise regions with an area smaller than a preset threshold.

[0103] Figure 4 This is a schematic diagram of the structure of the electronic device provided by the present invention, such as... Figure 4 As shown, the electronic device may include: a processor 410, a communications interface 420, a memory 430, and a communications bus 440, wherein the processor 410, the communications interface 420, and the memory 430 communicate with each other through the communications bus 440. The processor 410 can call logical instructions in the memory 430 to execute a security evaluation method for a visual language action model. This method includes: inputting initial video data into the visual language action model to be evaluated to obtain a baseline action vector output by the visual language action model; inputting multiple sets of perturbed video data into the visual language action model to obtain multiple sets of perturbed action vectors output by the visual language action model, where each set of perturbed video data is obtained by perturbing the initial video data with corresponding multiple perturbed patches; calculating the action offset of each set of perturbed action vectors relative to the baseline action vector, selecting a set of effective perturbed patches that cause task failure from multiple perturbed patches based on a task failure criterion, determining the externally observable decision boundary parameters of the visual language action model based on the distribution of action offsets corresponding to the effective perturbed patch set; initializing a candidate patch set based on the effective perturbed patch set using the externally observable decision boundary parameters as constraints, iteratively optimizing the candidate patch set based on the cross-entropy algorithm and the constructed scoring function until convergence conditions are met, outputting the candidate patch with the highest score as an adversarial patch, and generating a security evaluation result for the visual language action model.

[0104] Furthermore, the logical instructions in the aforementioned memory 430 can be implemented as software functional units and, when sold or used as independent products, can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, essentially, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0105] On the other hand, the present invention also provides a computer program product, which includes a computer program that can be stored on a non-transitory computer-readable storage medium. When the computer program is executed by a processor, the computer can execute the security evaluation method for the visual language action model provided by the above methods. The method includes: inputting initial video data into the visual language action model to be evaluated to obtain a baseline action vector output by the visual language action model; inputting multiple sets of perturbed video data into the visual language action model to obtain multiple sets of perturbed action vectors output by the visual language action model, wherein each set of perturbed video data is based on multiple corresponding perturbed patches applied to the initial video data. The perturbation is performed to obtain the action offset of each group of perturbation action vectors relative to the baseline action vector. Combined with the task failure criteria, the effective perturbation patch set that leads to task failure is selected from multiple perturbation patches. Based on the distribution of each action offset corresponding to the effective perturbation patch set, the external observable decision boundary parameters of the visual language action model are determined. With the external observable decision boundary parameters as constraints, the candidate patch set is initialized based on the effective perturbation patch set. Based on the cross-entropy algorithm and the constructed scoring function, the candidate patch set is iteratively optimized until the convergence condition is met. The candidate patch with the highest score is output as the adversarial patch, and the security evaluation result of the visual language action model is generated.

[0106] In another aspect, the present invention also provides a non-transitory computer-readable storage medium storing a computer program thereon, which, when executed by a processor, implements a security evaluation method for the visual language action model provided by the methods described above. This method includes: inputting initial video data into the visual language action model to be evaluated to obtain a baseline action vector output by the visual language action model; inputting multiple sets of perturbed video data into the visual language action model to obtain multiple sets of perturbed action vectors output by the visual language action model, wherein each set of perturbed video data is obtained by perturbing the initial video data based on multiple corresponding perturbed patches; calculating the perturbed action vectors of each set of perturbed video data. The action offset of the action vector relative to the baseline action vector is used to select a set of effective perturbation patches from multiple perturbation patches, based on the task failure criteria. The external observable decision boundary parameters of the visual language action model are determined according to the distribution of action offsets corresponding to the effective perturbation patch set. With the external observable decision boundary parameters as constraints, a candidate patch set is initialized based on the effective perturbation patch set. The candidate patch set is iteratively optimized based on the cross-entropy algorithm and the constructed scoring function until the convergence condition is met. The candidate patch with the highest score is output as the adversarial patch, and the security evaluation result of the visual language action model is generated.

[0107] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without any creative effort.

[0108] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus necessary general-purpose hardware platforms, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solutions, in essence or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., including several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods of various embodiments or some parts of embodiments.

[0109] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A safety evaluation method for a visual language action model, characterized in that, include: The initial video data is input into the visual language action model to be evaluated to obtain the baseline action vector output by the visual language action model. Multiple sets of perturbation video data are input into the visual language action model to obtain multiple sets of perturbation action vectors output by the visual language action model. Each set of perturbation video data is obtained by perturbing the initial video data with multiple corresponding perturbation patches. Calculate the action offset of each group of perturbation action vectors relative to the baseline action vector, and filter out the effective perturbation patch set that causes task failure from the multiple perturbation patches in combination with the task failure criteria. Based on the distribution of each action offset corresponding to the effective perturbation patch set, determine the external observable decision boundary parameters of the visual language action model. Using the externally observable decision boundary parameters as constraints, a candidate patch set is initialized based on the effective perturbation patch set. The candidate patch set is iteratively optimized based on the cross-entropy algorithm and the constructed scoring function until the convergence condition is met. The candidate patch with the highest score is output as the adversarial patch, and the security evaluation result of the visual language action model is generated.

2. The safety evaluation method for visual language action models according to claim 1, characterized in that, The externally observable decision boundary parameters include the attack success threshold, the minimum effective perturbation intensity, and the distribution of sensitive background regions.

3. The safety evaluation method for visual language action models according to claim 2, characterized in that, The step of determining the external observable decision boundary parameters of the visual language action model based on the distribution of each action offset corresponding to the effective perturbation patch set includes treating the perturbation test of the initial video data as a test round and repeating the test multiple times; in each test round, the following steps are performed: The action offsets corresponding to the effective perturbation patch set selected in this round are sorted from smallest to largest, and the lower quartile value after sorting is taken as the success threshold of this round of attack. Within a preset disturbance intensity range, the disturbance intensity of the test patch is gradually increased by a preset step size. The proportion of test patches that cause task failure under each disturbance intensity to the total number of test patches is counted. The disturbance intensity that first makes the proportion reach a preset success rate threshold is determined as the minimum effective disturbance intensity of this round. The execution results of multiple rounds of testing are summarized, and the average value of the success threshold of each round of attack obtained from multiple rounds of testing is determined as the success threshold of attack, and the average value of the minimum effective perturbation strength of each round of attack obtained from multiple rounds of testing is determined as the minimum effective perturbation strength. The location distribution of each perturbation patch with an action offset greater than or equal to the attack success threshold is statistically analyzed in multiple rounds of testing, and the area where each perturbation patch is concentrated is determined as the sensitive background area distribution.

4. The safety evaluation method for visual language action models according to claim 2, characterized in that, The cross-entropy algorithm and the constructed scoring function are used to iteratively optimize the candidate patch set, including iteratively executing the following steps until the convergence condition is met: The score for each candidate patch in the candidate patch set is calculated based on the scoring function. Based on the scores, all candidate patches in the candidate patch set are sorted, and a predetermined number of top-ranked candidate patches are selected to form an elite sample set. For the parameter distribution of each candidate patch in the elite sample set, fit a probability distribution model; New candidate patches are generated by sampling from the probability distribution model, and the candidate patch set is updated.

5. The safety evaluation method for visual language action models according to claim 4, characterized in that, The step of calculating the score for each candidate patch in the candidate patch set based on the scoring function includes: The candidate perturbation video data is input into the visual language action model to obtain the candidate perturbation action vector output by the visual language action model. The candidate perturbation video data is obtained by perturbing the initial video data based on the candidate patch. Calculate the candidate action offset of the candidate perturbation action vector relative to the baseline action vector, calculate the difference between the candidate action offset and the attack success threshold, and obtain the basic score; Obtain the texture complexity, size parameter, and position parameter of the candidate patch; calculate the texture penalty value based on the texture complexity; calculate the size penalty value based on the size parameter; and calculate the position penalty value based on the position parameter. Determine the first weight of the texture penalty value, the second weight of the size penalty value, and the third weight of the position penalty value; Based on the first weight, the second weight, and the third weight, the texture penalty value, the size penalty value, and the position penalty value are weighted and summed to obtain the total penalty value; The score of the candidate patch is obtained by subtracting the total penalty value from the base score.

6. The safety evaluation method for visual language action models according to claim 4, characterized in that, The convergence condition includes at least one of the following: In consecutive preset rounds of iteration, the average score change of each candidate patch in the elite sample set is less than a preset score fluctuation threshold. The number of iterations for optimization reaches the preset maximum number of iterations; The candidate patch with the highest current score has an attack success rate on the verification sample that meets the preset requirements, and the action offset caused by the candidate patch with the highest current score relative to the baseline action vector is continuously greater than the attack success threshold.

7. The safety evaluation method for visual language action models according to claim 1, characterized in that, Before inputting multiple sets of perturbed video data into the visual language action model, the method further includes: Based on the object detection model, according to the text prompts related to the current evaluation task, the target foreground region and non-target background region of each frame of the initial video data are identified. The target foreground region is the region where the target object related to the current evaluation task is located, and the non-target background region is the region outside the target foreground region. Based on the target foreground region, an undisturbed region is determined, and the non-target background region is set as a perturbable region to generate a video frame-level binary mask.

8. The safety evaluation method for visual language action models according to claim 7, characterized in that, Each of the aforementioned perturbation patches must satisfy the following physical constraints: The perturbation patch is located entirely within the perturbable region of the video frame-level binary mask and does not cover the undisturbable region. The size of the disturbance patch is within a preset size range; The pixel values ​​of the perturbation patch are within the pixel value range of the standard image; The texture complexity of the perturbation patch does not exceed a preset texture threshold to avoid abrupt texture changes. The pixel change amplitude of the perturbation patch does not exceed a preset perturbation intensity threshold.

9. The safety evaluation method for visual language action models according to claim 7, characterized in that, The step of determining an undisturbed region based on the target foreground region, setting the non-target background region as a perturbable region, and generating a video frame-level binary mask includes: Correct the positional offset of the target box between adjacent frames in the initial video data, and combine the motion trajectory of the target object in the adjacent frames to smooth the target foreground region to obtain a target foreground region with consistent time sequence. The target foreground region with consistent time sequence is set as an undisturbed region, and the non-target background region is set as a perturbable region to generate an initial video frame-level binary mask. Morphological processing is performed on the initial video frame-level binary mask to fill the holes at the edges of the undisturbed region, and connected component analysis is performed on the perturbed region to filter out noise regions with an area smaller than a preset threshold.

10. A safety evaluation device for a visual language action model, characterized in that, include: The baseline acquisition unit is used to input initial video data into the visual language action model to be evaluated, and obtain the baseline action vector output by the visual language action model. The perturbation query unit is used to input multiple sets of perturbation video data into the visual language action model to obtain multiple sets of perturbation action vectors output by the visual language action model. Each set of perturbation video data is obtained by perturbing the initial video data based on multiple corresponding perturbation patches. The boundary calibration unit is used to calculate the action offset of each group of perturbation action vectors relative to the baseline action vector, filter out the effective perturbation patch set that causes task failure from the multiple perturbation patches in combination with the task failure criteria, and determine the external observable decision boundary parameters of the visual language action model based on the distribution of each action offset corresponding to the effective perturbation patch set. The patch optimization unit is used to initialize a candidate patch set based on the effective perturbation patch set, constrained by the externally observable decision boundary parameters, and iteratively optimize the candidate patch set based on the cross-entropy algorithm and the constructed scoring function until the convergence condition is met. The unit outputs the candidate patch with the highest score as the adversarial patch and generates the security evaluation result of the visual language action model.

11. An electronic device comprising a memory, a processor, and a computer program stored in the memory and running on the processor, characterized in that, When the processor executes the computer program, it implements the security evaluation method for the visual language action model as described in any one of claims 1 to 9.

12. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the security evaluation method for the visual language action model as described in any one of claims 1 to 9.

13. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by the processor, it implements the security evaluation method for the visual language action model as described in any one of claims 1 to 9.