A gradient-induced multi-modal GUI agent image layer white-box adversarial attack method

Abstract

The present application relates to the technical field of artificial intelligence security and multi-modal learning, and particularly relates to a multi-modal GUI Agent image layer white-box attack method based on gradient induction. The method first presets double attack targets, and constructs a joint optimization objective function fusing text semantic bias loss and spatial coordinate offset loss; under the constraint of a preset perturbation amplitude, the gradient of the joint objective function is back-propagated from the model output layer to the input original GUI image, and the optimized adversarial perturbation is obtained by successive updating; the optimized adversarial perturbation is superimposed on the original GUI image to generate an adversarial sample; and the adversarial sample is used for security evaluation of the GUI Agent. The present application provides an extreme stress test means for security evaluation of multi-modal GUI Agent, can effectively find and quantify the essential vulnerability of the visual language model in the perception layer, and provides an important reference for subsequent defense strategy research and model security improvement.

Description

Technical Field

[0001] This invention relates to the fields of artificial intelligence security and multimodal learning technology, and in particular to a gradient-induced multimodal GUI agent image layer white-box adversarial attack method. Background Technology

[0002] With the rapid development of large-scale language models and visual language models, graphical user interface (GUI) agents based on multimodal understanding have become a research hotspot and application direction in the field of artificial intelligence. These agents use visual language models as their core decision engine, encoding and understanding visual information such as screenshots, and automatically generating interface operation text and click coordinates based on user commands. They can be widely applied in scenarios such as intelligent interaction, automated testing, and accessibility assistance.

[0003] The current mainstream implementation process of GUI Agents is as follows: features are extracted from GUI interface images through a visual encoder, and then the visual features are integrated into the language model embedding space through feature projection. Finally, a large language model decoder completes semantic understanding and spatial localization, outputting operation descriptions and interaction coordinates. Such systems highly rely on the visual perception, text generation, and visual localization capabilities of the visual language model, and their decision stability directly determines the system's reliability and security.

[0004] In the field of adversarial machine learning, researchers have extensively explored adversarial attack methods targeting deep neural networks, including mature methods such as FGSM, PGD, and C&W attacks. These methods can mislead the model into outputting incorrect results by adding small perturbations to the input data. However, existing adversarial attack research is mostly focused on traditional computer vision tasks such as image classification and object detection, while research on targeted attacks against multimodal GUI agents is still in its early stages.

[0005] Existing GUI security technologies primarily focus on the code level, defending against traditional network attacks by detecting DOM structure, blocking malicious script injection, and validating user input. This type of protection can only prevent code-level intrusions and cannot address subtle perturbations at the image pixel level, resulting in significant security blind spots.

[0006] In summary, the existing technology has the following drawbacks:

[0007] 1. There is a lack of systematic security assessment methods for the visual perception layer of multimodal GUI agents, and conventional robustness tests cannot reveal the vulnerability of the model to adversarial perturbations; 2. The code-level security detection mechanism cannot identify pixel-level tamper-proof attacks, which exposes the GUI Agent to bypass security risks; 3. There is insufficient research on adversarial attacks against the visual localization capabilities of visual language models, making it difficult to simultaneously manipulate the semantic output and coordinate output of the model; 4. Highly concealed pixel-level perturbation attacks are difficult to detect by existing systems and can easily lead to security issues such as financial fraud, privacy leaks, and malicious operations. Summary of the Invention

[0008] The present invention aims to at least partially solve one of the technical problems in the related art.

[0009] Therefore, the first objective of this invention is to propose a gradient-induced multimodal GUI agent image layer white-box adversarial attack method, comprising the following steps: S1 is a pre-defined dual attack target that includes malicious operation semantics and target interaction coordinates; S2, construct a joint optimization objective function that integrates text semantic bias loss and spatial coordinate offset loss, and impose synchronous constraints on the model's semantic output and localization output; S3, under the preset perturbation amplitude constraint, adopts a gradient-based iterative optimization algorithm to backpropagate the gradient of the joint objective function from the model output layer to the input original GUI image, and successively updates to obtain the optimized adversarial perturbation; S4. The optimized adversarial perturbation is superimposed on the original GUI image to generate adversarial examples that can be manipulated in a targeted manner to control the decision-making behavior and click position of the GUI Agent; S5 uses adversarial examples to assess the security of the GUI Agent.

[0010] In one embodiment of the present invention, S3 further includes: S31, initialize the adversarial interference value, and calculate the gradient of the joint optimization objective function with respect to the original GUI image using the input image pixel tensor as the independent variable; S32, based on the projection gradient descent algorithm, iteratively updates the perturbation along the opposite direction of the gradient, and projects the updated perturbation into the L_∞ norm sphere; S33. Repeat steps S31-S32 until the joint optimization objective function converges to a stable value or the number of iterations reaches a preset value, thus obtaining the optimized adversarial perturbation.

[0011] In one embodiment of the present invention, the calculation formula for iteratively updating the perturbation along the opposite direction of the gradient in step S32 is as follows: δ (t+ ¹ ) = δ (t) α sign ( L_total / X); Where α is a fixed step size parameter, δ (t) For the adversarial perturbation at time t, sign ( ) is a symbolic function.

[0012] In one embodiment of the present invention, the dual attack targets in step S1 include: a malicious operation text sequence to be induced and malicious click pixel coordinates to be targeted.

[0013] In one embodiment of the present invention, the joint optimization objective function in step S2 is: L_total = λ 1 ·L_text(f(X+δ), T_adv) + λ 2 ·L_coord(g(X+δ), C_target); Where X is the pixel tensor of the original GUI screenshot, and δ is the adversarial perturbation to be optimized. f(·) This represents the text output function of the model. g(·) The coordinate output function of the model. L_text For text cross-entropy loss, L_coord This represents the mean square error loss in coordinate prediction. l 1 and l 2 is the weight hyperparameter for balancing the two optimization objectives.

[0014] In one embodiment of the present invention, the algorithm for iteratively updating the perturbation in step S32 further includes the momentum iterative fast gradient sign method, the C&W attack method, and the AutoAttack method.

[0015] In one embodiment of the present invention, step S3 further includes: Extract the intermediate layer visual activation features of the target GUI component, add feature matching loss to the optimization target, and hijack the visual anchor points of the GUI Agent.

[0016] In one embodiment of the present invention, the method for hijacking the GUI Agent visual anchor is as follows: A31, extract the feature activation patterns of the target component in the intermediate layer of the visual encoder; A32 adds a feature matching loss term to the optimization objective, enabling adversarial examples to generate feature responses similar to the target component at specified locations; A33 incorporates feature matching loss into the optimization objective, hijacks the visual anchor points of the GUI Agent, and guides the perturbation distribution in a targeted manner.

[0017] To achieve the above objectives, a second aspect of the present invention proposes a gradient-induced multimodal GUIAgent image layer white-box adversarial attack system, comprising: The attack target construction module is used to pre-set dual attack targets that include malicious operation semantics and target interaction coordinates; A joint optimization objective function is used to construct a joint optimization objective function that integrates text semantic bias loss and spatial coordinate offset loss, thereby synchronizing the model's semantic output and localization output. The adversarial perturbation optimization module is used to apply a gradient-based iterative optimization algorithm under a preset perturbation amplitude constraint. The gradient of the joint objective function is backpropagated from the model output layer to the input GUI image layer, and the optimized adversarial perturbation is obtained by updating it time by time. The adversarial example generation module is used to overlay optimized adversarial perturbations onto the original GUI image to generate adversarial examples that can be used to manipulate the decision-making behavior and click positions of the GUI Agent.

[0018] To achieve the above objectives, a third aspect of the present invention provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the method described in the first aspect.

[0019] The method, system, and storage medium of this invention generate adversarial perturbations at the image pixel layer that are imperceptible to the human eye, which can bypass code layer security detection and have extremely strong concealment. The generated adversarial perturbations can also provide efficient security assessment and robustness testing methods for multimodal GUI agents, effectively revealing security vulnerabilities at the perception layer of visual language models. It is applicable to various GUI agent systems based on visual language models and has important security testing value and wide applicability.

[0020] Additional aspects and advantages of the invention will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of the invention. Attached Figure Description

[0021] The above and / or additional aspects and advantages of the present invention will become apparent and readily understood from the following description of the embodiments taken in conjunction with the accompanying drawings, wherein: Figure 1 This is a flowchart of a gradient-induced multimodal GUI agent image layer white-box adversarial attack method according to an embodiment of the present invention; Figure 2 This is a structural diagram of a gradient-induced multimodal GUI Agent image layer white-box adversarial attack system according to an embodiment of the present invention; Figure 3 This is a flowchart of a gradient-induced multimodal GUI agent image layer white-box adversarial attack method. Detailed Implementation

[0022] It should be noted that, unless otherwise specified, the embodiments and features described in the present invention can be combined with each other. The present invention will now be described in detail with reference to the accompanying drawings and embodiments.

[0023] To enable those skilled in the art to better understand the present invention, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present invention.

[0024] The following description, with reference to the accompanying drawings, describes a gradient-induced multimodal GUIAgent image layer white-box adversarial attack method and system proposed according to an embodiment of the present invention.

[0025] Example 1 This invention proposes a gradient-induced white-box adversarial attack method for multimodal GUI agents at the image layer. It primarily generates adversarial perturbations imperceptible to the human eye, manipulating the semantic understanding and spatial perception of the GUI agent to induce it to execute a specified malicious intent and precisely bias its output click coordinates. This provides an effective technical means for the security assessment of multimodal GUI agents, revealing the inherent vulnerability of visual language models at the perceptual level. The specific steps of this method include: S1 is a pre-defined dual attack target that includes malicious operation semantics and target interaction coordinates.

[0026] Specifically, the attack targets are defined, mainly including sequences of malicious intent. T_adv and malicious clicks on target coordinates C_ target = (x', y'), which correspond to the operation text to be executed by the GUI Agent and the screen pixel position to be targeted for click, respectively.

[0027] Among them, malicious intent sequence T_adv The text description of the operation intended to induce the GUI Agent to perform (such as "click the transfer button", "confirm payment", etc.); malicious click on target coordinates C_target The screen position where the GUI Agent is to be clicked is represented in pixel coordinates. By targeting the dual objectives mentioned above, the text output (semantic understanding) and coordinate output (spatial awareness) of the GUI Agent can be manipulated simultaneously, thereby achieving comprehensive control over the Agent's behavior.

[0028] S2. Construct a joint optimization objective function that integrates text semantic bias loss and spatial coordinate offset loss to synchronously constrain the model's semantic output and localization output.

[0029] Specifically, the joint optimization objective function is: L_total = λ 1 ·L_text(f(X+δ), T_adv) + λ 2 ·L_coord(g(X+δ), C_target); Where X is the pixel tensor of the original GUI screenshot, and δ is the adversarial perturbation to be optimized. f(·) This represents the text output function of the model. g(·) The coordinate output function of the model. L_text The text cross-entropy loss is used to measure the difference between the model's output text and malicious intent. T_adv The differences between them; L_coord The mean squared error loss for coordinate prediction is used to measure the difference between the model's output coordinates and the target coordinates. C_target The Euclidean distance between them; l 1 and l 2 is the weight hyperparameter for balancing the two optimization objectives.

[0030] S3, under the preset perturbation amplitude constraint, adopts a gradient-based iterative optimization algorithm to backpropagate the gradient of the joint objective function from the model output layer to the input original GUI image, and successively updates to obtain the optimized adversarial perturbation; Specifically, step S3 includes: S31, initialize the adversarial interference values, and calculate the gradient of the joint optimization objective function with respect to the original GUI image, using the input image pixel tensor as the independent variable. L_total / X.

[0031] As one implementation method, the method for initializing the anti-interference value is to set the disturbance δ. (0) = 0, or obtain the initial anti-interference value by random sampling from the uniform distribution U(-ε, ε).

[0032] S32, based on the projection gradient descent algorithm, iteratively updates the perturbation along the opposite direction of the gradient and projects the updated perturbation into the L_∞ norm sphere.

[0033] Specifically, the calculation formula for iteratively updating the perturbation along the opposite direction of the gradient is as follows: δ (t+ ¹ ) = δ (t) α sign ( L_total / X); Where α is a fixed step size parameter, δ (t) For the adversarial perturbation at time t, sign ( ) is a symbolic function.

[0034] Then, a projection operation is performed, and the corresponding formula is: δ (t+ ¹ ) = Clip(δ (t+ ¹ ) , -ε, ε), Where ε is the upper bound of the disturbance amplitude. To ensure that the disturbance is invisible, the upper bound of the disturbance amplitude is usually set to a small value (such as 8 / 255 or 16 / 255).

[0035] Furthermore, as another implementation method, algorithms for iteratively updating the perturbation also include the momentum iterative fast gradient sign method, the C&W attack method, and the AutoAttack method.

[0036] S33. Repeat steps S31-S32 until the joint optimization objective function converges to a stable value or the number of iterations reaches a preset value, thus obtaining the optimized adversarial perturbation.

[0037] To further enhance the perturbation strength, this invention replicates the activation feature pattern of the target GUI component in the high-dimensional feature space of the model by fine-tuning the pixel values ​​of a specific region, enabling the GUI Agent to recognize 'false anchors' as legitimate interactive objects.

[0038] The specific steps include: A31 extracts the feature activation patterns of the target components in the intermediate layer of the visual encoder, where the target components include buttons, links, etc.

[0039] A32 adds a feature matching loss term to the optimization objective, enabling adversarial examples to generate feature responses similar to the target component at specified locations.

[0040] A33 incorporates feature matching loss into the optimization objective, hijacks the visual anchors of the GUI Agent, and guides the perturbation distribution in a directional manner to achieve the 'anchor hijacking' effect at the visual feature level.

[0041] S4. The optimized adversarial perturbation is superimposed on the original GUI image to generate adversarial examples that can be used to manipulate the decision-making behavior and click positions of the GUI Agent.

[0042] The optimized adversarial perturbation δ is superimposed onto the original GUI screenshot X, resulting in the adversarial sample X_adv = X + δ. This adversarial sample is visually almost indistinguishable from the original screenshot, but it can effectively induce the GUI Agent to perform malicious operations specified by the attacker.

[0043] S5 uses adversarial examples to assess the security of the GUI Agent.

[0044] As an optional extension, this method can be combined with ensemble learning of the target model to improve the transferability of the attack by attacking multiple GUI Agent models simultaneously, so that the generated adversarial examples can produce attack effects on unknown models in black-box scenarios.

[0045] Example 2 To further illustrate the method for security assessment of GUI agents using adversarial examples in this invention, the attack process is described below using a specific attack embodiment as an example. The attack process includes the following steps: (a) Input the original GUI screenshot (X). For example... Figure 2 The screenshot shown is of the original GUI, which is a screenshot of a payment confirmation interface in the upper left corner, including the current interface information, amount details, and 'Cancel' and 'Accept' buttons.

[0046] (b) The original GUI screenshots are used to extract features through a visual encoder based on a CNN / Transformer architecture to generate visual feature representations.

[0047] (c) After the visual features are mapped through the projection layer, they are combined with the language model to generate operational decisions.

[0048] (d) Calculate the text cross-entropy loss L_text (used for semantic induction, with the target being the adversarial instruction T_adv) and the coordinate mean square error loss L_coord (used for coordinate partial derivative, with the target being the malicious click coordinates C_target).

[0049] (e) Calculate the gradient of the input image using the total loss L_total. L_total / X), gradient information is backpropagated from the output layer to the input image layer.

[0050] (f) Use gradient information to iteratively update the anti-perturbation δ using the projected gradient descent algorithm.

[0051] (g) Generate adversarial example images (X+δ), such as Figure 2 The bottom right corner shows the adversarial sample after adding adversarial perturbations. Visually, it is almost indistinguishable from the original screenshot, but it will induce the GUI Agent to make incorrect operational decisions.

[0052] Example 3 like Figure 3 As shown, this invention proposes a gradient-induced multimodal GUI agent image layer white-box adversarial attack system 10, comprising: The attack target construction module 101 is used to preset a dual attack target containing malicious operation semantics and target interaction coordinates; The joint optimization objective function 102 is used to construct a joint optimization objective function that integrates text semantic bias loss and spatial coordinate offset loss, and to impose synchronous constraints on the model's semantic output and localization output. The adversarial perturbation optimization module 103 is used to, under the preset perturbation amplitude constraint, adopt a gradient-based iterative optimization algorithm to backpropagate the gradient of the joint objective function from the model output layer to the input GUI image layer, and successively update to obtain the optimized adversarial perturbation; The adversarial example generation module 104 is used to superimpose the optimized adversarial perturbation onto the original GUI image to generate adversarial examples that can be used to manipulate the decision-making behavior and click positions of the GUI Agent.

[0053] The present invention also provides a computer-readable storage medium storing a computer program, which, when executed by a processor, implements the above-described method for gradient-induced multimodal GUI agent image layer white-box adversarial attacks.

[0054] In the description of this specification, the references to terms such as "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., refer to specific features, structures, materials, or characteristics described in connection with that embodiment or example, which are included in at least one embodiment or example of the present invention. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples. Moreover, without contradiction, those skilled in the art can combine and integrate the different embodiments or examples described in this specification, as well as the features of different embodiments or examples.

[0055] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include at least one of that feature. In the description of this invention, "a plurality of" means at least two, such as two, three, etc., unless otherwise explicitly specified.

Claims

1. A gradient-induced multimodal GUI agent image layer white-box adversarial attack method, characterized in that, Includes the following steps: S1 is a pre-defined dual attack target that includes malicious operation semantics and target interaction coordinates; S2, construct a joint optimization objective function that integrates text semantic bias loss and spatial coordinate offset loss, and impose synchronous constraints on the model's semantic output and localization output; S3, under the preset perturbation amplitude constraint, adopts a gradient-based iterative optimization algorithm to backpropagate the gradient of the joint objective function from the model output layer to the input original GUI image, and successively updates to obtain the optimized adversarial perturbation; S4. The optimized adversarial perturbation is superimposed on the original GUI image to generate adversarial examples that can be manipulated in a targeted manner to control the decision-making behavior and click position of the GUI Agent; S5 uses adversarial examples to assess the security of the GUI Agent.

2. The method according to claim 1, characterized in that, S3 further includes: S31, initialize the adversarial interference value, and calculate the gradient of the joint optimization objective function with respect to the original GUI image using the input image pixel tensor as the independent variable; S32, based on the projection gradient descent algorithm, iteratively updates the perturbation along the opposite direction of the gradient, and projects the updated perturbation into the L_∞ norm sphere; S33. Repeat steps S31-S32 until the joint optimization objective function converges to a stable value or the number of iterations reaches a preset value, thus obtaining the optimized adversarial perturbation.

3. The method according to claim 2, characterized in that, The calculation formula for iteratively updating the perturbation along the opposite direction of the gradient in step S32 is as follows: d (t+ ¹ ) = d (t) a sign ( L_total / X); Where α is a fixed step size parameter, δ (t) For the adversarial perturbation at time t, sign ( ) is a symbolic function.

4. The method according to claim 1, characterized in that, The dual attack targets in step S1 include: the malicious operation text sequence to be induced and the malicious click pixel coordinates to be targeted.

5. The method according to claim 1, characterized in that, The joint optimization objective function described in step S2 is: L_total = λ 1 ·L_text(f(X+δ), T_adv) + λ 2 ·L_coord(g(X+δ), C_target); Where X is the pixel tensor of the original GUI screenshot, and δ is the adversarial perturbation to be optimized. f(·) This represents the text output function of the model. g(·) The coordinate output function of the model. L_text For text cross-entropy loss, L_coord This represents the mean square error loss in coordinate prediction. λ 1 and λ 2 is the weight hyperparameter for balancing the two optimization objectives.

6. The method according to claim 2, characterized in that, The algorithm for iteratively updating the perturbation described in step S32 also includes the momentum iterative fast gradient sign method, the C&W attack method, and the AutoAttack method.

7. The method according to claim 1, characterized in that, Step S3 also includes: Extract the intermediate visual activation features of the target GUI component, add feature matching loss to the optimization target, and hijack the visual anchors of the GUIAgent.

8. The method according to claim 7, characterized in that, The method for hijacking the visual anchor point of the GUI Agent is as follows: A31, extract the feature activation patterns of the target component in the intermediate layer of the visual encoder; A32 adds a feature matching loss term to the optimization objective, enabling adversarial examples to generate feature responses similar to the target component at specified locations; A33 incorporates feature matching loss into the optimization objective, hijacks the visual anchor points of the GUI Agent, and guides the perturbation distribution in a targeted manner.

9. A gradient-induced multimodal GUI agent image layer white-box adversarial attack system, characterized in that, include: The attack target construction module is used to pre-set dual attack targets that include malicious operation semantics and target interaction coordinates; A joint optimization objective function is used to construct a joint optimization objective function that integrates text semantic bias loss and spatial coordinate offset loss, thereby synchronizing the model's semantic output and localization output. The adversarial perturbation optimization module is used to apply a gradient-based iterative optimization algorithm under a preset perturbation amplitude constraint. The gradient of the joint objective function is backpropagated from the model output layer to the input GUI image layer, and the optimized adversarial perturbation is obtained by updating it time by time. The adversarial example generation module is used to overlay optimized adversarial perturbations onto the original GUI image to generate adversarial examples that can be used to manipulate the decision-making behavior and click positions of the GUI Agent.

10. A computer-readable storage medium storing a computer program that, when executed by a processor, implements the method as claimed in any one of claims 1-8.

Images