A GUI agent defense method based on multi-modal semantic consistency verification and action sandbox

Abstract

The application relates to the technical field of artificial intelligence security and autonomous agent systems, in particular to a GUI Agent defense method based on multi-modal semantic consistency verification and action sandboxing. The method first pre-processes interface images input by a GUI Agent, eliminates adversarial perturbations and retains key visual features of the interface, and obtains reinforced interface images; visual semantic features of the reinforced interface images are extracted, and consistency verification and timing logic verification are performed in combination with semantic features of underlying structure data, so that operation instructions passing the verification are obtained; the operation instructions are subjected to sensitive action matching, sandbox simulation execution and manual confirmation, and executable safe operation instructions are obtained. Through the 'bidirectional cross verification' mechanism of visual semantics and underlying structure, the application effectively identifies various highly concealed pixel-level adversarial attacks and semantic deception attacks, meanwhile, multi-modal cross verification significantly improves the accuracy and recall rate of attack detection, and effectively makes up for the visual-structure semantic blind area.

Description

Technical Field

[0001] The invention relates to the field of artificial intelligence security and autonomous agent system technology, and in particular to a GUI agent defense method based on multimodal semantic consistency verification and action sandbox. Background Technology

[0002] With the rapid development of large-scale language models and visual language models, graphical user interface (GUI) agents based on multimodal understanding have become a core technology in the fields of human-computer interaction and automation. These agents can parse screenshots, understand user commands, and output interface operation intentions and interaction coordinates. They are widely used in scenarios such as intelligent assistants, process automation, and assisted operation, significantly improving the efficiency of human-computer interaction and the level of system automation. Current mainstream GUI agents use visual language models as the decision-making core, completing interface interactions through visual feature extraction, multimodal semantic reasoning, and action command generation, demonstrating strong understanding and execution capabilities in complex interface environments.

[0003] However, deep learning-based GUI agents inherently possess security vulnerabilities and are susceptible to various adversarial attacks. Attackers can use techniques such as pixel-level adversarial perturbations, visual malicious injection, and interface coordinate hijacking to induce the model to generate incorrect semantic understanding and operation positioning, leading to dangerous behaviors such as privacy leaks, illegal operations, and unauthorized access. Existing AI security protection technologies are mostly geared towards pure text large model prompt injection, single-modal image model adversarial sample detection, or traditional rule-based permission control designs. They are not adapted to the multimodal interaction chain of GUI agents, which consists of "visual screenshots + underlying structure + action sequences," making it difficult to effectively resist visual deception and cross-modal attacks.

[0004] Current protection solutions have significant technical flaws: First, they only target a single modality, failing to detect semantic conflicts between the visual rendering layer and the underlying code logic, making it difficult to intercept covert coordinate hijacking attacks. Second, they lack a security interception mechanism before action execution; dangerous commands are executed directly once generated, lacking a safety net. Third, input preprocessing cannot retain key interface features while eliminating adversarial noise, and it lacks contextual validation based on operation timing, making it unable to identify abnormal inducement sequences that violate business logic. These issues make it difficult to guarantee the security of GUI agents in adversarial environments, hindering their large-scale application in high-security scenarios such as finance, office work, and system management. Summary of the Invention

[0005] The present invention aims to at least partially solve one of the technical problems in the related art.

[0006] Therefore, the first objective of this invention is to propose a GUI Agent defense method based on multimodal semantic consistency verification and action sandbox, comprising:

[0007] S1, preprocess the interface image input by the GUI Agent to eliminate adversarial disturbances and retain key visual features of the interface to obtain the hardened interface image; S2, extract the visual semantic features of the reinforced interface image, and combine them with the semantic features of the underlying structure data to perform consistency verification and timing logic verification, and obtain the operation instructions that pass the verification. S3 performs sensitive action matching, sandbox simulation execution, and manual confirmation on the operation instructions to obtain executable and safe operation instructions.

[0008] In one embodiment of the present invention, S1 further includes: S11 employs an adaptive edge-preserving filter based on local pixel gradients to strongly smooth and denoise flat areas of the image while weakly smoothing and preserving features in edge areas. S12 performs non-linear randomized resampling on the image, disrupting the pixel-level adversarial perturbation mode.

[0009] In one embodiment of the present invention, S2 further includes: S21, extract visual semantic feature vectors from interface images and extract structural semantic feature vectors from underlying structural data; S22, calculate the cosine similarity between the two types of feature vectors. If the similarity is lower than the threshold, it is judged as a semantic conflict and blocked. S23, combined with the operation history queue, verify the rationality of the current action execution sequence, identify abnormal inducement sequences, and obtain operation instructions.

[0010] In one embodiment of the present invention, the formula for calculating the cosine similarity is: Similarity = cos(V_v, V_s) = (V_v · V_s) / (||V_v|| × ||V_s||) ; in, V_v For visual feature vectors, V_s These are structural feature vectors.

[0011] In one embodiment of the present invention, S3 further includes: S31, Establish a list of sensitive actions containing various high-risk operations, match operation instructions with the list of sensitive actions, and identify high-risk operations; S32 imports high-risk operations into an isolation sandbox for simulated execution and pops up an independent pop-up window that is not controlled by the agent. After the user manually confirms the second time, the operation is allowed to proceed, and a safe operation command is generated and executed. S33 logs exceptions and interception behaviors.

[0012] In one embodiment of the present invention, the nonlinear randomized resampling in S12 includes at least one of random resolution scaling, random quality factor compression, random cropping and padding, and random color space perturbation.

[0013] In one embodiment of the present invention, the sensitive action library mentioned in step S31 includes at least one of the following: financial payment, data export, permission change, and high-risk system operation.

[0014] To achieve the above objectives, a second aspect of the present invention proposes a GUI Agent defense system based on multimodal semantic consistency verification and action sandboxing, comprising: The visual perception hardening layer is used to preprocess the interface image input by the GUI Agent, eliminate adversarial perturbations and retain key visual features of the interface to obtain a hardened interface image. The semantic consistency cross-validation layer is used to extract the visual semantic features of the reinforced interface image and combine them with the semantic features of the underlying structural data to perform consistency verification and temporal logic verification, and obtain the operation instructions that pass the verification. Sensitive action sandbox and interception layer are used to perform sensitive action matching, sandbox simulation execution and manual confirmation on operation instructions to obtain executable safe operation instructions.

[0015] In one embodiment of the present invention, the semantic consistency cross-validation module includes: The dual-path feature extraction module is used to extract visual semantic feature vectors from interface images and structural semantic feature vectors from underlying structural data. The semantic consistency conflict detection module is used to calculate the cosine similarity between two types of feature vectors. If the similarity is lower than the threshold, it is judged as a semantic conflict and blocked. The timing logic verification module is used to combine the operation history queue to verify the rationality of the timing of the current action, identify abnormal inducement sequences, and obtain operation instructions.

[0016] To achieve the above objectives, a third aspect of the present invention provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the method described in the first aspect.

[0017] The method, system, and storage medium of this invention employ a three-layer defense-in-depth architecture—input preprocessing, multimodal semantic verification, and sensitive action sandbox auditing—to eliminate pixel-level adversarial disturbances while preserving key visual features of the interface. This architecture achieves bidirectional cross-verification of visual semantics and underlying structural semantics, and, combined with operation timing logic detection of anomalies, accurately identifies and intercepts covert attacks such as coordinate hijacking, visual injection, and unauthorized operations, significantly improving the defense accuracy and coverage of the GUI Agent. Simultaneously, the use of lightweight algorithms and modular design enables low-cost and rapid integration with existing systems without significantly increasing system latency and computational overhead. Furthermore, sandbox isolation and mandatory secondary manual confirmation mechanisms provide a safety net for high-risk operations, ensuring the user's ultimate control over critical operations. This forms a closed-loop protection system with full lifecycle and progressive layers. The overall solution combines strong robustness, high practicality, and good scalability, effectively ensuring the stable, reliable, and secure operation of the multimodal GUI Agent in high-security scenarios such as finance and office environments.

[0018] Additional aspects and advantages of the invention will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of the invention. Attached Figure Description

[0019] The above and / or additional aspects and advantages of the present invention will become apparent and readily understood from the following description of the embodiments taken in conjunction with the accompanying drawings, wherein: Figure 1 This is a flowchart of a GUIAgent defense method based on multimodal semantic consistency verification and action sandbox according to an embodiment of the present invention; Figure 2 This is a structural diagram of a GUIAgent defense system based on multimodal semantic consistency verification and action sandbox according to an embodiment of the present invention; Figure 3 This is a structural diagram of the semantic consistency conflict detection module of the present invention; Figure 4 This is a block diagram of the overall architecture of the multi-level defense system of this invention. Detailed Implementation

[0020] It should be noted that, unless otherwise specified, the embodiments and features described in the present invention can be combined with each other. The present invention will now be described in detail with reference to the accompanying drawings and embodiments.

[0021] To enable those skilled in the art to better understand the present invention, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present invention.

[0022] The following describes, with reference to the accompanying drawings, a GUI Agent defense method and system based on multimodal semantic consistency verification and action sandbox, according to an embodiment of the present invention.

[0023] Example 1 To address the aforementioned deficiencies in existing technologies, this invention aims to provide a GUI Agent defense method based on multimodal semantic consistency verification and action sandboxing. This method constructs a multi-layered security defense architecture. Through intelligent denoising preprocessing of input screenshots, cross-validation of visual semantics and underlying metadata, and sandbox auditing and physical interception of high-risk actions, it comprehensively enhances the security and robustness of the GUI Agent in complex adversarial environments, effectively defending against various security threats such as pixel-level adversarial attacks, coordinate hijacking attacks, and unauthorized operations.

[0024] like Figure 1 As shown, the present invention proposes a GUIAgent defense method based on multimodal semantic consistency verification and action sandbox, comprising: S1: Preprocess the interface image input by the GUI Agent to eliminate adversarial disturbances and retain key visual features of the interface to obtain a hardened interface image.

[0025] Specifically, step S1 includes: S11: Adaptive edge-preserving filtering based on local pixel gradients is used to strongly smooth and denoise flat areas of the image while weakly smoothing and preserving features in edge areas.

[0026] To preserve key visual features while eliminating noise, this invention employs an adaptive edge-preserving filtering algorithm to process the input image. This algorithm analyzes local pixel gradient information and adaptively adjusts the filtering intensity: a stronger smoothing filter is used in flat areas to eliminate high-frequency adversarial noise; a weaker filter is used in edge areas (such as button outlines and text boundaries) to preserve key visual features. As one implementation method, the filtering method can be implemented using algorithms such as bilateral filtering, guided filtering, and total variational denoising.

[0027] This step ensures that adversarial disturbances are effectively filtered out without affecting the model's normal positioning accuracy for GUI elements.

[0028] S12: Performs non-linear randomized resampling on the image, disrupting pixel-level adversarial perturbation mode.

[0029] As a specific implementation method, nonlinear randomized resampling can employ random resolution resampling (randomly selecting scaling ratios within a certain range), JPEG lossy compression (random quality factor), random cropping and padding, and random color space perturbation. These nonlinear operations can effectively disrupt the attacker's carefully designed pixel-level perturbation patterns, reducing the success rate of adversarial attacks.

[0030] S2: Extract the visual semantic features of the reinforced interface image, and combine them with the semantic features of the underlying structural data to perform consistency verification and timing logic verification, and obtain the operation instructions that pass the verification.

[0031] Specifically, step S2 includes: S21: Extract visual semantic feature vectors from interface images and extract structural semantic feature vectors from underlying structural data.

[0032] Specifically, a visual semantic feature vector is obtained by extracting features from screenshots of the GUI rendering layer using a visual encoder. V_v This represents the interface content that the Agent "sees" from a visual perspective.

[0033] Extract structured semantic feature vectors from underlying structured information sources (such as DOM trees, accessibility trees, UI automation interfaces, etc.). V_s It represents the actual code logic and attribute information of interface elements.

[0034] S22: Calculate the cosine similarity between the two feature vectors. If the similarity is lower than the threshold, it is determined to be a semantic conflict and the vector is blocked.

[0035] Specifically, this invention utilizes a lightweight neural network detection model to calculate visual feature vectors. V_v With structural feature vectors V_s Semantic cosine similarity between them: Similarity = cos(V_v, V_s) = (V_v · V_s) / (||V_v|| × ||V_s||) ; When the similarity is below a preset threshold τ, the system determines that there is a visual-structural semantic conflict. For example, if VLM visually recognizes the semantics of the 'Pay Now' button, but the underlying DOM structure shows that the coordinate area is actually a "transparent overlay" or a "hidden element", the system determines it as a potential coordinate hijacking attack and immediately reduces the trust weight of this decision or directly blocks it.

[0036] S23: Combine the operation history queue to verify the rationality of the current action execution sequence, identify abnormal inducement sequences, and obtain operation instructions.

[0037] The system performs a timing validity check on the current action by combining the historical operation context H. The system maintains an operation history queue, recording the sequence of actions recently executed by the Agent. For the current decision, the system checks whether it conforms to normal operation timing logic (e.g., "login" should precede "payment," "confirmation" should follow "fill in information," etc.). If an abnormal action violating the timing logic is detected, it is identified as a potential inducement attack and an alarm is triggered.

[0038] S3: Perform sensitive action matching, sandbox simulation execution, and manual confirmation on the operation instructions to obtain executable and safe operation instructions.

[0039] Specifically, step S3 includes: S31: Establish a list of sensitive actions that includes various high-risk operations, match operation instructions with the list of sensitive actions, and identify high-risk operations.

[0040] As one implementation method, the sensitive action list includes the following operations: financial payment (transfer, payment, withdrawal, etc.), data export (download, export, share, etc.), permission change (authorization, deletion, password modification, etc.), and system operation (installation, uninstallation, formatting, etc.). The list can be adjusted according to actual needs during use.

[0041] As another implementation, the present invention can update the preset threshold τ and the list of sensitive actions online based on the actual detected attack samples.

[0042] The user's actions are output as the Agent and matched with each action in the sensitive action list to identify whether a sensitive action threshold is triggered. If the threshold is triggered, the user's action is considered to have successfully matched a sensitive action and is considered to be a high-risk action.

[0043] S32: Import high-risk operations into an isolation sandbox for simulated execution, and pop up an independent pop-up window that is not controlled by the agent. After the user manually confirms the operation a second time, the operation is allowed to proceed, and a safe operation instruction is generated and executed.

[0044] Specifically, when a sensitive action (or high-risk operation) is detected, it is simulated and executed in an isolated sandbox environment. The sandbox environment replicates the interface and interaction logic of the real system, but is completely isolated from the actual system, and no operation will have a real impact. Through sandbox simulation, the potential consequences of actions can be predicted, and abnormal behavior patterns can be identified.

[0045] Furthermore, in order to improve operational security without affecting the user experience, for operations that trigger sensitive actions, this invention pops up a physical confirmation pop-up on the interface that is independent of Agent control, forcing the user to perform a second manual confirmation.

[0046] This pop-up window has the following characteristics: independent rendering (unaffected by agent operations), explicit notification (clearly displaying the upcoming operation and potential risks), and mandatory interaction (requiring the user to physically click to confirm or cancel). The system only allows the actual operation to be performed after explicit user confirmation; otherwise, the action is intercepted and logged.

[0047] S33: Log out exceptions and interception behaviors.

[0048] Specifically, for anomalies detected at the semantic consistency verification layer or action audit layer, the system generates alarm information in real time and records detailed logs, including: anomaly type, detection time, relevant screenshots, Agent output content, underlying structure information, etc. The logs are used for subsequent security audits and attack tracing analysis.

[0049] Example 2 like Figure 2 As shown, a GUI Agent defense system 10 based on multimodal semantic consistency verification and action sandbox includes: The visual perception hardening layer 100 is used to preprocess the interface image input by the GUI Agent, eliminate adversarial disturbances and retain key visual features of the interface to obtain a hardened interface image. The semantic consistency cross-validation layer 200 is used to extract the visual semantic features of the reinforced interface image and combine them with the semantic features of the underlying structural data to perform consistency verification and temporal logic verification, and obtain the operation instructions that pass the verification. The sensitive action sandbox and interception layer 300 are used to perform sensitive action matching, sandbox simulation execution, and manual confirmation on operation instructions to obtain executable and safe operation instructions.

[0050] Furthermore, the semantic consistency cross-validation module 200 includes: The dual-path feature extraction module 201 is used to extract visual semantic feature vectors from interface images and structural semantic feature vectors from underlying structural data. The semantic consistency conflict detection module 202 is used to calculate the cosine similarity between two types of feature vectors. If the similarity is lower than the threshold, it is determined to be a semantic conflict and intercepted.

[0051] As one implementation method, such as Figure 3 As shown, the semantic consistency conflict detection module 202 includes: (a) Visual branch: Visual semantic feature vectors are extracted from GUI screenshots through a visual encoder. V_v , represents the visual presentation content of the interface.

[0052] (b) Structural branch: Extracting structured semantic feature vectors from underlying data sources such as DOM / accessible tree. V_s, which represents the actual code attributes of interface elements.

[0053] (c) Decision Center: In the decision center module, the calculation is performed first. V_v and V_s The cosine similarity is used. If the similarity is below a threshold, it is directly identified as a pixel hijacking attack and blocked; if the similarity is above the threshold, a temporal logic check is performed in conjunction with the historical context H. If the action conforms to the context logic, it is allowed; if it does not conform to the context logic, it is identified as an abnormal inducement and blocked.

[0054] The timing logic verification module 203 is used to verify the rationality of the timing of the current action by combining the operation history queue, identify abnormal inducement sequences, and obtain operation instructions.

[0055] Furthermore, as another implementation, the system also includes an adversarial training enhancement module, which uses adversarial examples to perform adversarial training on the visual encoder and semantic detection model in the semantic consistency cross-validation layer 200, thereby improving the robustness of the model itself.

[0056] As another implementation method, the system can also introduce multiple independent agents for cross-validation, and the operation is only executed when the decisions of most agents are consistent.

[0057] Example 3 To illustrate the system proposed in this invention, specific examples are provided below.

[0058] like Figure 4 The diagram shown is the overall architecture of a multi-level GUI Agent defense system, which includes three main levels: (a) Input layer: Receives a screenshot X and operation history / context H as input. The screenshot X is preprocessed by the visual perception hardening module (edge-preserving denoising, non-linear resampling, etc.), and the context H is encoded by the context encoder.

[0059] (b) Operation Monitoring Layer: The preprocessed image is input into the multimodal large model (VLM) for visual semantic understanding, while the system obtains metadata / DOM structure information from the underlying layer. The two sets of information are cross-compared in the semantic consistency verification module to determine whether there is a visual-structural semantic conflict. If the detection passes, the process continues; if a conflict is detected, an anomaly warning / interception mechanism is triggered.

[0060] (c) Action Audit Layer: For decisions that pass semantic consistency verification, a compliance check for sensitive actions is first performed. If the action is determined to be high-risk, it enters the action sandbox simulation execution stage; if the sandbox execution result is safe, a pop-up window for secondary manual confirmation requests user authorization; only after the user explicitly gives permission will the physical device execute the actual operation.

[0061] The present invention also provides a computer-readable storage medium storing a computer program, which, when executed by a processor, implements the above-described method for GUI Agent defense based on multimodal semantic consistency verification and action sandbox.

[0062] In the description of this specification, the references to terms such as "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., refer to specific features, structures, materials, or characteristics described in connection with that embodiment or example, which are included in at least one embodiment or example of the present invention. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples. Moreover, without contradiction, those skilled in the art can combine and integrate the different embodiments or examples described in this specification, as well as the features of different embodiments or examples.

[0063] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include at least one of that feature. In the description of this invention, "a plurality of" means at least two, such as two, three, etc., unless otherwise explicitly specified.

Claims

1. A GUI Agent defense method based on multimodal semantic consistency verification and action sandbox, characterized in that, include: S1, preprocess the interface image input by the GUI Agent to eliminate adversarial disturbances and retain key visual features of the interface to obtain the hardened interface image; S2, extract the visual semantic features of the reinforced interface image, and combine them with the semantic features of the underlying structure data to perform consistency verification and timing logic verification, and obtain the operation instructions that pass the verification. S3 performs sensitive action matching, sandbox simulation execution, and manual confirmation on the operation instructions to obtain executable and safe operation instructions.

2. The method according to claim 1, characterized in that, S1 further includes: S11 employs an adaptive edge-preserving filter based on local pixel gradients to strongly smooth and denoise flat areas of the image while weakly smoothing and preserving features in edge areas. S12 performs non-linear randomized resampling on the image, disrupting the pixel-level adversarial perturbation mode.

3. The method according to claim 1, characterized in that, S2 also includes: S21, extract visual semantic feature vectors from interface images and extract structural semantic feature vectors from underlying structural data; S22, calculate the cosine similarity between the two types of feature vectors. If the similarity is lower than the threshold, it is judged as a semantic conflict and blocked. S23, combined with the operation history queue, verify the rationality of the current action execution sequence, identify abnormal inducement sequences, and obtain operation instructions.

4. The method according to claim 3, characterized in that, The formula for calculating the cosine similarity is: Similarity = cos(V_v, V_s) = (V_v · V_s) / (||V_v|| × ||V_s||) ； in, V_v For visual feature vectors, V_s These are structural feature vectors.

5. The method according to claim 1, characterized in that, S3 also includes: S31, Establish a list of sensitive actions containing various high-risk operations, match operation instructions with the list of sensitive actions, and identify high-risk operations; S32 imports high-risk operations into an isolation sandbox for simulated execution and pops up an independent pop-up window that is not controlled by the agent. After the user manually confirms the second time, the operation is allowed to proceed, and a safe operation command is generated and executed. S33 logs exceptions and interception behaviors.

6. The method according to claim 2, characterized in that, The nonlinear randomized resampling described in S12 includes at least one of random resolution scaling, random quality factor compression, random cropping and padding, and random perturbation of the color space.

7. The method according to claim 4, characterized in that, Step S31 The sensitive action library includes at least one of the following: financial payment, data export, permission change, and high-risk system operation.

8. A GUI Agent defense system based on multimodal semantic consistency verification and action sandbox, characterized in that, include: The visual perception hardening layer is used to preprocess the interface image input by the GUI Agent, eliminate adversarial perturbations and retain key visual features of the interface to obtain a hardened interface image. The semantic consistency cross-validation layer is used to extract the visual semantic features of the reinforced interface image and combine them with the semantic features of the underlying structural data to perform consistency verification and temporal logic verification, and obtain the operation instructions that pass the verification. Sensitive action sandbox and interception layer are used to perform sensitive action matching, sandbox simulation execution and manual confirmation on operation instructions to obtain executable safe operation instructions.

9. The method according to claim 8, characterized in that, The semantic consistency cross-validation module includes: The dual-path feature extraction module is used to extract visual semantic feature vectors from interface images and structural semantic feature vectors from underlying structural data. The semantic consistency conflict detection module is used to calculate the cosine similarity between two types of feature vectors. If the similarity is lower than the threshold, it is judged as a semantic conflict and blocked. The timing logic verification module is used to combine the operation history queue to verify the rationality of the timing of the current action, identify abnormal inducement sequences, and obtain operation instructions.

10. A computer-readable storage medium storing a computer program that, when executed by a processor, implements the method as claimed in any one of claims 1-7.

Images