A method and device for robustness evaluation of a large visual language model based on a double-layer guided sparse strategy

Abstract

The application discloses a kind of big visual language model robustness evaluation method and equipment based on double-layer guiding sparse strategy, proposes an innovative scheme for generating transferable evaluation samples in big visual language model.The method uses double-layer mask screening mechanism, generates sparse disturbance through group level and pixel level mask;Adopt three-stage optimization process, generate adversarial text, optimize adversarial image and generate final adversarial text in turn, construct adversarial image text pair.At the same time, the system includes data processing module, mask generation module, adversarial sample generation module and robustness evaluation module, each module works cooperatively.Double-layer mask mechanism can accurately control disturbance of different scales, improve the effectiveness of evaluation samples in evaluation.The method innovatively applies sparse disturbance to big visual language model robustness evaluation, provides a new technical means for accurately measuring the ability of model to resist adversarial attack, and helps to promote the improvement of big visual language model robustness performance.

Description

Technical Field

[0001] This invention belongs to the field of artificial intelligence security technology, specifically relating to the field of multimodal model robustness evaluation, and in particular to a method and device for robustness evaluation of large visual language models based on a two-layer guided sparse strategy. Background Technology

[0002] In recent years, with continuous breakthroughs in multimodal large models, large visual language models (LVLMs) that integrate image and text information have been widely used in practical scenarios such as image-text retrieval, intelligent question answering, and assisted creation. By jointly learning deep semantic representations of images and text, they demonstrate strong generalization and reasoning abilities in tasks such as image-text retrieval, visual question answering, and image description. However, this also makes them potential targets for attacks. Research shows that such models may produce erroneous responses when faced with carefully designed adversarial examples, leading to uncontrollable model output, manipulated behavior, and even serious security consequences.

[0003] Existing robustness evaluation methods for large visual language models mostly draw inspiration from unimodal tasks such as image classification, typically employing dense perturbation strategies. These strategies introduce minute changes to the overall input image and observe the resulting changes in the model's output. However, due to the complex cross-modal interaction mechanisms of large visual language models, their robustness evaluation is far more complex than that of unimodal tasks. Existing methods suffer from two main shortcomings when applied to large visual language models: first, the perturbation generation lacks specificity, failing to accurately target vulnerable areas in the model's decision-making and thus failing to accurately reflect potential robustness deficiencies; second, they lack cross-task and cross-model universality, leading to inconsistent evaluation results across different models or scenarios, thus limiting their value as a unified robustness evaluation framework.

[0004] To address the aforementioned issues, some existing technologies have attempted to enhance sample applicability by introducing mechanisms such as ensemble-level guided perturbations, but significant drawbacks remain: on the one hand, they sacrifice sample naturalness while improving generality, affecting the scientific rigor of the assessment; on the other hand, they rely on large perturbation budgets, resulting in high computational costs and insufficient practicality.

[0005] Existing multimodal robustness evaluation methods generally suffer from the following problems:

[0006] 1. Lack of fine-grained perturbation localization mechanism: The perturbation region of the evaluation samples often adopts random or global optimization strategies, without in-depth modeling of the relationship between the semantic structure of the input data and the model response. This results in the evaluation samples lacking specificity and failing to truly reflect the robustness boundary of the model.

[0007] 2. Limited evaluation scope and lack of cross-model consistency: Most evaluation methods are optimized for specific tasks or models, and the evaluation results are not comparable when facing different model architectures or training paradigms, which cannot support systematic robustness analysis.

[0008] Therefore, a robustness evaluation method with a precise perturbation screening mechanism, strong versatility, and high efficiency and low cost is needed. This method should be able to systematically quantify the robustness degradation performance of large visual language models under various input perturbation conditions while ensuring the naturalness and semantic consistency of input samples, providing a scientific basis for model optimization and security verification. Based on this need, this invention proposes a robustness evaluation method and device for large visual language models based on a two-layer guided sparse strategy. By collaboratively screening perturbation regions using pixel-level and group-level masks, it achieves systematic detection of potentially vulnerable regions in the model, improving the scientific rigor and applicability of the robustness evaluation. Summary of the Invention

[0009] This invention addresses the shortcomings of existing robustness evaluation techniques for large visual language models by proposing a robustness evaluation method and device based on a two-layer guided sparse strategy. This method employs a two-layer masking mechanism to perform perturbation optimization on the input data at both the group and pixel levels, generating highly natural and targeted evaluation samples to systematically detect the robustness performance of the visual language model under different input perturbation conditions.

[0010] The core theoretical basis of this invention lies in the fact that visual language models, within the image-text joint embedding space, exhibit high sensitivity to subtle semantic differences in input data. By appropriately introducing subtle perturbations, the behavioral changes of the model during semantic matching judgment can be observed, thereby scientifically and quantitatively evaluating its robustness level. To this end, this invention designs a dual-layer masking mechanism, identifying the regions most sensitive to the model's semantic decisions at both the group level and pixel-level fine-grained levels. This achieves precise perturbation control while maintaining the naturalness of the input samples, thereby systematically probing the performance changes and robustness defects of large visual language models under different input perturbation conditions.

[0011] The objective of this invention is achieved through the following technical solution:

[0012] A robustness evaluation method for large visual language models based on a two-layer guided sparse strategy includes the following steps:

[0013] S1: Extract the original image-text pairs from the dataset, preprocess the original images and text to obtain the embedding space representation;

[0014] S2: Construct the original text description set, generate adversarial text, calculate the contrast loss function between the original image and the adversarial text, and generate the corresponding adversarial text description set;

[0015] S3: Add a small perturbation to the original image to generate an adversarial image, calculate the contrast loss function between the adversarial image and the adversarial text, and then calculate the group-level mask and the pixel-level mask respectively;

[0016] S4: Iteratively update the adversarial image to generate the final adversarial image;

[0017] S5: Generate the final adversarial text based on the final adversarial image;

[0018] S6: Construct adversarial text-image pairs and evaluate the robustness of the large visual language model.

[0019] Further, in step S1, for the input original image x and original text t, the image encoder F... I and text encoder F T Encoding is performed to transform the original image x and the original text t into feature vectors, resulting in the corresponding embedding space representation F. I (x), F T (t).

[0020] Furthermore, in step S2, since one image in the original dataset corresponds to multiple text descriptions, an original text description set T = {t1, t2, ..., t...} is constructed. M Subsequently, multiple adversarial texts t' are generated based on the text adversarial generation method. i Calculate the contrast loss function between each adversarial text and the original image x, and take the one with the largest loss function as the result for subsequent calculations, expressed as:

[0021]

[0022] Among them, F I (x) represents the image encoding embedding vector, F T (t' i To counter text embedding, [t i ,ξ t ] indicates that the original text t is given i The added perturbation must be within the range of ξ, and argmax(·) represents the adversarial sample with the largest contrastive loss.

[0023] Generate the corresponding adversarial text description set T' = {t'1, t'2, ..., t' M}

[0024] Furthermore, in step S3, a small perturbation δ is added to the original image x to generate an adversarial image x', represented as:

[0025] x'=x+δ

[0026] δ will be constrained within the set disturbance range;

[0027] The contrastive loss function between the adversarial image x' and the adversarial text t' is calculated as follows:

[0028]

[0029] Where g(x') i ,s' i ) represents the scaling function, and S represents the set of scaling factors; its function is to generate adversarial examples with high transferability by maximizing the mismatch across the modal embedding space.

[0030] After calculating the loss function L, the group-level mask can be calculated, expressed as:

[0031]

[0032] Here, the adversarial example image is divided into P×Q non-overlapping image blocks (p=1,2,……P; q=1,2,……Q), with a step size of S to form local groups for perturbation analysis. The L2 norm within each group is then calculated, expressed by the formula:

[0033]

[0034] Where C represents the number of channels in the gradient;

[0035] At the same time, the loss function L is also used to calculate the pixel-level mask representation as follows:

[0036]

[0037] Furthermore, in step S4, after K iterations, the operation in step S3 is repeated in each iteration, and the adversarial image update formula is:

[0038]

[0039] Here, ⊙ represents point-by-point multiplication, and the final adversarial image is denoted as x'.

[0040] Furthermore, in step S5, the final adversarial text t' is generated based on the final adversarial image x'. f , represented as:

[0041]

[0042] Among them, F I (x') denotes adversarial image embedding, F T (t' f ) for the final adversarial text embedding, [t f ,ξ tDefined as a perturbation search space, ensuring that the generated adversarial text satisfies L. ∞ Norm constraints The operation aims to select samples from the set of texts that satisfy the perturbation constraints that maximize the cross-modal contrastive loss.

[0043] Its function is to make the final adversarial text t' f The image x' is far from the adversarial image in the embedding space.

[0044] Furthermore, in step S6, the aforementioned final adversarial text t' f The adversarial image x' and the final adversarial image form an adversarial image-text pair, which is then input into the large visual language model under test. Evaluation is performed on two downstream tasks: image-text retrieval and text-image retrieval. The image-text retrieval task involves finding a matching text description from a text library when an image is input, while the text-image retrieval task involves finding a matching image from an image library when a text is input. Robustness degradation rate (RDR) is used as the evaluation metric to assess the robustness of image-text retrieval in different scenarios. The formula for calculating RDR is as follows:

[0045]

[0046] Where N represents the number of test samples, δ i These are indicator functions used to determine degradation;

[0047] Using RDR as an evaluation metric can intuitively quantify its attack effect, thereby evaluating the robustness of the large visual language model.

[0048] This invention also provides a robustness evaluation device for large visual language models based on a two-layer guided sparse strategy, the device comprising the following modules:

[0049] The preprocessing module is used to preprocess the input raw image and raw text to obtain the embedding space representation;

[0050] The adversarial text generation module is used to construct the original text description set, generate adversarial text, calculate the contrast loss function between the original image and the adversarial text, and generate the corresponding adversarial text description set.

[0051] The mask calculation module adds a small perturbation to the original image to generate an adversarial image, calculates the contrast loss function between the adversarial image and the adversarial text, and then calculates the group-level mask and the pixel-level mask respectively.

[0052] The final adversarial image generation module is used to iteratively update the adversarial image and generate the final adversarial image;

[0053] The final adversarial text generation module is used to guide the generation of final adversarial text based on the final adversarial image;

[0054] The evaluation module constructs adversarial text-image pairs and evaluates the robustness of the large visual language model.

[0055] The beneficial effects of this invention are:

[0056] (1) Achieve precise positioning and control of sparse perturbations: Introduce a dual-layer guidance mechanism of group-level mask and pixel-level mask to screen input perturbation regions from two scales: local image region and pixel-level fine-grained scale. This makes the perturbation distribution more closely match the semantically sensitive region of the model, effectively avoids redundant perturbations, improves the representativeness and relevance of the evaluation samples, and reduces the interference of invalid perturbations on the evaluation results.

[0057] (2) Support for fine-grained robustness evaluation: Under the premise of ensuring that the overall semantics of the input image do not change significantly, diverse micro-perturbation image-text samples are generated for systematic testing of the sensitivity of the visual language model to minute semantic changes. Through this method, the performance differences of the model under fine-grained input perturbations can be accurately characterized, and its stability and robustness defects under subtle semantic changes can be comprehensively evaluated.

[0058] (3) Applicable to various evaluation environments and task scenarios: Robustness testing can be carried out in different evaluation environments such as black box and white box, supporting performance verification of the model at different stages such as development, optimization and deployment, and enhancing the scientificity and applicability of evaluation conclusions.

[0059] (4) Supports adversarial testing for bidirectional image-text retrieval tasks: The generated adversarial samples are applicable to both image-text retrieval and text-image retrieval tasks, and can comprehensively evaluate the robustness of the large visual language model under different input forms, thereby improving the coverage and completeness of model security testing. Attached Figure Description

[0060] Figure 1 This is a flowchart illustrating a robustness evaluation method for a large visual language model based on a two-layer guided sparse strategy, as provided by the present invention.

[0061] Figure 2 The diagram shows the structure of a robustness evaluation device for a large visual language model based on a two-layer guided sparse strategy, as provided by this invention. Detailed Implementation

[0062] The following is in conjunction with the appendix Figure 1 The present invention will be described in detail below. Unless otherwise specified, the features of the following embodiments and implementations can be combined with each other.

[0063] Based on the existing problems of robustness evaluation techniques for large visual language models, such as poor transferability of evaluation results between different models and poor naturalness of generated evaluation samples, this invention proposes a robustness evaluation method and device for large visual language models based on a two-layer guided sparse strategy. The main innovations of this technology are: (1) By using a two-layer masking screening mechanism, perturbation is precisely controlled at both the group level and the pixel level, concentrating the perturbation on the sensitive areas of the model, which improves the effectiveness and cross-model adaptability of the evaluation samples, and also enhances the naturalness of the evaluation samples; (2) An image-driven adversarial text optimization strategy is adopted to construct adversarial image-text pairs with cross-model transferability and structural consistency, which can carry out comprehensive robustness evaluation of multiple models, making them more suitable for evaluation needs in actual scenarios.

[0064] Terminology Explanation:

[0065] 1. Large Visual Language Model: A multimodal model that integrates image and text information. By jointly learning deep semantic representations of images and text, it demonstrates powerful capabilities in tasks such as image and text retrieval, visual question answering, and image description. However, it also faces security challenges and is susceptible to adversarial examples.

[0066] 2. Contrastive Loss: A function used to measure the degree of mismatch between adversarial images and adversarial text in the cross-modal embedding space. High-transfer adversarial examples are generated by maximizing this loss; its computation involves the outputs of the image and text encoders, as well as scaling functions, etc.

[0067] 3. Transferability: Within the scope of robustness evaluation of large visual language models, transferability refers to the consistent evaluation effect of generated evaluation samples in different models and different visual language task environments. It is used to measure the differences in robustness performance of different models when faced with input interference.

[0068] 4. Perturbation Search Space: This refers to the range of parameter values ​​constrained by perturbation parameters during the evaluation of large visual language to ensure the rationality and effectiveness of generated samples. This space limits the acceptable perturbation amplitude or perturbation pattern of the input samples, ensuring that misleading image or text samples are generated without compromising the semantic consistency and natural attributes of the input samples.

[0069] This invention provides a robustness evaluation method for large visual language models based on a two-layer guided sparse strategy, the method comprising the following steps:

[0070] S1: Extract the original image-text pairs from the dataset, preprocess the original images and text to obtain the embedding space representation;

[0071] S2: Construct the original text description set, generate adversarial text, calculate the contrast loss function between the original image and the adversarial text, and generate the corresponding adversarial text description set;

[0072] S3: Add a small perturbation to the original image to generate an adversarial image, calculate the contrast loss function between the adversarial image and the adversarial text, and then calculate the group-level mask and the pixel-level mask respectively;

[0073] S4: Iteratively update the adversarial image to generate the final adversarial image;

[0074] S5: Generate the final adversarial text based on the final adversarial image;

[0075] S6: Construct adversarial text-image pairs and evaluate the robustness of the large visual language model.

[0076] Corresponding framework such as Figure 1 As shown, the function of each part will be analyzed in detail below with reference to a specific embodiment:

[0077] S1: Obtain the original image-text pair (x,t) from the Flickr-30K dataset and input it into the large visual language model for feature extraction to construct the initial aligned representation in the cross-modal semantic embedding space for subsequent perturbation optimization and gradient analysis, aiming to clarify the semantic decision boundary of the model.

[0078] S2: Construct the original text description set T = {t1, t2, ..., t} M}, Based on Bert-Attack, generate multiple adversarial texts t' i Under the premise of freezing the original image embedding, the target semantic shift of the text is achieved by maximizing the cross-modal contrast loss between the adversarial text and the original image. In this process, a lexical-level replacement template is introduced, and combined with the context prediction capability of the large visual language model, a set of misleading adversarial descriptions is dynamically generated. The optimization objective is:

[0079]

[0080] Among them, F I (x) represents the image encoding embedding vector, F T (t' i To counter text embedding, [t i ,ξ t ] indicates that the original text t is given i The added perturbation must be within the range of ξ. This indicates that the adversarial sample with the largest contrast loss has been extracted.

[0081] The formula selects the text description least similar to the original image as the adversarial text. Since the quantitative relationship between the image and the text is one-to-many, it can form a set of adversarial text descriptions T' = {t'1, t'2, ..., t'}. M}

[0082] S3: Divide the image into P×Q image blocks, construct a group-level perturbation evaluation matrix, and generate a group-level mask by calculating the gradient norm of each image block with respect to the total loss function. This is used to filter high-response regions; then, pixel-level gradient responses are further calculated within each group to construct pixel-level masks. Combining the two yields the joint mask for the perturbation region: Where i represents the i-th iteration and ⊙ represents point-by-point multiplication, this mechanism can effectively control the perturbation range and energy concentration, thereby improving the quality of adversarial images.

[0083] S4: Calculate the contrastive loss function between the adversarial image x' and the adversarial text t', expressed as:

[0084]

[0085] Where g(x') i ,s' i ) represents the scaling function, and S represents the set of scaling factors;

[0086] Then calculate the group-level mask. and pixel-level mask The final adversarial image is generated as follows:

[0087]

[0088] in, That is, the added perturbation δ is limited by the perturbation amplitude.

[0089] In this embodiment, the number of iterations is set to K=10, the perturbation amplitude is ∈=8 / 255, and the perturbation area accounts for no more than 5% of the total number of image pixels.

[0090] S5: Based on the final adversarial image, guide the generation of final adversarial text. The goal is to further distance the text from the adversarial image in the embedding space, thereby increasing the degree of embedding mismatch. The optimization objectives of this process are as follows:

[0091]

[0092] Among them, F I (x') denotes adversarial image embedding, F T (t' f ) for the final adversarial text embedding, [t f ,ξ t Defined as a perturbation search space, ensuring that the generated adversarial text satisfies L. ∞ Norm constraints The operation aims to select samples from the set of texts that satisfy the perturbation constraints that maximize the cross-modal contrastive loss.

[0093] S6: In this embodiment, the evaluation task includes two directions: image-text retrieval and text-image retrieval, and the robust degradation rate (RDR) is used as the main evaluation indicator. Taking the image-text retrieval task as an example, under the original image x, the similarity ranking of the correct text is R. original (Scalar, the smaller the number, the higher the ranking), under the perturbation image x', the similarity ranking of the correct text is R. perturbed If R perturbed >R original Or R perturbed A threshold greater than m is considered a first-time degradation. Its formal expression is:

[0094]

[0095] N is the total number of test image samples.

[0096] Please see Figure 2 As shown, this invention provides a robustness evaluation device for large visual language models based on a two-layer guided sparse strategy. The device includes the following modules:

[0097] The preprocessing module is used to preprocess the input raw image and raw text to obtain the embedding space representation;

[0098] The adversarial text generation module is used to construct the original text description set, generate adversarial text, calculate the contrast loss function between the original image and the adversarial text, and generate the corresponding adversarial text description set.

[0099] The mask calculation module adds a small perturbation to the original image to generate an adversarial image, calculates the contrast loss function between the adversarial image and the adversarial text, and then calculates the group-level mask and the pixel-level mask respectively.

[0100] The final adversarial image generation module is used to iteratively update the adversarial image and generate the final adversarial image;

[0101] The final adversarial text generation module is used to guide the generation of final adversarial text based on the final adversarial image;

[0102] The evaluation module constructs adversarial text-image pairs and evaluates the robustness of the large visual language model.

[0103] The robustness evaluation device for large visual language models based on a two-layer guided sparse strategy of the present invention has the same operation and effect as the robustness evaluation method for large visual language models based on a two-layer guided sparse strategy described above. Therefore, the robustness evaluation device for large visual language models based on a two-layer guided sparse strategy will not be described again here.

[0104] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of a necessary general-purpose hardware platform, or by a combination of hardware and software. Based on this understanding, the above technical solutions, in essence or the part that contributes to the prior art, can be embodied in the form of a computer product. The present invention can take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0105] The above embodiments are only used to illustrate the design concept and features of the present invention, and their purpose is to enable those skilled in the art to understand the content of the present invention and implement it accordingly. The protection scope of the present invention is not limited to the above embodiments. Therefore, all equivalent changes or modifications made based on the principles and design ideas disclosed in the present invention are within the protection scope of the present invention.

Claims

1. A robustness evaluation method for large visual language models based on a two-layer guided sparse strategy, characterized in that, Includes the following steps: S1: Extract the original image-text pairs from the dataset, preprocess the original images and text to obtain the embedding space representation; S2: Construct the original text description set, generate adversarial text, calculate the contrast loss function between the original image and the adversarial text, and generate the corresponding adversarial text description set; S3: Add a small perturbation to the original image to generate an adversarial image, calculate the contrast loss function between the adversarial image and the adversarial text, and then calculate the group-level mask and the pixel-level mask respectively; S4: Iteratively update the adversarial image to generate the final adversarial image; S5: Generate the final adversarial text based on the final adversarial image; S6: Construct adversarial text-image pairs and evaluate the robustness of the large visual language model.

2. The robustness evaluation method for a large visual language model based on a two-layer guided sparse strategy according to claim 1, characterized in that, In step S1: The datasets include, but are not limited to, Flickr-30K and MSCOCO. The preprocessing includes encoding the original image and the original text using an image encoder and a text encoder, converting the original image and the original text into feature vectors, and obtaining the corresponding embedding space representation.

3. The robustness evaluation method for a large visual language model based on a two-layer guided sparse strategy according to claim 1, characterized in that, In step S2: The adversarial text is generated by applying a text adversarial generation method to the original text, and the text adversarial generation method includes, but is not limited to, Bert-Attack.

4. The robustness evaluation method for a large visual language model based on a two-layer guided sparse strategy according to claim 1, characterized in that, In step S3: The minute perturbation is constrained within a set perturbation range.

5. The robustness evaluation method for a large visual language model based on a two-layer guided sparse strategy according to claim 1, characterized in that, In step S4: In K iterations, the operation in step S3 is repeated in each iteration to update the adversarial image and generate the final adversarial image.

6. The robustness evaluation method for a large visual language model based on a two-layer guided sparse strategy according to claim 1, characterized in that, In step S5: The final adversarial text is generated based on the final adversarial image, such that the final adversarial text is far removed from the final adversarial image in the embedding space.

7. The robustness evaluation method for a large visual language model based on a two-layer guided sparse strategy according to claim 1, characterized in that, In step S6: The final adversarial text and the final adversarial image are combined to form the adversarial image-text pair. The evaluation will be carried out on two downstream tasks: image-text retrieval and text-image retrieval. The robustness degradation rate is used as the evaluation metric to evaluate the robustness of the large visual language model in different scenarios.

8. A robustness evaluation device for large visual language models based on a two-layer guided sparse strategy, characterized in that, include: A preprocessing module is used to preprocess the input original image and the original text to obtain the embedding space representation; The adversarial text generation module is used to construct an original text description set, generate the adversarial text, calculate the contrast loss function between the original image and the adversarial text, and generate a corresponding adversarial text description set. The mask calculation module is used to add a small perturbation to the original image to generate the adversarial image, calculate the contrast loss function between the adversarial image and the adversarial text, and then calculate the group-level mask and the pixel-level mask respectively. The final adversarial image generation module is used to iteratively update the adversarial image and generate the final adversarial image; The final adversarial text generation module is used to guide the generation of the final adversarial text based on the final adversarial image. The evaluation module constructs the adversarial graph-text pair and evaluates the robustness of the large visual language model.

Images