Multi-modal large language model robustness enhancement method based on antagonism self-game and antagonism opponent training framework

Abstract

The invention relates to a multi-modal large language model robustness enhancement method based on antagonism self-gaming and an antagonism opponent training framework, the method realizes capability co-evolution through iterative training based on an attacker model and a multi-modal defender model, and the method comprises the following steps: through a two-stage data generation process and multi-programmed check, the multi-modal large language model robustness is enhanced; constructing an initial antagonism data set; based on the initial antagonism data set and the clean data set, through iterative attacker-defender coevolution training, realizing coevolution of an attacker model and a defender model, and improving perception robustness of the defender model; wherein the attacker model is an attacker model with an image editing capability, and selecting a to-be-enhanced multi-modal large language model as an initial defender model. According to the method, under the condition that extra manual annotation is not needed, the perception robustness of the multi-mode large language model is continuously improved, and the illusion phenomenon is reduced.

Description

Technical Field

[0001] This invention relates to the field of multimodal large language model optimization technology, and to a method for enhancing the robustness of multimodal large language models based on adversarial self-game and an adversarial opponent training framework. Background Technology

[0002] Multimodal large language models have made significant progress on multimodal tasks by aligning powerful visual encoders with large language models. However, research shows that the perceptual foundation of these models is very fragile, especially in fine-grained spatial perception tasks, where they are easily affected by background noise. Existing solutions mainly fall into two categories: data augmentation, which expands training data through image transformation or adding noise; and dynamic evaluation, which uses generated noise to test the model, but is mainly used for evaluation rather than training. The implementation most similar to this invention is a training method based on limited adversarial datasets, such as the method proposed by Liu et al., which uses predefined visual noise (such as adding or inserting objects) to construct a dataset for fine-tuning; and methods that use fixed types of adversarial examples for training. In addition, in the text domain, there are reinforcement learning methods based on self-games, which improve capabilities through competition between models. However, in the vision-language domain, existing self-games are mostly concentrated on cooperative tasks or limited to text-level interactions, lacking adversarial generation and training frameworks that are robust to pixel-level visual perception.

[0003] Existing multimodal large language models exhibit severe perceptual vulnerability when facing complex visual scenes; that is, even minor changes in the visual scene (such as the introduction of interfering objects) often lead to inference failure. Current methods for improving model robustness mainly rely on limited, manually annotated adversarial datasets. This approach has two main problems: first, it is costly to build and difficult to scale; second, fixed datasets have limited attack patterns, making the model prone to overfitting and unable to cope with constantly changing attack methods. Existing training methods based on limited adversarial datasets have the following drawbacks: First, the data is static and limited, failing to cover diverse attack strategies. Models trained on fixed datasets are prone to overfitting, resulting in poor generalization ability and difficulty in dealing with unknown or more complex visual interference; second, there is a lack of targeted difficulty progression. Existing augmentation data is often randomly generated, lacking a mechanism to dynamically adjust the attack difficulty based on the model's current capabilities, resulting in training samples that are too simple (ineffective) or too difficult (the model cannot learn); third, there is a conflict between manual cost and scalability. High-quality adversarial examples usually require manual intervention or complex rule design, making large-scale automated expansion difficult. Summary of the Invention

[0004] This invention provides a robustness enhancement method for multimodal large language models based on adversarial self-game and an Adversarial Opponent Training (AOT) framework. This framework establishes a co-evolutionary mechanism between an image editing model (attacker) and a multimodal large language model (defender), enabling the attacker to automatically discover perceptual weaknesses in the model and generate targeted attack samples. The defender then continuously learns from these samples to improve robustness. This framework is implemented through an initial guidance phase (construction of the initial adversarial dataset) and an iterative co-evolution phase. This method eliminates the need for continuous manual annotation and can autonomously generate diverse and dynamically adaptable training courses. It aims to address the problem of limited and static existing training data. Through an automated adversarial game mechanism, i.e., adversarial reinforcement learning, it continuously improves the perceptual robustness of the multimodal large language model and reduces illusion phenomena without the need for additional manual annotation.

[0005] This invention mainly comprises two core components: First, the initial adversarial dataset (AOT-SFT dataset, AOT-SFT, Adversarial Opponent Training - Supervised Fine-Tuning) construction process. Through a two-stage pipeline including scene expansion and adversarial implantation of semantic distractors, an initial dataset for initiating adversarial training is constructed, solving the "cold start" problem of the attacker model. Second, an iterative attacker-defender co-evolutionary algorithm. By establishing a game-theoretic closed loop between the image-editing attacker model and the multimodal defender model, the attacker uses reinforcement learning to optimize strategies to generate adversarial images that can deceive the current defender while maintaining semantic consistency (allowing the attacker to automatically discover the model's perceptual weaknesses and generate targeted attack samples). The defender uses reinforcement learning to fine-tune on the dynamic course data generated by the attacker to improve recognition accuracy (while the defender continuously learns from these attack samples to improve robustness). The two train alternately and evolve together.

[0006] The first aspect of this invention provides a robustness enhancement method for a multimodal large language model based on adversarial self-game theory. This method achieves co-evolution of capabilities through iterative training of an attacker model and a multimodal defender model. The method includes the following steps: To address the cold start problem of attacker models failing to generate effective semantic interference in the initial stages, an initial adversarial dataset is constructed through a two-stage data generation process and multiple procedural checks. The two-stage data generation process includes a scenario expansion stage and an adversarial insertion stage for semantic interference. Based on the initial adversarial dataset and the clean dataset, iterative attacker-defender co-evolution training is used to achieve a spiral increase in the capabilities of the attacker and defender models, continuously realizing the co-evolution of the attacker and defender models and improving the perceptual robustness of the defender model; wherein, the attacker model is an attacker model with image editing capabilities, and the multimodal large language model to be enhanced is selected as the initial defender model.

[0007] Furthermore, it also includes: Select a source dataset and a clean dataset; the source dataset contains original images and pre-existing original questions (Visual Question Answering (VQA) type questions) corresponding to the original images, and the clean dataset is a set of original image-original question-answer pairs that are from the same source as the source dataset and have no semantic interference inserted.

[0008] Furthermore, an initial adversarial dataset is constructed through a two-stage data generation process and multiple checks, including: The scenario expansion phase includes: Images are selected from the source dataset (a visual question-answering dataset containing images and related question-answer pairs), and the image canvas is expanded using image outlining techniques to increase visual complexity and the space available for adversarial manipulation; and To ensure the quality of the expanded images and the effectiveness of the task, we designed a three-step filtering process to perform triple checks on the expanded images in order to retain valid expanded images. The adversarial implantation phase of semantic disruptors includes: For the expanded images that the initial defender model can correctly answer after the scene expansion phase, semantic interference proposals are generated based on the original question and the multimodal large language model. The semantic interference proposals include the location bounding box and text description of the object to be implanted. Double-check semantic interference proposals to ensure they do not undermine the validity of the original problem; and The double-checked semantic distractor proposals are input into a text-guided image filling model (flux.1-fill-dev-OneReward). This model uses the text descriptions included in the proposals (i.e., textual cues defining the appearance features of the distractor to be implanted, such as 'a shiny silver jar' or 'a small wooden clock') as generation conditions to generate and naturally blend corresponding distractor objects within specified location bounding boxes, thus generating candidate adversarial images. The validity verification of the candidate adversarial images based on the original problem includes: The candidate adversarial image is input along with the corresponding original question into the initial defender model to verify whether the candidate adversarial image can successfully mislead the initial defender model, i.e., to verify whether the model can correctly answer the original question; and Only valid adversarial images that can cause the model to answer the original question incorrectly are selected. That is, only valid adversarial samples that can cause the model to answer incorrectly are retained. Together with the corresponding extended image (clean image) and the original question, they form an extended image-original question-valid adversarial image triplet to form the initial adversarial dataset.

[0009] Furthermore, the triple check includes: Combination checks include using a general-purpose multimodal large language model (Qwen2.5-VL-72B) with high-order visual perception and logical reasoning capabilities to determine whether the expanded image is a coherent scene, rather than a patchwork of multiple unrelated scenes, to ensure that the answer logic of the original question is not affected; duplication checks are configured to ensure that the image rendering process does not introduce new instances of target objects related to the original question, to maintain the validity of the original question-answer pair; and Authenticity checks include filtering out expanded images that contain obvious traces of AI generation or illogical content to ensure the authenticity of the original question's answer scenario.

[0010] Furthermore, the double check includes: Location bounding box overlap checking is configured to ensure that the bounding boxes of semantic distractor proposals do not overlap with key target regions in the original image that are relevant to the original problem, thus avoiding direct occlusion or replacement of key information; and Text description checks are configured to ensure that the text descriptions of semantic interference proposals do not contain key target keywords of the original problem, preventing the generation of similar objects that alter the premise of the problem.

[0011] Furthermore, through iterative attacker-defender co-evolutionary training, including: Attacker initialization and evolution training, which includes: The initial attacker model is supervised and fine-tuned using the initial adversarial dataset to enable it to have basic image editing capabilities. Subsequently, in each iteration, a reinforcement learning strategy is used to optimize the current attacker model so that it can generate adversarial images that can deceive the previous generation fixed defender model while maintaining semantic integrity. A composite reward function is used during training, comprising a validity reward and a semantic integrity reward. The validity reward is determined based on whether the adversarial image successfully misleads the previous generation of the defender model. If the generated adversarial image successfully misleads the previous generation of the defender, causing it to produce an incorrect answer, a high reward is given. The semantic integrity reward uses an image semantic consistency metric to ensure that the image similarity metric of the key object region before and after editing is higher than a preset similarity threshold. An attack is considered valid only when the image region corresponding to the key object in the problem has an image similarity metric (e.g., structural similarity index SSIM) higher than a preset similarity threshold (e.g., SSIM value greater than 0.9), thus preventing attackers from winning by deleting or severely damaging key objects. Through evolutionary training, the attackers evolve into more powerful next-generation attackers; Defender training and enhancement, which includes: New adversarial images are generated on a clean dataset using the attacker model that evolves after each round of training, thus forming a new adversarial dataset. This new adversarial dataset is used to fine-tune the current multimodal defender model, enabling it to learn to defend against these novel attacks. Specifically, the current multimodal defender model is fine-tuned using a direct policy optimization algorithm, with its reward function determined based on the model's accuracy in responding to the new adversarial samples; that is, the reward function is based on whether the model's responses to the new adversarial samples are correct. Through training, the defenders evolved into a more robust next generation of defenders; and Loop iteration, which includes: Repeated attacker evolution training and defender training and enhancement are used to achieve co-evolution of the attacker and defender models, continuously improving the perceptual robustness of the defender model.

[0012] Furthermore, the image semantic consistency measurement tool includes an image similarity metric / loss function for measuring the semantic integrity of an image; the image semantic consistency measurement tool includes a localized structural similarity index (SSIM), a learned perceptual image patch similarity (LPIPS), and a learned perceptual loss function.

[0013] Furthermore, the reinforcement learning policy optimization algorithm is a diffusion reinforcement learning algorithm, including either the Flow-GRPO algorithm or danceGRPO; the direct policy optimization algorithm includes either the DAPO algorithm or GSPO; and / or The attacker model includes the Qwen-Image-Edit series model or an image editing model based on Stable Diffusion with fine-tuning instructions; the defender model includes the QwenVL series model, the InternVL series model, or other multimodal large language models with vision-language alignment capabilities.

[0014] Furthermore, through iterative attacker-defender co-evolutionary training, it also includes: After the new generation of attackers generates a batch of adversarial images, a course selection mechanism (i.e., a difficulty assessment mechanism based on random sampling) is used to select training samples of moderate difficulty and the most informative content for the defender model from the adversarial images generated by the attackers to form a new adversarial dataset. The course selection mechanism is specifically configured as follows: each candidate adversarial image is input into the previous generation defender model for multiple random inferences (e.g., 10 random samplings with a temperature coefficient of 1.0), and the number of times the defender answers correctly is counted; only images with the number of correct answers within a preset difficulty window (e.g., 3 to 7 correct answers out of 10, i.e., an accuracy rate between 30% and 70%) are retained as effective training samples; among them, samples with an accuracy rate higher than the upper limit of the preset window (e.g., 70%) are judged as too easy (i.e., not challenging for the defender), samples with an accuracy rate lower than the lower limit of the preset window (e.g., 30%) are judged as too difficult (i.e., exceeding the current learning ability), and the samples retained in the middle range are the training samples with moderate difficulty and the most informative content that are located near the model's decision boundary and can effectively detect the model's perceived weaknesses.

[0015] Furthermore, it also includes: The iteration terminates and the performance is verified to determine whether the training has achieved the expected results, and the final robust multimodal large language model is output.

[0016] This aspect also provides a framework for improving the perceptual robustness of a multimodal large language model through adversarial self-game. The framework includes an attacker model and a multimodal defender model, both of which achieve co-evolution of capabilities through iterative training. The attacker model is an attacker model with image editing capabilities.

[0017] This aspect also provides a method for generating an initial adversarial dataset for initiating adversarial self-games, the method comprising: The scenario expansion phase includes: Images are selected from the source dataset, and image rendering techniques are used to expand the image canvas, increasing visual complexity and the space available for adversarial manipulation; and Perform a triple check on the expanded image to retain the valid expanded image; The adversarial implantation phase of semantic disruptors includes: For the filtered expanded images, semantic distractor proposals are generated based on the original question and the multimodal large language model. The semantic distractor proposals include the location bounding box and text description of the object to be implanted. Double-check semantic interference proposals to ensure they do not undermine the validity of the original problem; and Double-checked semantic distractor proposals are input into a text-guided image infilling model to generate and naturally blend corresponding distractor objects within specified bounding box regions, thereby generating candidate adversarial images; and The validity verification of the candidate adversarial images based on the original problem includes: The candidate adversarial image is input along with the corresponding original question into the initial defender model to verify whether the candidate adversarial image can successfully mislead the initial defender model, i.e., to verify whether the model can correctly answer the original question; and Only valid adversarial images that can cause the model to give an incorrect answer to the original question are selected; that is, only valid adversarial examples that can cause the model to give an incorrect answer are retained. These, along with their corresponding extended images and the original question, form an extended image-original question-valid adversarial image triplet, forming the initial adversarial dataset; and / or The triple check includes: The combination check includes using a general multimodal large language model to determine whether the expanded image is a coherent scene rather than a splicing of multiple unrelated scenes, so as to ensure that the answer logic of the original question is not affected. Repeat checks, configured to ensure that the image extrapolation process does not introduce new instances of the target object relevant to the original question, in order to maintain the validity of the original question-answer pair; and Authenticity checks include filtering out expanded images containing obvious AI-generated traces or illogical content to ensure the realism of the original question's answer scenario; and / or The dual check includes: Location bounding box overlap checking is configured to ensure that the bounding boxes of semantic distractor proposals do not overlap with key target regions in the original image that are relevant to the original problem, thus avoiding direct occlusion or replacement of key information; and Text description checks are configured to ensure that the text descriptions of semantic distractor proposals do not contain key target keywords of the original problem, preventing the generation of similar objects that alter the premise of the problem. Both the scenario expansion phase and the adversarial implantation phase of semantic distractors include multiple procedural checks to ensure data quality and adversarial effectiveness; these multiple procedural checks include the triple check and the double check.

[0018] This aspect also provides a training method for an attacker model, using a reinforcement learning-based strategy optimization algorithm and employing a composite reward function to simultaneously optimize the effectiveness of adversarial attacks and the preservation of scene semantic integrity. The composite reward function includes an effectiveness reward and a semantic integrity reward. The effectiveness reward is determined based on whether the adversarial image successfully misleads the previous generation defender model. The semantic integrity reward uses an image semantic consistency metric to ensure that the key object region is highly similar before and after editing. An attack is considered effective only when the image region corresponding to the problematic key object remains highly similar before and after editing, thus preventing attackers from winning by deleting or severely damaging key objects; and / or Furthermore, the image semantic consistency measurement tool includes an image similarity metric / loss function for measuring the semantic integrity of an image; the image semantic consistency measurement tool includes SSIM, LPIPS, and a learned perceptual loss function.

[0019] This aspect also provides a dynamic training course generation mechanism, in which each generation of evolved attackers generates new and more challenging adversarial images for the next generation of defenders. The generation process includes a course selection mechanism to select the training samples that are most informative for the defenders, forming a dynamic training course. The course selection mechanism is specifically configured as follows: the generated candidate adversarial images are input into the previous generation defender model for multiple random inferences, the number of times the defender answers correctly is counted, and only images with the number of correct answers within a preset difficulty window (e.g., accuracy between 30% and 70%) are selected as effective training samples that are located near the model decision boundary and have moderate difficulty.

[0020] The present invention has at least the following beneficial effects: 1) The present invention realizes a scalable and adaptive training course. Existing technologies rely on static and limited adversarial datasets, whose difficulty and diversity are fixed. The self-game framework of the present invention can autonomously generate training data. As the defender progresses, the attacker will discover new and more complex attack strategies, thereby forming a dynamic course with automatically adaptable difficulty. This breaks through the "capability ceiling" imposed by static datasets, continuously improving the robustness of the model. After three rounds of iteration, the defender model of this invention outperformed the baseline model by 9.24 and 8.26 percentage points on the VStar and HRBench-4K perceptual robustness benchmarks, respectively, and significantly outperformed the baseline method trained using static "hard" datasets; 2) This invention improves the perceptual robustness and factuality of the model, effectively reducing illusions. By forcing the model to perform fine-grained perception and reasoning in complex environments full of semantic interference, it directly enhances its core perceptual capabilities. This not only improves the accuracy of the model in adversarial scenarios, but also reduces the illusion of object existence because the model can better answer based on visual evidence. While improving robustness, the method of this invention improves the POPE F1 score by 2.88 percentage points and HallusionBench score by 2.88 percentage points. aAcc improved by 1.68 percentage points, outperforming all baseline methods; 3) The adversarial courses generated by this invention have high transferability. The new adversarial training dataset generated by the framework of this invention is not only effective for the defender model itself, but also effectively improves the robustness of other multimodal models of different architectures and sizes. This means that this invention provides a general and transferable robustness enhancement scheme, rather than an "overfitting" solution for a specific model. Attached Figure Description

[0021] To further illustrate the above and other advantages and features of the various embodiments of the present invention, a more specific description of the various embodiments of the invention will be presented with reference to the accompanying drawings. It is to be understood that these drawings depict only typical embodiments of the invention and are therefore not intended to limit its scope. In the drawings, identical or corresponding parts will be indicated by identical or similar reference numerals for clarity.

[0022] Figure 1 This diagram illustrates the iterative process of a) attacker evolution and b) defender enhancement in this invention. Figure 2 This paper presents a comparison between the defender model in this invention and the standard multimodal large model. Figure 3 The flowchart of the two-stage process (scene expansion stage and adversarial implantation stage of semantic interfering objects) for creating the initial adversarial dataset AOT-SFT in this invention is shown. Figure 4An overview diagram of the attacker-defender iterative co-evolutionary framework in this invention is shown; Figure 5 This paper presents a qualitative example of the diverse attack strategies autonomously discovered by the attacker model in this invention. Detailed Implementation

[0023] It should be noted that the components in the accompanying drawings may be shown exaggerated for illustrative purposes and may not be to scale.

[0024] In this invention, the various embodiments are merely intended to illustrate the solutions of the invention and should not be construed as limiting.

[0025] In this invention, unless otherwise specified, the quantifiers “a” and “one” do not exclude scenarios involving multiple elements.

[0026] It should also be noted that, in the embodiments of the present invention, only a portion of the parts or components may be shown for clarity and simplicity. However, those skilled in the art will understand that, under the teachings of the present invention, the required parts or components can be added as needed for specific scenarios.

[0027] Furthermore, the embodiments of the present invention describe the process steps in a specific order. However, this is only for the convenience of distinguishing each step, and is not a limitation on the order of each step. In different embodiments of the present invention, the order of each step can be adjusted according to the process.

[0028] The following embodiments provide a framework for improving the perceptual robustness of multimodal large language models through adversarial self-game, the overall concept of which is as follows: Figure 1 As shown, Figure 1 This invention provides a conceptual explanation of the framework, using cartoon illustrations to depict the iterative process of the dual-model architecture from "cold start" to "capability advancement." It visualizes the core logic of the AOT framework, demonstrating the core idea of ​​attackers and defenders achieving co-evolution of capabilities through iterative competition. Figure 1 In this framework, Cold Start represents the initial attacker; Diffusion-RL (Diffusion Model-based Reinforcement Learning) is a reinforcement learning algorithm based on a diffusion model; Born Defender represents the initial defender; and MLLM-RL (Multimodal Large Language Model Reinforcement Learning) is a hybrid algorithm combining a multimodal large language model and reinforcement learning. The framework includes an attacker model and a multimodal defender model, both of which co-evolve through iterative training. The attacker model is an attacker model with image editing capabilities. This framework is implemented through an initial bootstrapping phase and an iterative co-evolution phase.

[0029] (1) Initial Adversarial Dataset Construction: To address the cold start problem of image editing attacker models failing to generate effective semantic perturbations in the initial stages, this embodiment designs a two-stage data generation process to create the initial supervised fine-tuning dataset AOT-SFT. The specific process is as follows: Figure 3 As shown. Figure 3 The left side of the diagram represents the "scene expansion stage to increase visual complexity," which expands the original image through image extrapolation and undergoes triple checks for combination, repetition, and realism. The right side represents the adversarial implantation stage for semantic distractors. Distractor proposals are generated from the checked image, and after bounding box and description checks, the image-filling model implants the distractors. Ultimately, only adversarial examples that successfully mislead the model are retained. The original input in the diagram is an image from the VStar spatial relation dataset (containing a bottle and a mobile phone); bbox1 is a small wooden clock (corresponding to the red box area in the image), and bbox2 is a shiny silver jar (corresponding to the red box area in the image). The bboxes represent the implanted distractors. As can be seen, after implanting the "small wooden clock," the standard Multimodal Large Language Model (MLLM) still answers correctly, indicating that the small wooden clock is an invalid distractor. After implanting the "shiny silver jar," the standard MLLM answers incorrectly, indicating that the shiny silver jar is a valid distractor. The "original image" refers to the image that has been expanded and checked in the scene expansion stage.

[0030] Phase 1 (Scene Expansion Phase): Scene expansion increases visual complexity. First, we select images from the source dataset and expand the image canvas using image rendering techniques, increasing scene complexity and the space available for adversarial manipulation. To ensure the quality of the expanded images and the effectiveness of the task, we designed a three-step filtering process: Combination check: Use a powerful multimodal large language model to determine whether the expanded image is a coherent scene, rather than a patchwork of multiple unrelated scenes.

[0031] Repeat check: Ensure that the external drawing process does not introduce new instances of the target object related to the original question, in order to maintain the validity of the original question-answer pair.

[0032] Authenticity check: Filter out images containing obvious AI-generated traces or illogical content to ensure the realism of the scene.

[0033] Phase Two (Adversarial Implantation of Semantic Disruptors): Adversarial implantation of semantic disruptors. For clean images that passed the Phase One screening and were correctly answered by the initial defender model, we utilize a multimodal large language model to generate proposals for semantic disruptors. Each proposal contains a bounding box of the object to be implanted and a textual description. These proposals undergo two integrity checks: Bounding box overlap check: Ensure that the bounding box of the proposal does not overlap with the critical target area related to the problem in the image, and avoid directly occluding or replacing key information.

[0034] Description check: Ensure that the text description of the proposal does not contain keywords related to key objectives, preventing the generation of similar objects that would change the premise of the problem.

[0035] Proposals that pass the inspection are fed into an image imputation model, which inserts distracting objects within specified bounding boxes based on the description, generating candidate adversarial images. Finally, we verify whether these adversarial images can successfully mislead the initial defender model. Only valid adversarial examples that cause the model to answer incorrectly are retained and, together with their corresponding clean images and questions, form a triple in the AOT-SFT dataset.

[0036] (2) Iterative Attacker-Defender Co-evolution: This stage is the core of this invention, and its process is as follows: Figure 4 As shown, after obtaining the initial AOT-SFT dataset for preliminary training on the attacker, the system enters an iterative adversarial loop. Figure 4 This diagram illustrates an overview of the iterative framework for attacker-defender co-evolution in this invention. The framework primarily comprises two training phases interconnected through reinforcement learning. Figure 4 As seen in the upper part, the attacker training phase, employs a diffusion reinforcement learning algorithm to optimize the current attacker N-1 model (flame icon indicating training), enabling it to generate adversarial examples that can deceive the previous generation's frozen defender N-1 model (snowflake icon indicating parameter freezing). The attacker edits and inspects images, generating images containing specific types of edits (such as changes in the color of locker handles). If the generated image successfully misleads the defender, the attacker receives a effectiveness reward. In the lower part, the defender enhancement phase, new adversarial examples are generated using the previously evolved attacker N (now frozen), and a reinforcement learning algorithm is used to fine-tune the current defender N model (now in training). The defender receives an accuracy reward based on the correctness of its responses on these examples. The entire process iterates, driving a spiral increase in the capabilities of both models.

[0037] Attacker Evolution: In each iteration, the goal of the currently active attacker model is to learn to generate adversarial image editing techniques that can fool the previous generation's fixed defender model. The attacker's training (the training method for the image editing attacker model) is based on a reinforcement learning policy optimization algorithm (Flow-GRPO algorithm). Its reward function is designed as a composite reward function, consisting of two parts: Effectiveness reward: A high reward is given if the generated adversarial image successfully misleads the previous generation of defenders, causing them to produce incorrect answers.

[0038] Semantic Integrity Reward: To ensure that attacks do not damage core objects related to the problem, we introduce a localized structural similarity index check. An attack is considered valid only if the image region corresponding to the key object of the problem remains highly similar before and after editing. This prevents attackers from winning through "trivial" methods such as deleting or severely damaging key objects.

[0039] Through training, the attackers evolved into a more powerful next-generation attacker.

[0040] Defender Enhancement: A new generation of more powerful attackers is used to generate a large number of more challenging adversarial examples on a clean dataset. These newly generated examples form a new training set. We then use this new adversarial dataset to fine-tune the current defender model, enabling it to learn to defend against these new attacks. The defender is trained based on the Direct Policy Optimization (DAPO) algorithm, whose reward function is based on whether the model's responses on the new adversarial examples are correct. After training, the defender evolves into a more robust next-generation defender.

[0041] This cycle of "attacker evolution" and "defender enhancement" repeats continuously, causing the capabilities of both sides to spiral upwards, thus providing the defender with a dynamic and adaptive training course that continuously forges its perceptual robustness.

[0042] Figure 2 To demonstrate the actual effectiveness of the framework, an example is used to compare the standard multimodal large model and the defender model trained by the framework of this invention. Through the "scenario comparison before and after adversarial attack", the effect of "the defender of this invention can correctly answer the original question, while the ordinary model will be misled" is shown. Figure 2 middle: Scenario and Problem Background Original image: There is a bottle (marked in green) and a cell phone (marked in blue) on the bedside table; Original question: Describe the position of the phone relative to the bottle; The real answer is: The phone is on the left side of the bottle.

[0043] Performance of the standard model vs. the defender of this invention in a simple scenario The inference process of a typical multimodal large language model (MLLM): 1. Identify the bottle: The bottle is on the bedside table; 2. Identify the phone: The phone is also on the bedside table, located to the left of the bottle; 3. Conclusion: The phone is on the left side of the bottle (reasoning correct).

[0044] The reasoning process of the Defender model in this embodiment: 1. Observe the bottle: This bottle is a small, upright container with a light green hue; 2. Observe the phone and its position: The phone is a rectangular, dark-colored device placed on the left side of the bottle; 3. Conclusion: The phone is on the left side of the bottle (reasoning correct).

[0045] Performance of the standard model vs. the present invention's defender after introducing interfering factors. The reasoning process of a typical multimodal large language model (MLLM) after introducing contextual distractors (jars): 1. Identify the bottle: The bottle is located on the bedside table between the bed and the window; 2. Phone recognition: The phone was placed closer to the bed; 3. Determining the location: The bottle is on the left side of the bedside table, and the phone is on the right side of the bedside table (incorrect reasoning).

[0046] The reasoning process of the defender model in this embodiment: 1. Identify the bottle and its context: This bottle is a jar on the bedside table, next to a cat; 2. Phone recognition: The phone is clearly visible on the same bedside table; 3. Reasoning: Since the bottle is on the right side of the bedside table, we can deduce that the phone is on the left side of the bottle (correct reasoning).

[0047] Figure 2 A comparison was made between a standard multimodal large model and a defender model trained using the framework of this invention, using an example. In simple scenarios, both could answer correctly. However, when a contextual distractor (the jar) was introduced, the standard multimodal large model was misled and produced incorrect reasoning, while the defender model of this invention, with its strong perceptual robustness, was able to eliminate the distraction and make correct reasoning.

[0048] Figure 4This diagram presents an overview of the attacker-defender iterative co-evolution framework. In the "Attacker Evolution" loop, the activated attacker is trained using Flow-GRPO with the goal of deceiving a fixed previous-generation defender; successful deception yields an "effectiveness reward." In the "Defender Enhancement" loop, newly trained attackers are used to generate challenging samples to train the activated defender; their performance yields an "accuracy reward," trained using DAPO. This loop repeats continuously, progressively improving the capabilities of both attackers and defenders. Figure 4 In this context, Diffusion Reinforcement Learning is the core algorithm for attacker training, while Reinforcement Learning is the core algorithm for defender training.

[0049] Figure 5 This illustrates a qualitative example of diverse attack strategies autonomously discovered by the attacker model. Figure 5 The image shows the performance of the same scene under different attack strategies. Each group includes the original image, the edited image of the attacked image, and a difference map highlighting the modified areas. Figure 5 The attack types listed include: imperceptible attacks, i.e., pixel-level perturbations that are almost imperceptible to the visual sense; perceptible replacement attacks, such as replacing a blue object with a red one; perceptible removal attacks, i.e., erasing a specific object from the scene; perceptible addition attacks, i.e., implanting a new object out of thin air into the scene; and hybrid attacks, i.e., complex attack strategies that combine replacement and addition techniques. These examples demonstrate that attacker models can go beyond initial simple instructions and evolve into rich and complex visual manipulation capabilities. Figure 5 It demonstrates a variety of attack types that attackers learn during co-evolution, in addition to the only "object addition" attack in the initial SFT dataset. These include subtle pixel-level perturbations, object replacement, object removal, and hybrid attacks combining multiple operations. The modified areas are indicated with difference icons.

[0050] This invention has been extensively experimentally verified, demonstrating its feasibility and effectiveness. The experiments were conducted based on the Qwen2.5-VL-7B (as defender) and Qwen-Image-Edit (as attacker) models, and evaluated on multiple public benchmarks.

[0051] (1) Perceptual robustness verification: On the high-resolution perceptual robustness benchmarks VStar and HRBES-4K, the AOT framework of this invention demonstrates significant performance improvements. As shown in Table 1, after three rounds of co-evolutionary iterations, the accuracy of the defender model improved from 71.01% to 80.25% on VStar and from 64.12% to 72.38% on HRBES-4K. Each iteration brought stable performance growth, proving the effectiveness of the co-evolutionary dynamics. Compared with strong baseline methods that rely on static adversarial datasets, the final model of this invention outperforms by 4.20 percentage points on VStar and 5.26 percentage points on HRBES-4K, demonstrating the superiority of the dynamic adaptive curriculum.

[0052] Table 1 Comparative experimental results of robustness and hallucination control ability of multimodal large language models Table 1 compares the co-evolutionary framework of this invention (shown in three iterations) with baseline models, ablation experiments, and various baseline methods based on static data augmentation. The best performance for each metric is highlighted in bold, and the gain relative to the baseline model (△, in red in the table) is calculated. In Table 1, Base(Mdef(0)) is the initial baseline model (the original multimodal model without any robustness enhancement); Ablation (Clean Data) is the ablation experiment: the model is trained using only "clean datasets without interference" (verifying the effect of "clean data training"); Training withfinite Augmentation Datasets is the existing method: the model is trained using "limited static augmentation datasets" (traditional data augmentation method); Our Method (Iterative Co-evolution) is the method of this invention: based on the AOT framework of "iterative attacker-defender co-evolution"; Defender Iter.1 is the performance of the defender model after 1 iteration; Defender Iter.2 is the performance of the defender model after 2 iterations; Defender Iter.3 is the performance of the defender model after 3 iterations. Yang et al.

[48] Corresponding references: Yue Yang, Shuibai Zhang, WenqiShao, Kaipeng Zhang, Yi Bin, Yu Wang, and Ping Luo. Dynamic multimodalevaluation with flexible complexity by vision-language bootstrapping. arXivpreprint arXiv:2410.08695, 2024; Liu et al.

[25] Corresponding references: Ming Liu, HaoChen, Jindong Wang, and Wensheng Zhang. On the robustness of multimodallanguage model towards distractions. arXiv preprint arXiv:2502.09818, 2025.

[0053] (2) Validation of the influence of model hallucination: The experiment evaluated the model's factuality on two mainstream hallucination benchmarks: POPE and HallusionBench. The results, shown in Table 1, demonstrate that the proposed method significantly reduces hallucinations while improving perceptual robustness. The final defender model, compared to the baseline model, improved the POPE F1 score by 2.88 points and the HallusionBench aAcc score by 1.68 points, also outperforming all data augmentation baseline methods. This indicates that enhancing core perceptual capabilities helps the model respond better based on visual evidence.

[0054] (3) Emergent attack strategy analysis: Qualitative analysis such as Figure 5 As shown, this demonstrates that the attacker model possesses the ability to autonomously discover diverse attack strategies. Although the initial AOT-SFT dataset only contained one attack type, "adding objects," during the co-evolution process, the attacker spontaneously learned more complex strategies, including object replacement, object removal, and hybrid attacks combining multiple techniques. This proves that the framework can create a rich and varied adversarial environment.

[0055] (4) Course selection strategy ablation experiment: To verify the rationality of the course selection mechanism design in this invention, we conducted ablation experiments. As shown in Table 2, the experimental results demonstrate that a course with balanced difficulty and semantic integrity is crucial. Training only on the most difficult adversarial examples severely degrades model performance; for instance, accuracy on VStar drops by more than 4 percentage points, proving that overly difficult courses are harmful. Furthermore, removing the SSIM check used to ensure semantic integrity leads to a significant decrease in model performance on the HRBench-4K benchmark, highlighting the necessity of maintaining the integrity of core scene information when generating adversarial examples. These results collectively validate that the course construction strategy proposed in this invention, which includes balanced difficulty selection and integrity checks, is optimal.

[0056] Table 2 Ablation Study Results of Course Selection Strategies Table 2 illustrates the importance of each component of the course selection mechanism in this invention. It compares the performance of the complete method of this invention (using a balanced course with integrity checks) with several variants, including: using all adversarial examples, using only the most difficult examples, using only the simplest examples, and a variant that removes the semantic integrity check. The results show that the complete method performs best, and any variant leads to a performance degradation, thus validating the effectiveness of the balanced difficulty and semantic integrity check design in this invention. In Table 2, under Our Method (Balanced w / Integrity Check), Balanced: The course selection mechanism filters samples with "balanced difficulty" (removing samples that are too easy / too difficult); Integrity Check: "Semantic integrity check (such as SSIM)" is added to ensure that adversarial examples do not violate the core semantics; Ablations on DifficultyRange is an ablation experiment targeting the "course difficulty range" (verifying the necessity of the course selection mechanism); All AdversarialExamples: The course selection mechanism is removed, and all adversarial examples generated by attackers are used for training; Hardest ExamplesOnly: Only the most difficult adversarial examples are used for training; Easiest Examples Only: Only the easiest adversarial examples are used for training; Ablation on Data Integrity is an ablation experiment targeting "data integrity check" (verifying the necessity of semantic integrity check); w / o Integrity Check (SSIM): "Semantic integrity check (SSIM)" is removed, allowing attackers to generate adversarial examples that violate the core semantics.

[0057] (5) Validation of the generalization and transferability of adversarial courses: To verify the generality of the adversarial curriculum generated by this invention, we applied it to train a series of external models with different architectures and sizes. As shown in Table 3, the curriculum exhibits strong transferability. Applying the curriculum generated by Qwen2.5-VL 7B to other models, such as the Qwen3-VL series and the Gemma-3-IT series, yielded significant and consistent performance improvements. For example, the Qwen3-VL (8B) model achieved a 4.62 percentage point improvement in accuracy on VStar compared to the baseline fine-tuned with only clean data; while the Gemma-3-IT (27B) model achieved a 3.13 percentage point improvement in accuracy on HRBench-4K. The experimental results confirm the strong intra-family and cross-family generalization ability of this curriculum, demonstrating that this invention provides a widely applicable and general robust enhancement scheme.

[0058] Table 3. Validation results of the generalization ability of the iterative adversarial course. Table 3 aims to demonstrate the transferability of the adversarial training data generated by the framework of this invention. It shows the performance after applying the course to several external models of different architectures and sizes (including the Qwen3-VL and Gemma-3-IT series). The table compares the base performance of the models, the performance after training with clean data, and the performance after training with the adversarial course for different iterations, and calculates the gain (Δ) relative to the clean data baseline. The results show that the course can consistently improve the performance of different model families and sizes, confirming the strong generalization ability of the training data generated by this invention. In Table 3, Qwen3-VL(4B) / Qwen3-VL(8B) are Qwen3-VL models of different scales (4B / 8B refers to the number of model parameters, in "billions"), which belong to the multimodal large language model developed by Alibaba Qianwen; Gemma-3-IT(4B / 12B / 27B) are Gemma-3-IT models of different scales (multimodal large language model developed by Google), where IT stands for "Instruction-Tuned"; BaseModel is the "original baseline" of the model (the initial version without any training); +CleanData(Baseline) is the model trained only with "clean dataset without interference" (as a baseline for course training); +Curriculum-Iter1 / 2 / 3 is the model trained using the "dynamic training course" of this invention, where Iter1 / 2 / 3 represents the "round of course iteration" (corresponding to the attacker-defender iteration in the previous table).

[0059] In some embodiments, the Qwen series model used in this embodiment can be replaced by any other multimodal large language model (such as the InternVL series) and image editing model (such as a model based on Stable Diffusion and fine-tuned by instructions) with corresponding capabilities.

[0060] In some embodiments, this embodiment starts with the VStar spatial relation dataset, but in principle, it can start with any visual question-answering dataset that contains images and related question-answer pairs to guide different types of perceptual robustness training.

[0061] In some embodiments, the SSIM used in the attacker's semantic integrity reward in this embodiment can be replaced by other image similarity metrics (such as LPIPS) or a learned perceptual loss function to constrain the rationality of the editing from different perspectives.

[0062] In some embodiments, the attacker's Flow-GRPO algorithm in this embodiment can be replaced by other diffusion reinforcement learning algorithms (such as danceGRPO); the defender's DAPO algorithm can also be replaced by other algorithms (such as GSPO).

[0063] While some embodiments of the present invention have been described in this application, those skilled in the art will understand that these embodiments are merely illustrative. Numerous variations, alternatives, and improvements will arise in those skilled in the art under the teachings of this invention without departing from its scope. The appended claims are intended to define the scope of the invention and thereby cover methods and structures within the scope of the claims themselves and their equivalents.

Claims

1. A robustness enhancement method for multimodal large language models based on adversarial self-game theory, characterized in that, This method achieves co-evolution of capabilities through iterative training of an attacker model and a multimodal defender model. The method includes the following steps: An initial adversarial dataset is constructed through a two-stage data generation process and multiple procedural checks; the two-stage data generation process includes a scene expansion stage and an adversarial insertion stage of semantic interference. as well as Based on the initial adversarial dataset and the clean dataset, iterative attacker-defender co-evolution training is used to achieve co-evolution of the attacker model and the defender model, thereby improving the perceptual robustness of the defender model. The attacker model is an attacker model with image editing capabilities, and the multimodal large language model to be enhanced is selected as the initial defender model.

2. The robustness enhancement method for multimodal large language models based on adversarial self-game as described in claim 1, characterized in that, Also includes: Select the source dataset and the clean dataset; The source dataset contains original images and pre-existing original questions corresponding to the original images. The clean dataset is a set of original image-original question-answer pairs that are from the same source dataset and have no semantic interference.

3. The robustness enhancement method for multimodal large language models based on adversarial self-game as described in claim 2, characterized in that, An initial adversarial dataset is constructed through a two-stage data generation process and multiple checks, including: The scenario expansion phase includes: Images are selected from the source dataset, and the image canvas is expanded using image outlining techniques to increase visual complexity and the space available for adversarial manipulation; and Perform a triple check on the expanded image to retain the valid expanded image; The adversarial implantation phase of semantic disruptors includes: For the filtered expanded images, semantic distractor proposals are generated based on the original question and the multimodal large language model. The semantic distractor proposals include the location bounding box and text description of the object to be implanted. Double-check semantic interference proposals to ensure they do not undermine the validity of the original problem; and Double-checked semantic distractor proposals are input into a text-guided image infilling model to generate and naturally blend corresponding distractor objects within specified bounding box regions, thereby generating candidate adversarial images; and The validity verification of the candidate adversarial images based on the original problem includes: The candidate adversarial image is input along with the corresponding original question into the initial defender model to verify whether the candidate adversarial image can successfully mislead the initial defender model; and Only effective adversarial images that can cause the model to answer the original question incorrectly are selected, and together with their corresponding extended images and the original question, they form an extended image-original question-effective adversarial image triplet to form the initial adversarial dataset.

4. The robustness enhancement method for multimodal large language models based on adversarial self-game as described in claim 3, characterized in that, The triple check includes: The combination check includes using a general multimodal large language model to determine whether the expanded image is a coherent scene rather than a splicing of multiple unrelated scenes, so as to ensure that the answer logic of the original question is not affected. Repeat checks, configured to ensure that the image extrapolation process does not introduce new instances of the target object relevant to the original question, in order to maintain the validity of the original question-answer pair; and Authenticity checks include filtering out expanded images that contain obvious traces of AI generation or illogical content to ensure the authenticity of the original question's answer scenario.

5. The robustness enhancement method for multimodal large language models based on adversarial self-game as described in claim 3, characterized in that, The dual check includes: Location bounding box overlap checking is configured to ensure that the bounding boxes of semantic distractor proposals do not overlap with key target regions in the original image that are relevant to the original problem, thus avoiding direct occlusion or replacement of key information; and Text description checks are configured to ensure that the text descriptions of semantic interference proposals do not contain key target keywords of the original problem, preventing the generation of similar objects that alter the premise of the problem.

6. The robustness enhancement method for multimodal large language models based on adversarial self-game as described in claim 1, characterized in that, Through iterative attacker-defender co-evolutionary training, including: Attacker initialization and evolution training, which includes: The initial attacker model is supervised and fine-tuned using the initial adversarial dataset to enable it to have basic image editing capabilities. Subsequently, in each iteration, a reinforcement learning strategy is used to optimize the current attacker model so that it can generate adversarial images that can deceive the previous generation fixed defender model while maintaining semantic integrity. A composite reward function is used during training, comprising a validity reward and a semantic integrity reward. The validity reward is determined based on whether the adversarial image successfully misleads the previous generation defender model. The semantic integrity reward uses an image semantic consistency metric to ensure that the image similarity metric before and after editing the key object region is higher than a preset similarity threshold. An attack is considered valid only when the image region corresponding to the key object remains highly similar before and after editing, thus preventing attackers from winning by deleting or severely damaging the key object. Through evolutionary training, the attackers evolve into more powerful next-generation attackers; Defender training and enhancement, which includes: New adversarial images are generated on a clean dataset using the attacker model that evolves after each round of training, thus forming a new adversarial dataset. This new adversarial dataset is used to fine-tune the current multimodal defender model, enabling it to learn to defend against these novel attacks. Specifically, the current multimodal defender model is fine-tuned using a direct policy optimization algorithm, with its reward function determined based on the model's response accuracy on the new adversarial samples. Through training, the defenders evolved into a more robust next generation of defenders; and Loop iteration, which includes: Repeated attacker evolution training and defender training and enhancement are used to achieve co-evolution of the attacker and defender models, continuously improving the perceptual robustness of the defender model.

7. The robustness enhancement method for multimodal large language models based on adversarial self-game as described in claim 6, characterized in that, The reinforcement learning policy optimization algorithm is a diffusion reinforcement learning algorithm, including either the Flow-GRPO algorithm or the danceGRPO algorithm; the direct policy optimization algorithm includes either the DAPO algorithm or the GSPO algorithm; and / or The attacker model includes the Qwen-Image-Edit series model or an image editing model based on Stable Diffusion with fine-tuning instructions; the defender model includes the QwenVL series model, the InternVL series model, or other multimodal large language models with vision-language alignment capabilities.

8. The robustness enhancement method for multimodal large language models based on adversarial self-game as described in claim 6, characterized in that, Iterative attacker-defender co-evolutionary training also includes: After the new generation of attackers generates a batch of adversarial images, a course selection mechanism is used to select training samples of moderate difficulty and the most informative content for the defender model from the adversarial images generated by the attackers to form a new adversarial dataset. The course selection mechanism is configured as follows: each candidate adversarial image is input into the previous generation defender model for multiple random inferences, and the number of times the defender answers correctly is counted; only images with the number of correct answers within a preset difficulty window are retained as valid training samples; among them, samples with an accuracy rate higher than the upper limit of the preset window are judged as too easy, samples with an accuracy rate lower than the lower limit of the preset window are judged as too difficult, and the retained samples in the middle range are the training samples of moderate difficulty and the most informative content that are located near the model decision boundary and can effectively detect the model's perceived weaknesses.

9. A framework for improving the perceptual robustness of multimodal large language models through adversarial self-game, characterized in that, The framework includes an attacker model and a multimodal defender model, both of which achieve co-evolution of capabilities through iterative training. The attacker model is an attacker model with image editing capabilities.

10. A method for generating an initial adversarial dataset for initiating adversarial self-games, characterized in that, The method includes: The scenario expansion phase includes: Images are selected from the source dataset, and image rendering techniques are used to expand the image canvas, increasing visual complexity and the space available for adversarial manipulation; and Perform a triple check on the expanded image to retain the valid expanded image; The adversarial implantation phase of semantic disruptors includes: For the filtered expanded images, semantic distractor proposals are generated based on the original question and the multimodal large language model. The semantic distractor proposals include the location bounding box and text description of the object to be implanted. Double-check semantic interference proposals to ensure they do not undermine the validity of the original problem; and Double-checked semantic distractor proposals are input into a text-guided image infilling model, which then generates and naturally blends the corresponding distractor objects within a specified bounding box region, thereby generating candidate adversarial images; and The validity verification of the candidate adversarial images based on the original problem includes: The candidate adversarial image is input along with the corresponding original question into the initial defender model to verify whether the candidate adversarial image can successfully mislead the initial defender model; and Only effective adversarial images that can cause the model to give an incorrect answer to the original question are selected, and together with their corresponding extended images and the original question, they form an extended image-original question-effective adversarial image triplet to form the initial adversarial dataset; and / or The triple check includes: The combination check includes using a general multimodal large language model to determine whether the expanded image is a coherent scene rather than a splicing of multiple unrelated scenes, so as to ensure that the answer logic of the original question is not affected. Repeat checks, configured to ensure that the image extrapolation process does not introduce new instances of the target object relevant to the original question, in order to maintain the validity of the original question-answer pair; and Authenticity checks include filtering out expanded images containing obvious AI-generated traces or illogical content to ensure the realism of the original question's answer scenario; and / or The dual check includes: Location bounding box overlap checking is configured to ensure that the bounding boxes of semantic distractor proposals do not overlap with key target regions in the original image that are relevant to the original problem, thus avoiding direct occlusion or replacement of key information; and Text description checks are configured to ensure that the text descriptions of semantic interference proposals do not contain key target keywords of the original problem, preventing the generation of similar objects that alter the premise of the problem.

11. A method for training an attacker model, characterized in that, An optimization algorithm based on reinforcement learning is used, employing a composite reward function to simultaneously optimize the effectiveness of adversarial attacks and the preservation of scene semantic integrity. The composite reward function includes an effectiveness reward and a semantic integrity reward. The effectiveness reward is determined based on whether the adversarial image successfully misleads the previous generation defender model. The semantic integrity reward uses an image semantic consistency metric to ensure high similarity between the edited and unedited regions of key objects. An attack is considered effective only when the image regions corresponding to the key object remain highly similar before and after editing, preventing attackers from winning by deleting or severely damaging key objects; and / or The image semantic consistency measurement tool includes an image similarity metric / loss function used to measure the semantic integrity of images.

12. A dynamic training course generation mechanism, characterized in that, Each generation of evolved attackers generates new and more challenging adversarial images for the next generation of defenders. The generation process includes a course selection mechanism to select the most informative training samples for the defenders, forming a dynamic training course.

Images